As someone who deeply loves magic, be it in fantasy, or Science Fiction, I have always been excited by different systems and the ways in which they are integrated within the story. Brandon Sanderson — a huge inspiration on various axes — has an entire theory about how magic should be used. I greatly recommend reading his First Law article, which sets down a strong correlation between magic systems and plotting. The entire thing is worth the time, but the law itself reads:
An author’s ability to solve conflict with magic is DIRECTLY PROPORTIONAL to how well the reader understands said magic.
On the surface of it, this makes a ton of sense — the less explicable your supernatural forces are, the more unsatisfying it is when they resolve a problem. Gandalf can wave his hands and divert storms. He can win willpower battles against gods and ancient spirits. But do we understand what the parameters and limitations of his magic are? We do not. Would we like it if, after three books of toil and strife, he shows up and just solves all problems? We would not. We are barely ok with the Gandalf Ex Machina already present in the trilogy!
But magic is not binary. Sure, prophecies, totemic and symbolic powers, and destiny are lazy ways to solve a problem. But they are certainly awesome at creating one. A world can be governed by laws far different from the reality we inhabit, and its heroes can still fight against those different laws. Magic doesn’t need to be systematically defined, in order to be part of a satisfying story. It just can’t solve that story’s main problems.
However, I think those two ways of approaching supernatural powers aren’t mutually exclusive. “Soft” magic can be your inciting incident. It can fully govern the world of your story and create all its problems. “Hard” magic can save the day. Because it is bound by rules and limitations which the reader understands, it is perceived as just another skill in the characters’ toolbox, and as such, it does not break the ability of the story to put them through the grinder.
Of course, this is all relative. Rules exist to be broken. Your story can be about the inevitability of failure in the face of overwhelming powers. Or it can be about a godlike being and their awesome abilities that we understand fully, but which no one can stand against. Or your “soft” magic can be a higher form of a hard system that you have just not revealed to the reader yet.
Still, to break a rule, you have to understand it. And I believe that Sanderson’s First Law is a unique attempt at discovering how the tropes of speculative fiction actually apply to literary structures. Magic is a great tool we have. But unless you are writing a role-playing system, or do not care about structure in your story, you have to use that tool in a way that enhances your work, rather than breaks it.