Mostly it's a muscle memory thing. If the menu to get to light, light snake is always one left and one up, for example, I can know exactly how to cast it with a gesture and it takes me half a second to do it, even while on the run. In fact, being able to assign spells to mouse gestures would be awesome all by itself. I can't live without the mouse gestures addon for firefox, though, so maybe that's just me.
If I'm dodging shadow bats and I have to move my hands off the movement keys to get to inventory, find the spell in the 4 or 5 line high stack, move my aiming reticule off the bat to get to the spell, cast it, then again hit inventory to close it, it's a bit of trouble. I know, I'm overstating my case a little bit here. But it is still a little inconvenient, especially when you're trying to focus on dodging in circles around a cave.