To pick up a new weapon, click on it. Sometimes you have to play hunt-the-pixel a little bit depending on where it's laying on the ground. There's no way to drop something you have without picking up something else though. For the knockdowns: are you using the personal shield a lot? Just the shield + self cast thing that makes you glow? Because that thing is terrible for anything that does physical damage. You get knocked down all the time. I thought those goblin archers were the worst thing ever until I turned off my shield and they stopped playing ping-pong with me.
If you're having any trouble killing anything, try the combination QFASAA. It combines steam, which gets them wet, with lightning, which does double damage to wet targets, and arcane, so you get a beam out of it. It also does massive damage as an AOE. You hardly need to use anything else. Also that healing staff is just a silly thing, and the amount of healing that it gives to enemies will be drastically outpaced by the damage you can output, especially with this combination.
I am also annoyed that there's not really a good test bed to experiment with magic. There are a couple times Vlad shows up to talk to you and he's invincible. Try zapping him for testing!