The question is, can you ban them?
All you need is 4 motors, 4 propellers, a raspberry pi with a GPS and a few other off-the-shelf bits and bobs, and off you go.
Without banning radio controlled cars and STEM education I don't see any practical way of outlawing them. Besides, they potentially have lots of legitimate uses. You might be able to stop a Maplins hooligan, but anybody truly intent on malice will still be able to do whatever they want. You would affect literally millions of hobbyists worldwide, but without really solving the problems.
The potential for small drones to be used in military conflict means that I think we'll soon need to sort out some robust defences against them. Any reasonably large nation could field a million drones each capable of carrying half a kilo of explosive a thousand miles. Just keeping them out of the hands of civilians isn't going to be enough:
BBC NEWS | Europe | Model plane goes transatlantic