Smaller drones represent an insignificant risk to large aircraft like an A320, 64,500 Kg versus 1 Kg is unlikely to end in favour of the drone. The larger 5Kg and upwards semi professional models that are capable of flights to several thousand feet are clearly a potential hazard.
Perhaps larger drones should be fitted with a mandatory TCAS receiver that causes the drone to immediately land in event of a conflict with an aircraft. A permanent serial number burnt into the computer chips or a system to permanently disable the drone operating system if it is operated in conflict with aircraft could be useful. Something along the lines of disabling stolen mobile phones.
Culprits then either lose the use of their drone permanently, or get brought before the courts to give reasons for flying the drone in potentially dangerous circumstances.
Given how cheap complex electronic devices have become, it shouldn't be hard to devise and implement such systems without compromising drone performance. Modern drones are supposed to have geo fencing built in to prevent flight near airports in any case.
So we are left with legacy drones that can still be flown anywhere. No real solution for these although their numbers should decrease over coming years.
As far as the risk of these things being used as some sort of very slow guided missle, radio controlled planes with far greater performance and payload capacity have been flying for years without similar concerns being raised. Why?