Money seems to be a big factor in drop outs, when clubs are charging £120 for a wrecked PA28 that will just about 2 people off the ground it becomes expensive. It seems shares are the way to go to find yourself a decent aircraft, you know the history of and you dont have to keep looking out of the window to check the wings are still attached.
The cost/requirements of keeping rating current i.e. ME/IR puts people off progressing beyond the PPL.
Another reason I have heard a few people say (and this probably backs up what some on here are saying about more organised activities), but getting a PPL for like getting a driving licence. Once you get it its a novelty and you fly around in circles a bit, after that in becomes like repetively driving round the M25 and you lose interest. If you aren't actually going anywhere for a reason there is no point spending the money, same as you wouldnt just take your car out - there is just no point boring holes in the sky - unless you are really bored that is
I have to say I have never heard anyone give up because they werent confindent with their flying abilities, I knew one guy who dropped out part way through my PPL course for this reason as he put plane into a spin which instructor recovered from I have subsequently heard that he got his confidence back though and now has his PPL.