I have to say I have also experienced such things. I was spiked with some rather potent LSD aged 15 and think it was the sense of completely losing control of everything around me that triggered it. I hyperventilated until I passed out, in the middle of a city centre on a Saturday lunchtime.
They are extremely unpleasant events. I know a local nurse, lovely little thing, who had a panic attack herself and ended up in A&E; despite having seen people have them herself she was that scared she didn't recognise it for what it was and literally thought she was about to meet her maker.
Don't let it dent your confidence. You just need to tune your mind to the point where you control it, instead of letting it control you. Personally I found that gradually pushing the limits of my comfort zone (motorcycle racing, aerobatics etc..) was great therapy. Obviously I'm no psychologist, but it worked for me. Anxiety is best described as "the fear of fear", and you can't let it beat you. Unspecified worrying about things that will almost certainly never happen is something you need to stamp on, however how you do that is a matter very personal to the individual.
I find these days that flying with other pilots and pushing one's limits is a great mutual confidence booster. I also notice you're in Manchester, so if you ever fancy getting some airtime with a fellow freak, just drop me a PM. I have a Cardinal based at Leeds, several hundred hours of experience and am always looking for an excuse to empty my bank account.
Best of luck with it either way.