All sounds a bit strange beeps, but sadly not completely surprising. There are a whole bunch of questions that come to mind from what you describe. The ones that are important to you are...
Did the escort driver brief you at all about what they would do?
Did your internal escort say anything to you at any time?
Why did you think that you were left in a vulnerable position? (This may help you to argue that it was safer to move than to stay where you were).
What was between you and the security gate?
What are the rules for escorting a vehicle?
Overall, if you were not briefed by your escort (either driver or internal) and you otherwise did what you were told it would seem that the aerodrome operator let you down badly.
Another question - when you say 'they' are trying to blame you, who is they? People on the Security Gate or someone else?
If it helps, in general terms, a vehicle driver in your position should be briefed about what the escort driver will do and what you must do - both if everything goes well and if there's a problem (if you break down for example). If you are not familiar with the airside environment you should never be left unsupervised - hence the question about what your internal escort said or did - if she couldn't tell you what to do you shouldn't have been left alone.
I'm not trying to suggest I don't believe you - but we've only got your side of the story. Anything like this should be properly investigated to get everybody's perspective on events - a professional approach from the aerodrome would be to try and find out how this happened rather than to simply try to blame you for breaking the rules. Depending on what happened at the time and has happened since I wouldn't be inclined to tell the CAA unless you are absolutely in the right and you have exhausted other avenues of pursuit.
Sorry for a bunch of random thoughts but time is a bit tight for me right now but drop me a PM with a bit more info if you want and I'll see if I can come up with anything more specific.