I know what I'd have done.
I wouldn't argue. I'd just smile sweetly and then I would've turned around, gone straight to the coffee shop, ordered a moccachino, picked up the mobile phone and informed crew control that security had refused me access to the aircraft, and would they please call the reserve. Or sort out the security guy.
I've used this little tweak a few times already, in various annoying scenarios, and its amazing what gets done when it becomes THEIR problem instead of YOURS.