Conscious of my non-pilot status I did read the ANO hence my above response; he doesn't appear to have broken any rules - and I can't find any that require permission from anybody to be overhead any airfield.
You still persist in your allegations here instead of phoning him, as already suggested, and as you had already agreed to do.
Presumably then this is a personality conflict that you have with the man - maybe it was one of those that got you booted out of Sywell?