I am a toughie but some threats are enough to keep you awake at night.
Did you take along someone (a witness) with you to this face to face meeting?
Maybe time to involve local MP. Or even one of the 'excrement stirring' TV shows masquerading as a Current Affairs show?
Like my late Father (Upon retirement a senior Law Clerk) said on occasion;
"Don't get mad, get even. With interest!"
Just sayin'