It's not at all obvious that a passenger refusing to move to free up a space committing any offence,
Completely in your mind.
It is an OFFENCE to not follow cabin crew instructions. I have off loaded a passenger in Australia for smoking an E cigarette who was slow to desist when asked by cabin crew. I do not have to tolerate someone on my aircraft who will not follow instructions from cabin crew.
So in the US, he failed to follow their instructions so the police and/or security were called. Pretty clear breach of aviation law.
As to whether he could be removed, what do the terms of carriage say. I can only speak for Australia and the UK where the airlines guarantees to provide NOTHING in return for your booking.