Did you pause long enough to listen for circuit traffic?
As long as he didn't call finals until after you've entered the runway, I'd say he's got no right to complain.
"Making someone go around" is not really possible unless your at the controls of their aircraft.
The decision to go around was entirely his and it was the correct one but that's what flying at A/G aerodromes is like. He should get used to it.
Don't beat yourself up over it.