Landing aircraft have priority. However, if he declared "finals" and then immediately reckoned that he had to go around because you entered the runway , then it would appear that his "finals" call was far too late. If he had called finals in good time on a proper full circuit, then you should not have created a problem for him. It is hardly reasonable to criticise someone who lines up having cleared the conventional approach path visually, only to find another aircraft making making some sort of egg shaped swooping circuit and only calling finals when almost at the theshold.