Sorry, but you've shot yourself slightly in the foot here. You should ALWAYS keep a good lookout, just because you are on final approach doesn't mean you shouldn't look out incase the situation you are complaining about happens.
We all have a responsibility to try and lookout as well as we can, it is by no means perfect (as FD has shown) but combine good lookout with listening on the radio properly (obviously as long as other a/c are radio equipped) and this sort of problem should be minimised as much as is humanly possible.
Right of way is one thing if both aircraft know where the other is, but I wouldn't bet my life on the other guy having seen me just to make a point.