I'd have thought the aircraft already in the circuit, because they're already there.
I suppose in theory if you were joining a right hand circuit, then you could argue that you had right of way on any aircraft to your left on downwind already in the circuit, but that doesn't scan for me. As someone joining an active circuit, I'd still make way.
If there was plenty of scope for me to join without upsetting a craft in the circuit e.g. said craft just turned onto downwind and was still aways away, then I would.
Also, if I was entering an active busy circuit, I'd probably bore the socks off everyone with my constant position reporting!