Good points, anotherthing.
I concede you would never know to a certainty that the aircraft you were visually avoiding is the one triggering the RA in a high density environment. You can't either be sure to a certainty that the other aircraft is complying with his RA against you.
Let me be clear that I don't advocate not complying with the RA vertical command first and foremost to any other action. Once you do, though, unless you are IMC, a set of eyeballs, assuming a two-place aircraft, should be back outside looking once RA compliance is assured.
Your point about turning into other traffic is valid, too. But I'm not suggesting a sustained course alteration but a turning maneuver of short duration; one just adequate to quell the conflict, which I wouldn't think would displace course by more than a mile at most, and would not, I hope, be likely to cause loss of traffic separation if course is then quickly resumed.
I've only had a few RAs, the last one on approach to Newark 22L last year while being line checked. The trigger was a fast climbing bizjet out of Teterboro whose rate of climb on departure broke into the RA equation even though he leveled off 1,000' below as cleared. Never saw him, but I was definitely looking for him once the VSI was in the green. The check airman didn't seem to mind.
I didn't know for sure the other guy was going to level off. But I damn sure would have turned if I'd spotted him coming at me.