I think this contribution is the point - Oxenos has put the finger on the nub:
Originally Posted by
oxenos
The danger of ignoring an RA is that you assume that the aircraft you can see is the one which triggered the TCAS.
The TCAS may be warning of another aircraft which you have not seen.