The RA has to be followed because the traffic you see may not be the one that is generating the RA and it is unsafe to delay the reaction while trying to figure out exactly what is happening.
Which was exactly the situation which caused the San Diego mid air, which was one of the main drivers behind TCAS / ACAS.