Perhaps the flight attendants are trained in unobtrusive testing; if you argue with them about reading the safety card, you are probably not the best person to be seated where the safety of hundreds of people depends upon your ability and willingness to follow instructions. If the FA says not to open the door because the flames are on that side, you don't want somebody who is going to open it anyway.