Mode S does not help in this case because the initial sweep (which generates a list of targets to interrogate selectively on subsequent sweeps) will see a garbled return.
I don't think that's the case. During the All-Call period, Mode S transponders can be told to reply with a certain probability. On some interrogations, one aircraft will reply while the other is silent. Once the reply is received, the interrogator instructs that transponder not to reply to All-Call interrogations, leaving only the other aircraft replying, sometimes, to the All-Call. Over a number of cycles, all the aircraft will be able to reply, and subsequently will be included in the selective Roll-Call.
Mode S radars should have no problem at all with Mode S transponders in formation.