I have only ever been involved with one job that needed a surface swimmer. We only decided to use him to recover survivors from a liferaft in a V rough sea state, which was unfortunately empty anyway. After the empty liferaft was sunk, the surface swimmer proved helpful to recover the two bodies that should have been in the liferaft.
I think this also could have been carried out safely, (maybe not as efficiently), just using a winchman connected to the wire though.
This leaves the question of: "do we need divers/swimmers on watch constantly, or just a standby available elsewhere"?