According to Quentin Smith ("Q"), after his ditching near Antarctica, he was able to swim 100m to his life raft in near freezing water, with his survival suit flooded (and just barely made it). So swimming 50m in similar conditions is not impossible.
https://www.youtube.com/watch?v=2sDIFT6ZqN4 (skip to around 13:30 for Q's ditching story)