Perhaps they did see land on the radar, and it didn't raise any questions because if they thought they were at Blacksod they would be expecting to see land on the radar.
In the cold light of day I'm sure we could discuss at length the shape of the coast etc, but at that time of the morning they may not have been quite so assiduous.
Edited to add: as stated in the post above, the CVR will hopefully shed some light.