It's not possible to answer without knowing aircraft details, altitude, etc, and thus possibly get an idea of the pilot's capability and actual flight conditions.
If you posted an mp3 of the radio exchange, it might be revealing
The UK has a negligible history of ATC helping pilots with weather. It's just not a part of the system - unlike the USA where they have the established PIREP system.
Personally, I would obviously not disregard an ATCO's concern but I would also explain to him what my plan was to get back down. The least I would say is e.g. vis up here is 50-100nm, blue skies, etc.