whether an airport ATC tower can deny access of a VFR traffic because of lower than VFR conditions exist?
Definitely if the airport is in CAS.
In the UK, each such airport has VFR minima in its operating rules. Generally 1500ft cloudbase, etc.
This can be a huge hassle if you are in say France and have no IR. 1200ft cloudbase but you are not departing. I was stuck at Biarritz like that for several days.