Gertrude,
Yup, you're correct. It's is one of the anomolies of that has yet to be ironed out. Your basic PPL states that outside controlled airspace you must remain in 3k vis, clear of cloud and in sight of the surface. Nowhere in the ANO does it state you must remain
VMC or VFR.
ANO Schedule 8.1(c)
He shall not, unless his licence includes an instrument rating (aeroplane) or an instrument meteorological conditions rating (aeroplanes), fly as pilot in command of such an aeroplane:
(i) on a flight outside controlled airspace when the flight visibility is less than 3km;
(ii) on a special VFR flight in a control zone in a flight visibility of less than 10 km except on a route or in an aerodrome traffic zone notified for the purpose of this sub-paragraph; or
(iii) out of sight of the surface.
So, your in Class G at 4000 feet with 4 km vis, but your clear of cloud and in sight of the surface (one real hot and hazy summers day), the conditions are below VMC, but as a basic PPL you are flying legally as you satisfy the criteria of licence issue. Legal yes, sensible methinks no.