I'm VFR and the controller is responsible for terrain separation in class D Airspace because it is a radar control service (from a pilot with a lot of hours flying a scheduled service VFR in the daytime I kid you not).
Not quite sure where you're coming from there TS. From the AIP ENR section:
6.2 Controllers will ensure that levels assigned to IFR flights when in receipt of a Radar Control Service and to flights in receipt
of a RAS will provide at least the minimum terrain clearances given below:.....
6.2.3 Radar Controllers have no responsibility for the terrain clearance of, and will not assign levels to, aircraft in receipt of a RIS or aircraft operating Special VFR or VFR which accept radar vectors.
At my unit, VFR clearances are along the lines of "Not above 1500ft VFR/SVFR." No terrain clearance built in there!!