But at 500ft AGL from an elevation of 2700ft you would be at 3200 ft AMSL over a non populous area, and according to Annex 2, above 3000ft AMSL but below 10000ft AMSL, you need to be 1000ft vertically from cloud. If you operated at 500ft AGL (3200ft AMSL) you would be 800ft vertically from cloud (where the base is 4000ft) and therefore busting minima. So I would still say the answer is No you cannot fly.
Once you climb through 3000ft AMSL you have to comply with the higher minima for cloudbase, because operating 1000ft vertically from cloud would put you only at 300 ft AGL and below the low flying limit.