I know exactly what you mean. I flew from Blackbushe to Compton Abbas yesterday and it was really rough. I didn't get above 3500'; you could see the inversion layer clearly as a black smudge. Had a couple of interesting moments on the approach coming in over the trees too !
Looking back I'm really glad I went as it's all good experience but at the time I was wishing I was back on the ground !.. then I got the £300 bill just to top it off.