What time are you starting/finishing shift?
Used to do J4 M3 to LGW. I never had a problem getting to work (always got there way to early) and never a problem getting back after a day shift. Getting back after a night shift though, forget it on the M25, especially now there are the roadworks from the M3.
Depending on where you are there are a couple of back ways that should avoid most of the trouble. We have quite a few LHR staff that commute from LGW but can't say I envy them.