The London TMA is an old design which hasn't changed much since everythign revolved around Heathrow. The airspace hasn't changed to accommodate the relatively massive increase in traffic from Stansted, Luton, Gatwick, Birmingham, Northolt, London City, Southend, Oxford, Cranfield etc.....
I also disagree with the 3nm argument to a certain degree. Yes, when dispensing with vertical separation when two aircraft are 3nm apart, it is more often than not necessary to have them on headings in the UK. That's how we are trained. If someone makes a habit of not using headings on aircraft which are 3nm apart, they could very well lose their validation. This is a given but as has been said, headings are often used when there are no aircraft apparently in the vicinity. (Remember though, TCAS is rubbish in the horizontal plane and is not a substitute for radar). The reason for this is positioning.
From a personal perspective, most aircraft I work are placed on headings for positioning to the next sector to prevent conflictions further up the line, and not for separation. For example, all the northbound traffic from the TMA is is usually ordered geographically for it's desitnation within 30nm of departure.
Take for example a Heathrow WOBUN departure going to Aberdeen and a Stansted BUZAD departure going to Belfast. A good controller will endeavour to position the Heathrow departure to the east of the Stansted departure before around FL140. That requires "harsh" headings which take the aircraft quite a bit from the SID route but it solves many conflictions further up the line and makes things easier for the subsequent sectors. It also makes giving continuous climb easier.
This is the result of ancient airspace design. A new design is on the drawing board but it is currently posing a lot of problems and it may well be some time before it is implemented. In the meantime we have to suck it up and do our best with what we have.
We don't take you off the SID to make life harder for the flight crew, we do it to provide the best service we can to all aircraft in our airspace.