IRPilot,
A question, for you... you're probably right from what has been discussed on here, perhaps an incorrect SID issued, but what would you have done if you'd gone RT fail transferring between TWR and Radar?
As for the ATCO on radar, i suspect they probably noticed you on a heading rather than the SID and just got on with it, thankful that nothing untoward happened. Apportioning blame is pretty pointless but I would be a bit bemused if an aircraft helped itself to a heading!! Perhaps some questioning would have been appropriate but as other threads go on about here, most agree that it's unprofessional to criticise fellow professionals on the RT (myself included).
Ultimately a good learning point (and seemingly yet another missed readback in there somewhere).
WW