I haven't read the books either but I do remember sitting on a airfield in 1968, listening to two Boscome Down test pilots discussing it (anything was better than trying to teach me to fly).
They said that the reason that it was scrapped was that it would not do what it was meant to do - fly to Moscow supersonic at low level and return. It could not carry enough fuel.
The same thing applied to the F111 which is why we didn't buy that either.
They could both do the trip just subsonic but the Buccaneer could do that better than either of them. This was why the Buccaneer was transferred from the Navy to the RAF.
As I said, this was the informed opinion of some current RAF test pilots at the time. I do remember reading a novel at about this time which seemed to be based on the TSR2 and, to stop the Soviet's cunning plan to takeover the West, the Brits had to demonstrate that they could deliver a bomb into Red Square. They used the TSR2 knowing it could not get back, and then quickly pick up the crew from a crash landing in the North Sea to show that they had returned. Can't remember the author bu it might have been James Beatty.