EESDL
You're quite right, the reasons for the PanAm building crash had nothing to do with operating to rooftops. The point is not
why it happened, but
where it happened. Incidents may have different outcomes depending on where they occur. The 777 that landed short of Heathrow runway 27L in 2008 made it onto the grass and there were no fatalities (
G-YMMM, 17 January 2008). They might easily have ended up on the busy road just outside the fence. Same cause (fuel starvation), very different potential outcome.
A recent helicopter example: a S-92 lost yaw control while landing at a North Sea platform (
G-WNSR, 28 December 2016). The crew very skilfully managed to put it down on the edge of the helideck, with a few feet to spare (look at the photo in the report) – no damage, everyone walked away, but the pilots seem to have been understandably discomposed (report section 1.5.3). The cause of that accident also had nothing to do with where it occurred. But now let's do a thought experiment: imagine a similar mechanical failure occurring in various surroundings, and suppose that it was such that despite their skill the pilots were unable to put the aircraft down where they did, but ended up those few feet farther out:
(1) an airport or similar open space: they land outside the landing area, but very likely safely and with minimal damage, depending of course how far away they were from the H;
(2) the helideck where the accident occurred: they fall off the edge into the sea, or if they're farther out, maybe they manage a more or less controlled ditching; either way, a safety boat is on hand and nobody outside the helicopter is endangered;
(3) a rooftop helipad on a tower in a city centre: they fall off into the street below, with the consequences you can imagine.
I think scenario (3) is what worries people. Of course helicopters have operated to city centre tower helipads surrounded by streets and other buildings, probably thousands of times. Helicopters have certainly operated thousands of times to offshore platform helidecks. I don't know how many accidents have occurred during those helipad / helideck landings and takeoffs (helicopters have of course crashed with serious loss of life in other phases of offshore operations), but let's suppose that there haven't been any serious outcomes apart from that PanAm crash. Even so, people see that helicopters do crash from time to time during landing or takeoff, and of course worry about what would happen if such incidents did occur in a city centre rather than somewhere else.
I'm referring here to the OP's question "How come none of these high-rise buildings have an HLS on them?" Battersea and similar sites are IMHO a different issue.