Indeed, the signature can be damaging (significant or minor)/distressing/benign/non-existent (at ground level) depending on a bunch of parameters relating to atmospherics, height, and manoeuvring.
As a friendly balance to your Concorde observation, when the initial proving flight to Australia was conducted (as I recall around 1971 ?), the doublet (which was recorded by a DCA team enjoying the Central Australian sunshine at the time) as the bird came over at cruise was a benign pop-pop sort of audio thing.
However, I have a variety of mates who are ex-MIL FJ and they have tales by the legion relating to broken windows etc., etc. Hence the regulatory problems which beset Concorde's general commercial success with limitations restricting supersonic flight to overwater and some overland areas.
We all have experienced similar and higher overpressure levels when we hear the crack of the shock wave associated with thunderstorm lightning strikes ... well do I recall one in Jo'burg when I was working there some years ago .. and the bolt strike blew off a goodly section of the roof on the adjoining motel room ... while I was idly lazing about at the door looking at the rain outside ... now, that got my attention ... not to mention causing a degree of tinnitus for the next day or so.
I guess that, when I relax a bit and ease out of the politically correct, I am a lover of Speys, early JT8Ds, F111s, F16s and the like .... if it's not noisy ... what use is it ?