This extract from Norman Harpur’s 1966 paper on “The Structure of the Concorde” explains it pretty well I think. Norman was the chief structures engineer on the British side so he can be classed as someone with definitive knowledge.
“At Mach numbers of about 2 it pays to paint the external surface white. Despite what the textbooks say, a white surface can be made almost as good as a black surface at radiating heat away from itself whereas it is much better than a black surface in reflecting solar radiation. Under these circumstances, at these speeds, a white surface will result in a cooling of something like 10 deg C. If we increase the speed of the aircraft, up to say a Mach number of 3, far more heat is transmitted by skin friction and the effect of solar radiation is relatively small. In these conditions it now becomes important to have the highest possible emissivity on the surface to reduce the heat as much as possible and here, even though relatively small, the gain between white paint and black paint is important. Therefore the Mach3 supersonic transport should really be painted black.