Any wing will develope more drag once the aircraft is deccelerated below Vimd (Velocity Indicated Minimum Drag). It's just more pronounced on a swept wing aircraft and particularly noticeable on a delta. The 'overspill' of the higher pressure air under the wing starts at the root on a swept wing creating a large vortex above the wing which generates masses of drag. However, this vortex (also known as the ram's horn vortex after it's shape) can be beneficial. The accelerated air in the vortex creates a lower pressure than would have normally been present and this extra lift allows the aircraft to fly slower. Deltas don't have flaps, but this vortex helps nicely. The only drawback is that you need masses of power to accelerate form such a slow speed.
Speaking from experience, the Victor Mk2 could reach FL500. The difference between the slow speed and high speed (Mach) buffet was only a few knots. The Victor's surplus of power allowed it to operate in this regeime.