I think the answer is - "it depends". Every aircraft is different. To give a few examples:
The Harrier is generally LL speed limited by drag - that huge fan which is designed to produce a lot of thrust at low (no) speed is not so great at 450 kts.
A Tornado with, say, external fuel tanks is LL speed limited by an aircraft limit - the tanks can't go as fast as the aircraft or they drop off. Drag is definitely not the limit!
Most FJs accelerate faster the faster you go (basic jet engine theory) but above about M0.9, transsonic (or whatever they call it these days)drag becomes a factor - perhaps to the extent that the aircraft can't get above M1.0.
Also, as mentioned above, engine design is a factor. The Tornado F3 has moving intake ramps, which operate at about M1.3. Acceleration is noticeably better at M1.4 than it is at M1.2
Hope that helps.