That is because it has to provide protection for its IFR traffic and it can't do that if VFR traffic isn't doing what it is told. You don't know what the clearance of the IFR traffic is and where it is going but ATC does - that is why they will control you to ensure safe separation.
Crab i refer you to the correct answer given by Gegg concerning Class D airspace.
It is damn rude and condescending to presume VFR traffic cannot separate itself from IFR traffic especially in good VMC. Even though sometimes there are grounds for worryiing.
The phrase "I have your Galaxy downwind in sight" is all the controller needs to know regardless of how indespensible they feel. ( own navigation, report when you have the xyz trafic in sight etc). Advise turn left 30 degrees to avoid conflict with Kyrgzstani Cessna 152 on 3 mile final. Please arrange your flight to pass behind that traffic. Advise when East of RW 13.
The problems with being non standard in this matter are potentially serious.
1 IFR aircraft in the downwind does not a busy airfield make.