tail ballast was changed when altering radically from inline to radial, which was rare.
I have changed/added tail ballast for several engine changes I have approved, to assure that the airplane C of G remained within the required limits, and added aerodynamic changes a few times to improve handling within the C of G range, following engine changes. In all cases, handling was affected, though within what was acceptable for the type. None were fighter/air combat types, I'm just getting civilians from A to B, albeit in changed airplanes. These changes did not affect the rate of turn, though did affect control feel during maneuvering.
Yes I do think something that gigantic could easily be hiding in plain sight of decades of air show flying, since that is all they do.
If gigantic forces associated with propeller vs wing loading forces were hiding, or cancelling each other out in flight, surely there would be correspondingly different handling and performance during power off maneuvering, as one of these sources of force was dramatically reduced. Though power off maneuvering usually involves trading off altitude, I have otherwise found it to be within the handling expectations for the type. To offer an example that propeller powered airplanes with a decent power to weight ratio can be very nicely flown in maneuvering power on, or power off, I would refer you to Bob Hoover's demonstrations in the Shrike. He seemed to suffer nearly no handling defect power off through some very impressive maneuvering and turns.....