Tacpot as Bose-x says you've got it right from a practical point of view. Without an IMC or IR you can't fly IFR in any controlled airspace therefore SVFR is the only option, even with an IMC you can't fly IFR in Class A. You can fly IFR outside controlled airspace and SVFR is not available. Clear?
