Never had any real problems transiting mil airspace on the east coast, even when its busy. Its usually easier if you let them know well in advance.
I have seen many near misses in mil flying; Two Macchis on take off passed about 100 feet below me at Nowra (Husbands DZ) just after exiting the para door of a Herc (parachuting), that ended in a very bad feet apart landing.
Also, a pair of F-111s passed scarily close at scary speed and full A/B as our 'Bou entered the zone at Amberley in 2001. I have heard of some close calls in Pearce airspace, so the danger is there.
Some controllers will let you through, some will not in the same circumstances. I guess some are more competent than others. Not much you can do about it unfortunately