Fuji,
I agree with you and would feel much safer in the aircraft of a high hour ppl with aerobatic quals rather than a low to medium hours pilot with IR or IMC.
I have also noted on a personal level that those with their own aircraft can also become complacent ie: "nobody knows my plane better" and other cliches you will have undoubtably heard before.
I am not suggesting that you r average pilot with say a thousand hours including an IR in his own plane is unsafe, but they do tend to have a certain confidence and cockieness that may catch them out one day.
I also know of a few glider pilots who regard themselves as far safer than any motorised pilot due to their better understanding of weather and blah blah blah.
Personally I get at least one flight a year with a cfi just to brush up and highlight any bad habits that may be forming.