Chuks
Study the composition of the NTSB - that is the BOARD MEMBERS - and you'll find that ultimately, although they rely on specialists to make the investigation, the actual board members who have the final say are, in fact, "amateurs" - and often bring an outside perspective to the final report.
edit: took some finding...
Current Members of the Board are:
* Mark V. Rosenker, Chairman
* Robert L. Sumwalt , Vice Chairman
* Deborah A. P. Hersman, Member
* Kathryn O'Leary Higgins, Member
* Steven R. Chealander, Member
Links to their bios are
here None are "trained accident investigators" by any stretch of the imagination (two were pilots, but that's not the same thing, obviously)
As to "why is aviation picked on" - it isn't. Ask any, say, nuclear industry employee if he thinks they are immune to "armchair experts". Or the military in general. Or the police forces. Or ....