Hi Mike.
The interesting difference between cargo and pax ops is that cargo/mail ulds and items are specifically weighed while pax and most baggage use notional weight values, so you could say that cargo type errors are occasional while notional issues are systematic. However, there are plenty more aspects that make me agree with your view.
Of course CG limits are set by the manufacturer with that and many other things in mind, but I'm talking about the times when all the fat tourists are in the back of the plane while the jockey club fills the front, 50 suit carriers of business class baggage in the front hold and the 200 bulging y/c bags in the back. The CG is already at its aft limit (according to notional weights for pax and bags), and then someone mistakingly loads 1000kg of undocumented cargo in hold 5 while the load controller goofs the stab fuel because he wasn't trained for that aircraft type or wgt/bal computer system.
We all know that accidents happen when enough things coincide and conspire to produce a chain of events. It is also true that most ops in most airlines get it right most of the time and when they do make mistakes they are spotted and corrected most of the time. This still leaves other layers of protection like the resilience of the aircraft design to such abuse and ultimately to the flight crew that operate it.
Remember also that the crew rely on the CG location to set the stab trim and weight for performance data, so the actual CG location does not have to be outside of the envelope for the pilots to experience control problems if they are given the wrong location (Fine Air DC8) or the weight to above limits to have perormance issues (SQ tailscrape).
What I'm trying to research is the performance of load control and what sort of consequences it can have from the pilots' perspectives. Having worked in load control for a few years, I know that mistakes do happen and the scenario I describe above is one I have seen. The frustrating part is that we never got much feedback from flight crews, not even for that one, while the training did not deal with it at all. I've already talked to a number of pilots about this over the years, but every pilot has a different and usually enlightening perspective, so the bigger the sample the clearer the overall picture.