"Anyway, between myself, Rainboe, and numerous other Captains on here, we probably have over a million hours of experience compared to a few hundred hours that "test pilots" have."
This may have been a tounge-in-cheek comment, but there have been incidents where airline pilots have done experimental flying with a full load of passengers with bad results. When you deviate from the Flight Manual you are a test pilot without a test plan.
It may not be widely known, but the first 737-100 in the Boeing test program was fitted with bail-out provisions. The Fwd baggage door was hinged on the leading edge and had a hydraulic actuator that could push it open against the airstream. A section of floor panel was removed above it and a grab handle was installed above the hole so that crew members could hang from the handle so that gravity would align them with the open floor and cargo door, g forces permitting. It was never used, but there were a couple of anomalies.
The 737 leading edge devices did not have locks on them initialy because wind tunnel data showed that aero forces would keep them tight against the wing in flight. Boeing has thousands of engineers with every applicable specialty that test and analyze every possibility prior to test flights. During one of the high speed tests the leading edge devices popped out and were badly damaged. The crew regained control despite the resulting asymetric leading edge, and managed a flaps up landing at BFI.
Another anomally was that the original TR's actually caused the 737 to speed up when they were applied. Both TR doors deflected thrust under the wing which created lift that reduced braking effectiveness greater than the reverse thrust forces. The fix was to clock them so that one TR door deflects thrust above the wing.
One in service "test flight" that went badly was a 727 flown about 35 years ago by a fellow named Hoot Gibson. Hoot decided that he could deploy a litte flaps at cruise to achieve a higher altitude. The 27's did have locks on the leading edge devices, but guess what? Hoot just unlocked them! The result was much the same as the 37 real test flight. Hoot took a 30,000 foot spiral dive. Initial press reports extolled his hero flying skills, but when the facts were known he was unemployed.
There is good reson to do it by the book.