Part 91 ops can be every bit as safe as 135, even safer, if you look at corporate jet operations.
But it there is no oversight of 135 ops and every "Fred's Flying Circus" can offer charter flights imagine the chaos. Cutting corners to undercut the competition is the business model of illegal charter operators. Under or unqualified pilots, maint. staff, skimping on repairs and flying outside of wx, fuel, rest requirements would be the norm...and sometimes it is already.
The best example that comes to mind is the KTEB CL-600 crash in the early 2000s. Illegal charter operated in every conceivable illegal manner got away with it for years but finally got bit. Now the principals are in prison.