And that's the other aspect. There is an expectation of the level of safety offered by commercial air transport.
While a consenting adult should be permitted to make a choice as to whether they wish to be exposed to a particular level of risk, they need to have a way of knowing what that level of risk is. Otherwise we would just ban aviation for GA pilots and passengers alike unless we could reach the standards of the airlines. As long as the passenger knows what they're getting themselves into, they should be permitted to choose to take the risk, just as the pilot does.
But the expectation that operations under Part 135 (or for that matter an EASA AOC for single pilot ops in a light aircraft) radically transform safety is somewhat illusory. Here are the scores on the doors for 2002-2017 from the
NTSB:
Part-121 scheduled: 0.03 fatal accidents per million flight hours
Part-121 non-scheduled: 0.96 fatal accidents per million flight hours
Part-135 commuter: 1.54 fatal accidents per million flight hours
Part-135 on-demand: 3.6 fatal accidents per million flight hours
GA (Part-91): 12.3 fatal accidents per million flight hours
So Part-135 on demand is 100 times less safe than Part-121 scheduled, but a factor of 3-4 safer than GA.
One could argue that the factor of 3-4 is worth having, but do we really believe that passengers boarding a Part-135 on demand flight know they're 100 times more likely to be involved in a fatal accident than on an airliner?