There's something else - an X-factor of some sort involved. It's not skills alone
You could start with picking a suitably nice day to fly
The passengers will also enjoy it a lot more.
Even when flying alone, I will chuck it in and drive instead. Despite really disliking driving, I drove to a meeting
at an airport the other day (150 miles each way) - because the return trip would not have likely been possible due to wx.
On my long trips to the south, usually with Justine my girlfriend, we set aside 3 days to get out of the UK, and about 75% of the time we get out on the planned day. I avoid drilling holes through frontal weather, which is an obvious recipe for collecting ice, not to mention scaring the hell out of passengers with turbulence in IMC. Even though a lot of warm frontal weather is smooth, and you may get lucky on icing, if you badly scare passengers just once they will never fly with you again.
Most people know there is an increased risk in GA flight, but they take it on trust that you won't kill them. If they get scared even once, that trust is gone and that's it, game over.
Piloting skills come into it of course but only to a limited degree, during takeoff and landing. With modern cockpit automation, the flight should be pretty straightforward and is largely a process of constantly evaluating the wx ahead, climbing, etc. Not having oxygen and an IR also cuts off your best wx avoidance options.
It takes a lot of judgement to suss out the likely wx - an area very poorly taught in the PPL. In my PPL (done 2001) the internet was never mentioned and today this would deprive you of 99% of the required information.
I find that most of the 20k-hour pilots one reads about getting killed actually had little GA experience. They built the time flying fully deiced big jets with a 5000+ fpm rate of climb and enough TAS to completely avoid icing enroute due to the airframe heating (AF447 was a case where even this didn't work). Of the airline pilots who fly GA, the huge vast majority avoid IFR; they prefer basic rag-and-tube flying, which is understandable (makes a nice change I am sure) but this deprives them of getting any experience relevant to going A to B in real wx.
but it's just a sad state of the times we live in that our kids are not allowed to go out by themselves and play, explore and learn
They can if you live somewhere where they won't get run over if they step outside the door. That unfortunately usually involves spending more money on the house purchase