Hi Paul, I was in a very similar position to yours a few years ago and I have stopped power flying - there are some aspects of my decision I am happy with, some that I am not.
6 years ago I got my PPL. I struggled to maintain the amount of flying that I need ed to do to keep "safe" due to a combination of cost(its a bit hard to spend an evening on the internet trying to save £100-£200 on the family holiday then blow the same amount the next day in an hour), plus, after the real challenge of getting the PPL the £200 coffee became less interesting. I would have liked to have taken things further, but aeros/ IMC etc all cost a lot of money to do.
Safety - whilst, when I passed my PPL, I was very current and could fly in my sleep, I found that, flying fewer hours, the basics of flying a plane took more of my attention, leaving less for the feared engine fire/failure. Similar thoughts and concerns for my family as you are having added to the situation.
Just before my 2 years renewal I was one hour short then had a transient medical problem that stopped me flying briefly - end result no renewal.
Anyway, about 2 years after I booked on a gliding course on a bit of a whim, and loved the new challenge. The concept of having an hours flying with an instructor for less than £30 was incredible, and the club atmosphere (so lacking in power flying) was very stimulating. I also believe that gliding is much safer than power flying, and, having been through both routes, that glider pilots are much more skilled at handling an aeroplane. Plus any fear of landing without an engine is far offset by losing the engine failure/ fire concern. I am now progressing through the ranks and, paradoxically, the more flying I do the cheaper it gets - flying for a couple of hours for £8 can't be beaten.
My PPL has now lapsed - 6 years - and I actually very much regret this. I know that I will never invest the time and money to do it again. What I wish I had done was to have gained a TMG (touring motor glider) rating on my PPL, then kept up my licence flying a TMG - this would have kept my PPL at a much reduced cost. I could then have kept the option to fly TMGs or tugs (or my dormant dream of a spitfire...) later on. But now I can't and I am kicking myself.
I know many people here have extolled the virtues of gliding, and I agree with them- however with a young family it is very time inefficient, but in a more pleasantly inefficient way than your power issues - much more helping out and chewing the cud in-between flights than in your power clubhouse when you have to wait. Still politics around, but generally in a pleasant atmosphere with interesting people, who are all interested ion the same thing as you.
So what I am saying is- back off till you have more time but possibly try to eek your license along in as cheap a format as possible by gaining a rating that allows you to maintain your PPL in a cheaper type. Spoil yourself if you get the chance by booking a week's course in a gliding club (there are several glider pilots with young families but I accept that I wasn't ready to make that kind of commitment at your stage) - but don't lose your PPL, you may well live to regret it.
Don't forget that you have 5 years after your last renewal before it is "game over" - another way is to get your renewal, not fly for 4 years then invest in a few hours to get up to speed and do your skills test again. This buys you another 5 years. You don't need to do the writtens again. You need to check the new EASA regs though.