It depends on how much you want to do it. I know that probably doesn't help, since you are keen enough to have got this far, but feel as though you still at the foot of the mountain.
I wonder whether your 'time management' could be improved to reduce the time you are away from home? It's a while ago, but I remember when I was doing the PPL course I very much had to 'manage' the school. Phone before you leave home; if the aeroplane's U/S or the weather doubtful, don't be afraid to cancel and save the money and time for a better day.
One thing I would say is don't worry about the safty aspect. It can happen to any of us, but mercifully it's rare and very much under your own control - unlike motor accidents where you can be killed by the HGV whose driver fell asleep through fatigue. When I became a dad many years ago I contemplated giving it up, but it's important to keep a sense of proportion. I'm certainly very glad I didn't give up.
If you are really not enjoying it, it may be better to stop. But it's normal to feel a bit overwhelmed at the point in the training you're at. It gets better after solo - a real confidence boost and probably closer than you think. If you can see *any* light at the end of the tunnel, stick at it ;~)
Hope this helps.
SSD