Adrian
I know that you probably took up flying with taking the family for days out firmly in mind.
I was taught when I started flying and had just got my PPL that getting the PPL was a licence to start learning to fly!
Do you really want to be learning with the most precious cargo you can have on board?
I would suggest that in the early days if you want to take the family take an instructor or experienced pilot friend along.
Also consider that these are not babies but probably strapping 13 14 year olds so you may be looking at renting a six seater or when solo an aircraft equipt to carry 5.
Everything in time but new PPL? get some hours under your belt first.
Pace