If you're talking about recreational flying, take the kids! I did my PPL before I had children, but as soon as my son was about 2 he was begging to come flying and still does (aged 9 he is getting pretty competent at straight & level and level turns...).
There are two areas you need to be comfortable with:
a. the cost - are you happy that you can provide everything your children need and still afford the flying? (A bit of a stretch for me but just about OK.)
b. the safety. This has been done to death on here before (sorry, unfortunate choice of phrase) but after a lot of discussion it came out as about the same as riding a motorbike - not a cautious option, but not completely unreasonable.
Tim