I took my oldest for the first time when he was 5, and my youngest when he was 6. My older boy was more nervous about things in general. We spent a lot of time just sitting in the aircraft (Cessna 140). I never suggested we go flying, just talked it up and waited until he asked me to go. First flight was 40 minutes, I was ready to make it shorter if there was any sign that he was not 100% into it. My second son is on the autistic spectrum, though very high functioning. His first trip consisted of a single trip around the circuit. Just wanted to see how he would do. Transitioned both kids to fly in the glider with me. They both like it a bunch.
The booster seat is critical. I have flown many kids, if they can see out to the front, they will look out. Otherwise they look inside and that can be bad.
At a young age scare then once, or some other bad experience, and they are done. Make your objective not to maximize flying now, but to maximize flying later. Now is introduction/brainwashing. Despite your desire to share flight with your kids, these flights should be biased around them, not around you.
My absolute best flights have been when I take my kids. It is a joy.
-- IFMU