Total,
Absolutely not! I'm ex RAF rearcrew (had the bug, but not the smarts for pilot), and due to family committments when I left, could only start to learn in my early 40s. You're miles ahead. Just keep in touch, and as your circumstances change, pick up the pace.
CG