Nerve head drusen are rare and have varied presentations - usually, like you, as an incidental finding.
While there are many literature case reports and small cross sectional studies there are no decent longitudinal studies demonstrating progression - don't believe everything on Wikipedia.
I don't see amy reason why you shouldn't persue a career in aviation. Given the predominantly innocuous and static nature of drusen the CAA would be hard pushed to deny you a license.