I think it would depend on what you failed it on. Obviously there are some serious conditions that would preclude you from piloting at all, but some merely restrict the kind of work that they'd allow you to do. For example: I did my C1 a couple of weeks ago and while chatting to a doctor/nurse was told that one of the things they check is curvature of the spine. If it's present then they might not clear you to fly helicopters and a military career would be out. Fortunately I passed it all with no problems. (And no spine issues thankfully as I want to fly helos!)
Si