My idea on AvMed is that it should be more like the ATSB. They investigate and publish causes of Aviation incapacitation, publish aviation related medical issues and papers and that is accessible by medical professionals who treat pilots as a guide. As well as providing guidance on what constitutes fitness to fly, rather than being the traffic lights on the subject. This then can be legally used by the medical practitioners to make reliable decision on a patients fitness to fly. The pilot then can work with their GP and specialists to maintain their own fitness, with some form of basic certification each year to ensure each pilot is maintaining their fitness.
Because of the liability implications you will not be getting GPs to just sign dodgy certificates so it would be a robust system.
Even without a doctors signature, you will still have to pass AFR/BFR/IPC or whatever regularly, so it would be obvious to those testers who is obviously unfit and who isn't. After all you were issued an non expiring license that says you are eternally competent as a pilot, and other approved pilot can deem you suitable to exercise your privileges.
And the mechanism which continues to have the single most profound practical consequences in the regulatory regime is always available to get around "unforgiving and cumbersome" regulations: Exemptions. That's why there are plenty of pilots flying 'legally' and 'safely' out there who are not obliged to tell Avmed anything and about whom Avmed knows nothing.
I think safely may be a fine term, but legally is what I'm disputing. Just because AvMed does not know, does not mean it's legal. It would be up to a court/coroner report to find if you were legally able to fly on such an occasion, and depending on who's out to get you will alter how hard it will be to get out of that corner.