The reason they're not allowed is that this would further erode the last bastion of resistance for a very small group of aviation optometrists who are clutching on to their raison d'être by the finest of threads.
There is very little justification for the massive costs incurred by the CAA medical department, a task which could easily be administered by the DVLA with oversight over AMEs, and every post they lose would weaken their position further.
Therefore, they dream up excuses for maintaining the status quo over CVD and conjure up invalid objections to any positive progress.
It is all about protectionism of jobs, perks and final salary schemes.