The key issue is not whether he gets specific medical info from your GP, but whether you are excluded from a process that I believe you should be orchestrating.
There may be a misundertanding. The AME would not consult your GP without your knowledge and consent.
In fairness to FAA AMEs, I have heard from older friends with medical issues of the AME suggesting additional tests etc that could be brought to the table in building a better case for Oklahoma City to review and act on.
That is how it works in the UK too, with cases referred to the CAA Medical Department. In my experience, they are very helpful.