I had a problem when I had my Class 1 initial that stemmed from exposure to loud bangs while in the army. I had a dirty great hole in the high frequency band in BOTH ears. OK it cost me £140 to have a specialist write a letter to say that it is unlikely to get worse but it was worth it. I have had several standard audiograms since all of which still show the problem and every time I have to tell the AME that I have had it checked by a specialist blah,blah,blah. Fortunately I asked for a copy of the results from the specialist so that an AME can renew my medical without refering to the CAA each time. I suggest that you do the same.
Finally don't worry believe it or not the CAA are on side with this sort of thing they just want a specialist to tell them what they really already know. It's an @rse covering exercise.
Good luck