When I was younger I had similar problems. I failed my initial CME - doctor thought I had a cold. Went back, still the same but I passed after passing that dreadful test where similar sounding words are spoken with a deafening engine noise as the background. Subsequent medicals all the same. One AME referred me to an ENT specialist. Audiogram showed loss at mid and lower frequencies but extreme sensitivity to high freqs. All then forgotten for many years.
About 20 years later did my medical at Heathrow and the doc said: "Used to shoot without ear defenders did you?" Yes - I did as an Air cadet in my teens. "That's why you have that hearing loss", he said! Apparently his son had identical problems due to the fact that we were never told to wear ear defenders for either .22 (indoors) or .303 (outdoor) shooting.
Now I'm in my 60s and my hearing is deteriorating I need the TV turning up so much my wife is nearly deafened (but she used to shoot too!). But, when we're out bird watching I can still hear the high pitch of Goldcrests, Tree Creepers and other birds which she cannot.