The new in ear head sets can circumvent that problem. I use a Plantronics headset with acoustic tubes connected to ear molds. Today you can take such a headset to an audiologist to fit Comply tips.
An optometrist can change the earpieces on your glasses to something that will be reasonable under a headset.
Maybe even try cutting a slot in the headset padding, but the cure may be worse than the disease.