I have the same problem, which is worst when map reading. Half-frame reading specs, perched on the end of my nose help, but I still find them a distraction when landing. However, on low sun days, sunglasses are essential, so I've had a pair of tinted bifocals made by my optician. It would have been better if I could have sat in the cockpit while he marked out the lense area, because I just had to guess where the top of the lense should start, and I guessed too low.
Headsets can make it difficult to change or remove specs in flight, but I have the
Clarity Aloft ones, which allow you to push the specs up to the top of your head when you don't need them.