Far be it for me to tell you how to fly but.........
I experimented with something similar a few years ago when I was flying single pilot ops over fairly long sectors and was getting extrememly bored. I concluded that it was not a good idea.
You only need to miss one radio call and that's it. If you really must have your music then how about a CD player/walkman with only one earpiece inserted. You could wear your headset over the top and with the balance set to that side you could hear it all (not in stereo but hey, it's better than a mid-air).