What BackPacker said.
If I have one COM available, the last thing I worry is listening to 121.5 - it's nice to assist somebody in urgency/distress, but safety of your own aircraft should come first, so it's better to listen out to ATC for instructions/advice/traffiic information concering your aircraft. As for your handheld radio goes, spare it in case you have communication failure - it would be pretty embarrasing having to comply with comm-failure procedures when you have a serviceable handheld radio on board and you've instead used it to monitor 121.5, where the only messages was some air-to-air chit chat.
Beside the fact that BackPacker written about FIS having much greater coverage on 121.5 and direction finding capabilities, you have to be aware that VHF communication is based on line-of-sight principle. So basically, if you don't have the aircraft in distress in line of sight, listening on 121.5 doesn't help a thing. For example, if you're flying through tight valley (below sorrounding peaks) - the only traffic on 121.5 you'll hear is from previous turn to the next turn - not very usefuly, isn't it?
I think it's better to leave listening 121.5 to transport aircraft, which have 3 VHF stations and usually one of them is tuned to 121.5 - being high in the sky they have a lot greater coverage, so they will most probably (if transmitter is working quite OK) hear any traffic on 121.5 in excess of 200NM. Of course, if you have two VHF COM stations and you don't need the second one for ATIS/weather/listening to overflying small airfields - do set it to 121.5, but in your case I believe it's just too much workload with barely noticable effect on overall flight safety, if any at all.