As others have noted, the US fighters didn't often face heavy bombers in either the European or Pacific Theaters, and the 50 cal. did the job quite well on the smaller aircraft (especially the Japanese which lacked defensive armor and such).
My dad fought in the South Pacific - Guadalcanal, New Guinea, Philippines. By the time he got to Guadalcanal in late 1942 the P-39 was pretty much outclassed for air-to-air, but was very effective for air to ground with that big cannon in the nose (TBM - I thought all the P-39s had 37mm cannon but perhaps some used the 20mm). According to my dad, when the P-39 fired that 37mm cannon during a strafing run, he could see the aircraft 'stutter' from the recoil. If his observation was correct, I can't help but think that firing that big cannon during a dog fight would make aircraft control very tricky. Further, for air-to-air, mixing (relatively) slow cannon with higher speed 50 cal. would make leading the target for both nearly impossible.
The picture that Just This Once posted helps, but it is hard to appreciate how big a 50 cal round really is if you haven't seen one. I have a bottle opener made out of an inert 50 cal round and it is HUGE!