To my mind, in the same category as Apple (perhaps slightly worse, if anything) is the Real Player.
This gets removed from my system toot sweet. A very suitable replacement can be found
at free codecs dot com (also available at several mirrors, including
File Hippo. The player looks a little sixties, but it works, is light, and doesn't spread tendrils through all the nooks or crannies, pester you for more toys, or ask if it's bum looks big.
A version is also available for the files the Apple player takes care of.