I find the 'Swiss Army Knife' approach to be the best policy..make sure you have at least
QuickTime,
RealPlayer and
Windows Media Player (augmented by
Flip4Mac for Mac users) installed. An increasing number of sites are also using
Flash Player to replay video (not its original purpose), so make sure you have the latest version installed (this probably came pre-installed on your machine).
You should also consider adding
DivX support and the wonderful VLC Player from
Videolan. This should have you all set to be able to view 99% of web video content