Youtube videos are often available in a range of bitrates/resolutions and tend to default to a lower setting.
iPlayer can be a PITA at times, depending on demand / line speed. Streaming over wireless networking slows it as well.
My experience is that the ITV service is even slower -though that could just be a local peculiarity.
But in essence, all you can do is optimise your internal network, check the connection speed from the ISP, and make sure the software on the machine is optimised. Make sure you have the later version of Flash, if you are downloadin for storage make sure you have the latest Adobe Air version, get rid of old temporary internet files - and try other browsers. I hate to say it, but I find Google Chrome tends to give slightly better results - but that could just be a peculiarity of my computer. You may also find that having RealPlayer installed on the PC may slow things down as well: the Youtube download feature can hinder flash streaming, its not compatible with FlashPlayer
What else? Make sure the hard drive isn't full, fit extra RAM, and make sure the virtual memory settings are set to "automatic"
Finally, during playback right click the video and set the flashplayer settings to allow "unlimited storage"