frostbite,
The issue of speed on the internet is a strange and complex business.
PPrune is not the place to explain it, it would take far too long to even start describing it to a non-technically inclined audience.
The one thing you need to remember about ADSL is that speeds are not guaranteed. There is contention everywhere along the way. As a home user, you are subject to much higher contention ratios than business users.
There is a BT technical document that describes the speeds that your line can slow down to before they consider it to be a problem, and you might find that speed is surprisingly low for what the marketing guys call a "8Mb" line.
There are certain points on the internet where lots of ISPs group together to interconnect and exchange traffic free of charge. Here is a graph of the biggest such place in the UK, which might explain why you are seeing things speed up as the evening wears on.
