Hypothetically, small packets can slow things down. Don't know much about broadband per se, but I'd start by looking to make sure your ip is loaded okay and fiddle with the hardware config on your network card i.e. not irqs etc but the properties, e.g. flow control, media type etc.
Unique (to you) problems are hard to solve through such a medium as this board. Anyway, if your FO knows what the problems is, get him/her round our house to fix it!