seacue's tip is valid IF THE OS IS WinXP, as it allows you to add and ditch network connections on the fly. However, previous MS OSes will not let you do this- you will require a reboot.
Also, there is usually an option in the admin page(s) of the router to set the "drop connection after xx minutes of activity" option.
Try logging on to the router and see if this fixes the problem.
If not a may be an issue with the lease lengths for individual IP's assigned by the router if your are using DHCP. Most people think DHCP is OTT for a home network (although I use it) and therefore you could also try using static IP addresses.
Regards,
Shuttlebus