Check that the WEP or WPA (or whatever) security set in the router is the same as the setting in the WiFi in the laptops.
If the router is WPA and the laptops don't have WPA, that'll stop you.
The laptops should have a facility to scan for wireless networks in range, and should see your new one. If you tell the laptop to connect to that, it will ask you for the WEP etc code.
I assume you've left the connections to be handled by DHCP? If the old router had fixed addresses for each machine, that could cause the new router to be picky.
I had several Belkin routers in the past, and they all caused me grief and were abandoned. The last was a couple of years ago, so they may have fixed it now. Netgear are generally well liked, as are Linksys.