I got myself an HP "JetDirect EX plus" box from ebay (cost about 15 quid) that does exactly what you want. Box plugs into the router via ethernet and the printer plugs into the box. Then you set the printer up in Windows as a network printer, give it the IP address of the HP box and off you go. It's about the size of a VHS tape, sits there quietly doing it's job and never goes wrong. Perfect.
Newer routers often have a printer port built in. Mine, the Netgear 834G, doesn't.