If you take it back, change it for a DSL WiFi router. Then you have your DSL modem and router in one package so it cuts one link in the problem chain.
I'm on cable here so a regular router does me, but if you have DSL then get a router with an inbuilt dsl modem.
As I say, there shouldn't be an issue with getting a signal through the floor unless you have 1m thick floors. In my house here the floors are 0.5m reinforced concrete and I have no issues with signal strength.