So they're both connected to the router, and both working OK.
If they can't see each other, the problem lies somewhere in the Microsoft networking. You could play for hours in there...
Try a cheat:
On one machine,
Start Windows Explorer (Windows_key and E will do that)
Click Tools - Map Network Drive
Pick a letter that isn't allocated to anything already (somewhere in the range N - S is usually safe).
Click "Browse", then the + beside Microsoft Windows Networking
Hopefully you'll see a local network name - either one you created, or "Workgroup". Click the + beside that.
If you can see the other machine, drill down to see if you can connect that as the drive letter you chose above.
If that works, try it the other way round (from the other machine).
Let us know if that works, or what error you get.