Whilst on holiday in NY I've stayed a couple of times [not recently ] at the US Army Base [in the Adams Guest House] at Fort Hamilton which is located at, and almost under, the eastern end of the Verrazano Narrows Bridge. It's a bit out of town [Brooklyn] but very safe etc and, of course, not expensive.Reasonable access to public transport as well if you are considering not driving in New York City.
It's worth a phone call and a bit of Brit charm to see if they'll accommodate you.