A turn towards the dead engine isn't a problem in itself, as long as the usual caveats are observed such as above Vmc etc.
Sounds like a fast approach, speed not bleeding off in the flare as fast as the end of the strip seems to be approaching so a go-around started.
As has been said, without the correct configuration the only altitude gain will be by trading off airspeed. If persisted then a Vmc departure is the likely outcome.