It's a no-brainer. It is either authorised by the FM/Ops Manual and therefore legal, or it isn't. If it is, and the captain chose not to do it, he is either wise or silly depending on why he thought it "dangerous".
In any case, gear-down ferries are normally done without pax.