You have to love PPruNers .. dogmatic answers from their own narrow viewpoint.
In a 3 or 4-engined aircraft above specific TOW you do need to retract the flaps ASAP, so that if another engine fails you are flying fast enough to survive on the remaining one/two.
In a twin this is not a consideration. As NoD pointed out, the A320 has a procedure entitled "IMMEDIATE VMC LDG FOLLOWING ENG FAILURE ON T/O" which involves remaining at FLAP 1 for a visual circuit.
I have no knowledge of other modern twins, but many years ago, on suitable days, a BAC-111 skipper used to finish his EFTO briefing with "but in reality I will level off at a thousand feet, turn downwind and land...."
So my answer is... it depends.