Once I've turned crosswind, I might think about it, especially if there's a crosswind runway available, but I'd be a lot more inclined to land upwind.
If you are an experienced pilot (which I assume you are), you should not be so quick to be certain of such things. Circumstances on the day could dictate the best course of action being a 180 (or partial 180 in my case).
If the choice is between turning back downwind and (maybe) crashing on a well equipped airfield, I will take being close to the airfield any day in preference to eating pine cones straight off the trees.
My Schiff book detailing this is in the office so shall collect it tomorrow to post the reference, but basically the 180 is both theoretically and practically possible, dependant on a number of circumstances which will only be aparrent to the pilot on the day. For a low hours pilot, the experience to make the best judgement call may not be available for decission making in the available time (although your decission making does become mighty quick in these circumstances), and for these pilots, land ahead HAS to be the best option.
It would be good to get more input on this thread from pilots who have suffered REAL EFATOs in SE AC. I noticed on another thread that things are descending into FS2004 on this topic
Edit: The Schiff book which dedicates a chapter to the turnback is The Proficient Pilot Volume 1, Chapter 34. The Schiff books are all well worth a read.