Grogmonster, what elevation was your departure field?
I've had a similar experience with slightly different results.
MTOW departure, 9 POB in a PA-31 but from a field over 3000' AMSL, with a very cold OAT luckily. The engine let go at about 100 AGL and was secured and feathered by 700' AGL. From that point on the performance was not positive at all, but thankfully the RoD was only about 200 feet per min. There was no way that the aircraft was going to climb and a constant descent via the circuit back to the runway was only just possible with the gear coming out over the fence just before touch down.
Below is an incident involving a PA-31P that diverted from Young to BK after an engine failure and shutdown (at FL170) and there was plenty of performance available...
http://www.atsb.gov.au/publications/...502231_001.pdf