I managed to open a DMG in Fedora/RedHat/Linux, maybe at a last attempt you could try that.
However, i have used the following tool
http://www.acutesystems.com/scrtm.htm
Its called transmac, there is a 30day demo, so you can see if it works, and its about £30.
From a little searching on the internet, the best tools i could see are.
UltraISO
MagicISO
There are a few tools that convert DMG to iso's but they seem to be unreliable.
I have used TransMac, i know it works, so i would give it a shot first.
You say you have used one of the tools i listed, so i am guessing the other is no good. You can download a copy of VMWARE Player and Fedora for free, Linux and MAC/BSD use a similar file system.
I always try WINRAR from RARLABS to extract anything and everything, have you tried that?
Having lost my partition recently, i have been trying some data recovery tools, they seem to find loads, stuff i deleted a fair while back and a few os installs ago, i use pretty much all my hdd, so it didnt find too much as i am always going over the hdd. worth a shot.
Have a go at DMG to iso too, its free, but i am lead to belive its pretty hard to use?
Last thing i could find is
http://www.mediafour.com/products/macdrive6/
its supposed to ipen DMG, well be able to mount.