Well with regards propeller efficiency, is the unit not small enough to drive a fan instead of the jet in a high-bypass turbo-fan engine? That might go some way to assist cooling as well. I've no idea about what the effects of high altitude might be on it, so there's a potential pitfull for commercial use.
That's a great video Stephen on how rotary vane engines actually work!
Another potential power plant could be the
Energy Catalyzer which (supposedly) works by a Low-Energy Nuclear Reaction, although since it produces steam-power lugging around tons of water probably makes it inpractical for aviation.