I have been wondering about this ever since it was demonstrated to me at work 6 months ago. But, I had other things to think about, and I didn't think too hard about it.
When the rotor is winging around, by the Bernouli theory of lift there is low pressure on the top of the blade, and high pressure underneath. Now, a helicopter hovering in free air is going to experience a couple of things. First, the low pressure area above the blades will tend to pull the ambient pressure air from above the disk down through the blades. Second, the air ejected out the bottom will tend to jam into the ambient pressure air, causing a local area of high pressure. Differential pressure = lift.
Now consider ground effect. We all know that ground effect reduces induced drag. My own little pea brain also likes the 'ground cushion' description, which I interpret as the outflow gets jammed into the ground, creating an area of local higher pressure.
So what happens at the ceiling? Inflow into the disk is crap. There is a ceiling in the way, so you don't have an infinite reservior of air to get pulled into the low pressure. So, the local pressure just above the rotor disk gets really low, because it's hard to get air back above the rotor disk. It has to come from the sides, and the spanwise flow tends to be outward rather than inward.
I would reject Nick's hypothesis that it is the same as ground effect because what I've seen is a 50 to 75% reduction in power (estimated). It's way stronger than ground effect, especially with a coax with no flybar where it can get really close.
Meanwhile, I'll get my team of crack test pilots to work on the problem some more.
-- IFMU