A collegue of mine had that exact failure some month back. They were cruising at FL380 when the packs went offline. According to him, they agreed faily swift on initiating a descent but still used a few seconds to evaluate the circumstances first.
They were, cabin rate of climb = 500 ft/min. Time available = x minutes (can't remember what he told me exact), and MSA about 4000ft.
They initiated a normal descent (in open descent) with a turn off the airway, declared a Mayday to clear the way ahead and monitored the situation.
Cabin never came higher than 9000 ft and no masks were deployed. Pax had no complaints as it was not uncomfortable and cabin crew managed to control their trolleys with no danger for themselves and pax. Mayday cancelled again and they diverted with no stress.
Seems to me to be a job well done, and there were no stress as they used the time needed (but not more than that) to evaluate factors. The A/C was fairly new, so that might have been one reason why the cabin climb was kept low.