In addition to the above, or expanding on the points...
Hyd accumulators don't actually represent a lot of stored energy; you couldn't rely on them for very long to power things like flight controls, which have continuous demands. Even for brakes, often the guidance is to use the brakes, when on emergency/accumulator scenarios, in a single smooth application and not pump or cycle them, as you'll soon deplete the accumulator.
And additionally to that, and to the point in the OP - most hydraulic systems have multiple power sources (pumps or transfer units of some kind) - in fact, I'm tempted to say "all" but there will of course be counter examples for less-critical systems. What that means is that to lose hydraulic pressure in a system requires either a multiple pump failure scenario - which means it's really not your day, as it's pretty unlikely - or a system leak, which is the most likely way to lose a whole system. if there's a leak, all the accumulator does is help push the fluid out of the leak - when the system is dry, you have no system whether there was an accumulator or not.