This failure means that the whole valve is open like in the ground. Not just the small flap valve which can be open when airborne in hot days when the intermediate configuration is active.
As you say, it doesn't seem likely to have that failure during the cruise. It seems to me that is possible for the outflow compensate for the extra "hole" and keep climbing to cruise altitude. Or maintain pressurization if the failure occurs at cruising altitude...
But the extract valve is not designed for that purpose, so there could be a risk of estructural damage or perhaps problems in the avionics bay due to an excessive flow or something like that if we ignored the ECAM procedure.
So we better follow the blue instructions...