MM43 already described the arrangement of the ULB on the FDR. An image is seen here:
https://commerce.honeywell.com/webap...tegoryId=39509
If a pressure housing does not implode on initial exposure to high pressure, it is unlikely (though not impossible) to do so at a later time, from a structural standpoint. What can cause a time-delayed failure is perforation of a housing or sealing surface due to poor material choice.
I have seen a stainless-steel housing in, seawater, fail from perforation where an adhesive label was applied. The label excluded oxygen from the surface, and thus allowed crevice corrosion to proceed.
This sort of failure can be avoided by appropriate choice of material and and/or by correctly sizing sacrificial anodes for the surface area and duration to be protected. Let's hope that Honeywell's design will exceed the minimum requirement.
As others have pointed out, solid state memories may be adequately protected, by their encapsulation, from direct exposure to high pressure seawater for a significant duration. It is possible to remove the encapsulation and gain access to the chip even if the external contacts have corroded away.