And sometimes the testing is TOO good.
About 4 years ago, a Debian (Linux) developer kept getting error messages about stuff referencing uninitialised memory, so fixed it.
Great - except the reason for using the uninitialised memory was to use it as a high source of entropy for random number generation for cryptographic keys. Removing the source of entropy weakened the key strength enormously and reduced the time taken to crack them to something practical. New keys were needed to be generated for machines out there to make sure they were secure - the fix the developer made had been around for 2 or so years at this point.
See
Debian and Ubuntu OpenSSL generates useless crypto keys for more detail.