Originally Posted by
Fran Tick
Forget millions of lines of code. It only takes on byte, in the wrong place to cause such a problem.
(Finding it , among all the lines of code, that's the fun part).
Yes, agreed... But the change from night to day configuration is carried out by code that must have executed hundreds (if not thousands) of times. So the question is more: what changed?
Agreed, it could be some obscure bug in the code (suddenly reaching that point where a short int no longer works and you need a long one), but personally I think that's unlikely.
My guess:
a) Something broke that wasn't redundant;
b) Somebody implemented a change and it went wrong / had unforeseen consequences (or the backout plan failed / made it worse);
c) There is redundancy in the system but as with all major outages which are a result of a series of unfortunate minor faults that unfortunately occur at the same time, the redundancy was unavailable.