Thursday night our company was moving servers for a critical app from one datacenter to another. Well, they imaged the servers to new ones and set them up at the new datacenter. Rollback if the migration failed is just pointing the DNS back to the old servers.
One guy was supposed to test the system by running fake transactions.
Yesterday morning I get to work and everybody's freaking out. Both the new environment and the original one stopped processing transactions. A conference call is started with developers and the project manager and such. Nobody can figure out what's wrong.
I go in and on instinct decided to audit the test transactions.
"Hey guys, I see all the transactions right here. What's the problem?"
The tester had failed to keep an accurate count of the test transactions and forgot he used different email addresses for the source and recipient. It had never occurred to anybody the test itself would be the problem, or more accurately, the interpretation of the test.
Woot! DurocShark to the rescue!
One guy was supposed to test the system by running fake transactions.
Yesterday morning I get to work and everybody's freaking out. Both the new environment and the original one stopped processing transactions. A conference call is started with developers and the project manager and such. Nobody can figure out what's wrong.
I go in and on instinct decided to audit the test transactions.
"Hey guys, I see all the transactions right here. What's the problem?"
The tester had failed to keep an accurate count of the test transactions and forgot he used different email addresses for the source and recipient. It had never occurred to anybody the test itself would be the problem, or more accurately, the interpretation of the test.
Woot! DurocShark to the rescue!
Facebook
Twitter