I can imagine how stressful the situation was, but it was pleasure to read. It again goes to show you that no matter how prepared, how optimized/over optimized you want to be, there will always be a situation you have never accounted for and sh*t always hits the fan, that is the reality of IT ops.