Saturday, August 18, 2007

Everything has to have a beginning

It happened to me this week.

I was making a routine scheduled change, and suddenly things started to go very, very badly. A database cluster in smoking ruins at my feet, caused by the conflict between what the sysadmin knows is realistic and what the customer thinks should be possible.

After all, they got it working in their Ubuntu virtual machine and that's practically the same as a production system, right?

Every sysadmin knows the moment. You watch the output scrolling across the screen and something nasty catches your eye. Your heart leaps into your throat and your stomach sinks. This moment is best described as "Oh shit".

This blog is for everyone who's ever had that moment at 3am sitting in a cold, dark server room squinting to see on the ancient CRT attached to the KVM. Let the war stories commence.

2 comments:

4fthawaiian said...

Oh, wow.. Do I ever know that feeling.. Congratulations, you captured it well. I could feel my stomach sinking just reading that.

Olwen said...

Thanks! I hope you enjoyed reading :)