Thursday, November 29, 2007

Insert tab A into duck 7.

For the most part, documentation is something that you really want to see written and followed, and ideally kept up to date.

But what happens when you have bad documentation? You get hilarity.

$otherTeam was following documentation on how to set up their application. Alas, the documentation seemed to be written to assume no-one could ever resize disks, and required that the installer unmount /tmp and then symlink it deep into application land like /var/application/fluff/bits/things/tmp. We were using kerberos logins, which requires a writable /tmp. So $otherTeam unmounts /tmp and boom, no-one can log in any more. And then they exit their shell.

Oops.

and all the cake is gone

The project I'm currently being punished with has gone wrong in far too many ways to count, but today's is extra fun.

Throughout the project we've been plagued with DNS issues - the customer manages their own DNS in this environment, and the server I'm currently setting up has had it's IP address recycled from a recently decomissioned development box.

They updated the A record for the server when it was comissioned - but it seems they forgot all about reverse. For a long time the A record and the PTR didn't match, which caused all sorts of grief with software that expected the reverse DNS to resolve to the name they had in their configuration.

We finally managed to convince the customer to fix up this issue. Except that it seems they've painted themselves into a corner.

KB: "Windows 2003 AD does not allow you to delete DNS names with uppercase letters"

They made the entry in all uppercase, and it seems that means we're basically stuck with it. Even better, while stuffing around today they managed to make it not resolve to anything at all, so now kerberos is refusing to work. No-one can log in. At all.

I'm going to go and have some coffee since I can't actually get onto this box to do any work. Yay!