Every year recently our wedding anniversary is interrupted by some kind of outage.
Two anniversaries ago an FTP port was left open and someone ran cracking software and brute forced their way in to our Weblogs, Inc. servers. Once in, they defaced many of our sites and we had a hell of a time getting it all restored. We also parted ways that weekend with the first developer who worked me on Blogsmith.
Last year the outage was my back. I threw it out on the way to visit relatives in Virginia and lost a couple of days. It's all better now. I could tell that you were wondering. Thanks.
Last night, on our way to the Yankees game with the kids, Christoph let me know that our mail server was having trouble. More specifically, the giant drive that has all of our mail and account data on it was dying.
The hosting company quickly added another drive which failed to format. Then they added a working drive and Christoph began the multi-hour process of copying over the mail files and getting it all back on track.
During all of this we learned that the nightly ISP backups did not include the drive that has our mail on it -- which is the only thing that needs to be backed up on that server. Amazing.
It's making me paranoid. Not paranoid enough to start using a vacation surrogate, but I will definitely start doing extra backups the day before our anniversary from now on.
http://www.brianalvey.com/2007/07/19/i-know-im-counting-good-times-but-there-were-just-as-many-bad/
