I wouldn't normally write about this, because normally, service interruptions at webcomics host Comicgenesis don't affect me directly. I still have a website (two, actually) there, but it hasn't been my main webhost for a long time, and I can take or leave a new update there being on time.
However, now that the updater problems have gone into their second day, they're making this massive multi-artist crossover event I'm in a bit confusing. I mean more confusing than it would be if everything was still running smoothly. So, a few practical things:
1. All comics are being collected at the Crossover Wars hub, including the ones linked from my archives that aren't currently updating. CameoComic for April 3. CameoComic for April 4. This is done manually, so there's still a delay, but the material is finding its way online through people's Photobuckets and alternate hosting solutions. (I would particularly invite cartoonists to get a free account on Webcomicsnation because setting up a series there is easy, and the cross-promotion works, but there are plenty of other good, free solutions. BTW, I would definitely classify Comicgenesis as both free and good, these days, even taking into account this annoying glitch.)
2. I am keeping the links in my archives as they were originally planned, unless someone pipes up from Comicgenesis and says that the problems will persist for the long term. If that happens, I will start linking to people's permanent, alternative solutions. Otherwise, I'll have the devil of a time reversing all the changes once the problem is solved. In the past, I've found that crossovers are high-maintenance things to have in an archive, and I don't want to give myself more of a maintenance load than is strictly necessary.
3. Pimpette still has long-term problems getting back on track, so I'll hold off on linking to any more of Norla and Ottar's adventures in that webcomic until things are normal again.
Things happen. Points to reader Alun Clewe, creator of the webcomic Soup, for predicting that something like this would happen, but more points go to L.P. Hogan, organiser of the event, for taking precautions.
So, what's the problem? Well, as far as I can tell, it started with hardware failure in the box that handles their Domain Name Service. They had two servers for that, but both broke at roughly the same time. Yet another case of redundancy not working. As it happened, they were working on one of those boxes when the other broke, so they could fix that relatively quickly. But ever since, the updater has run very slowly. The updater is a script that creates static HTML pages at the time set by the account holder - a new front page, a new archive page, refreshed pages that link to the new archive page, and any other stuff that the account holder may have queued up to be generated - for all accounts. And there are thousands of accounts. The current script has been revamped since I had ROCR.net on Comicgenesis, and is presumably a lot more powerful, robust and intelligent than it was then, but it's still the most vulnerable in case of something breaking, and when it gets behind on running through its queued updates, it can take it forever to catch up, especially when some other part of the system is still broken. So Caitlin and Ti-Phil and all the others involved are now having to wait for that to finish. They're producing their comics all right, but the site automation isn't taking them. Latest reports I've read on the forums said the queues were only getting longer, which is something I've seen before. In the past, that's been the harbinger of an even more catastrophic problem. While I don't think this will be the case this time, it may be a good idea for CG cartoonists to back up their archived material.