Launching ColbertNation.com
This has been a very busy and exciting summer for us, and probably the best example of that for me happened last month when we launched ColbertNation.com. As you might know, the site is the official fan site for The Colbert Report on Comedy Central. The show is insanely popular and I knew it already had a huge audience. That means there’s a lot of eyes watching as we launch and that there’s not much room for error. So of course since I was confident that everything I could control was all set, there would be problems with things out of my control.
Knowing this, I had double-checked all the code and server settings, and scheduled the launch for a time when we’d have the least traffic. It coincidently coincided with the week that Stephen Colbert’s face was staring out at me from my desktop page-a-day calendar. I’ll back up for a second: This year I’ve been enjoying America the Calendar, which is the calendar version of America the Book written by John Stewart and the Daily Show writers. I’m a huge fan of the Daily Show and of course Colbert, and the calendar is filled with jokes about politics and media. But for the week of July 18th there was a week long gag demonstrating the different TV Journalist expressions. It’s funny on it’s own, but his expressions from that calendar were a day by day reflection of our launch status.
July 18: The Day Before Launch
Stephen’s arched brow is glaring at me. “Are you absolutely sure we’re ready for launch?”
July 19: Launch Day
All set! I’ve convinced him we’re ready, and we’re going to launch tonight.
July 20: First Day After Launch
Looking good. No broken links, all our pages and cms are working, we’ve got video, and the new messageboard is starting to take off. Maybe this went off without a hitch…
July 21: We have a problem.
Damn, he looks pissed! And with good reason. The site is performing terribly. We’ve got excessivly long load times, and the message board is barely loading. The funny thing is that I’m watching our traffic in real time and it’s well within reasonable bounds. I start re-checking all the files and settings, but everything checks out. To be frank, I’m a bit stumped over what the source of the slow performance is.
July 22: Our problems are miraculously gone.
A new day, a new developement. Today everything seems back to normal. The funny thing is that I didn’t make any major changes. (I hadn’t found a problem to fix!) I hadn’t heard from our host’s tech support yet, so I’m still unsure what the issue was. Still, we’re all grateful that the site is back to normal and working well.
July 23: We recieve our answers.
We finally find out what happened, and why we hadn’t heard a complete answer from tech support. We had reasonably normal July weather here in Philly and NY, but our datacenter is in LA, where they were experiencing one of the worst heat waves in recent memory. Well, that was a bit too much for the data center and they were struggling to keep everything up and running and not trigger a blackout. Sure it’s disappointing to hear, but I can respect that. Since then we’ve had very good performance, and they’ve put measures in place to keep this from happening again. We’ve also added more robust back-up systems on our end to cover for new problems as they arise.
The punchline for me is in the timing. It all had to happen simulatneously, my launch, the heatwave and power issues at the host, and then these calendar pages staring out at me as I’m dealing with each issue as it comes up. That’s what makes it surreal.