Calendar of upcoming unplanned outages
Today was Jim Colten's last day in OIT. I think all of us have worked with Jim at some point in his 35+ years at the University. It was a definite pleasure to work with him.
At Jim's send-off party this afternoon, I said a few words about my time with Jim Colten. I remember that Jim always was ready with a story or a piece of wisdom. One truism that I will carry with me is "there is no calendar of upcoming unplanned outages".
This is underscored by the unexpected outage in one of our data centers last week. Try as you might, you can never really predict the future. Bad things sometimes just happen, even unlikely things such as a power loss inside a data center. So it's important to plan ahead and put in place technical practices for unplanned disaster recovery. I think you all did a great job in responding to the outage, but every emergency uncovers something that could be improved.
I ask that everyone - DBA, storage, systems, production services - look to your processes, and make an honest determination if things are the best they can be. Improve what we have, identify and move forward in the new world as it exists now. By doing so, we can make it easier on ourselves the next time we need to respond to an emergency.