June 2010 Archives

What's your mistake strategy?

We all work in IT, and we know that things are never 100% perfect. Things happen. We all make mistakes, typos, errors. The best advice is not to live in the past. Don't dwell on previous mistakes. Instead, learn from them, and move on.

But if you have made a mistake, what do you do next? When things go wrong, it's natural to panic, to start thinking about all the bad things that might happen next. That's human nature. As a friend of mine from ITLP says: "Get over it".

When I make a mistake, I like to create a "mistake strategy". Creating a strategy allows me to stop worrying about what just went wrong, and start focusing on making things better.

For example, when I first got into IT, I was a Unix systems administrator for a small company. Really, I was just a junior sysadmin, but I worked alone most of the time because there were only 3 of us on the IT support team, and the other 2 admins focused on the Windows systems.

The sysadmins reading this probably won't be surprised that, as an unsupervised junior administrator, I eventually ran the "rm" command as root in the wrong directory. I wiped out the /etc directory. My first clue was when the command reported that it couldn't delete subdirectories that shouldn't exist in the directory that (I thought) I was in. (Fortunately, I hadn't used the "-r" [recursive] or "-f" [force] options, since I thought I was just deleting some temp/cache files for a program of ours.)

I freaked out! A million thoughts and questions ran through my head: Did I just destroy an important server? What was going to happen to the system? And, would I get fired?

Immediately, I told my supervisor. Despite the urgency, he took a few minutes to do some coaching with me. That helped me to focus. I started to think more about what I was going to do next, and less on the stupid thing I had just done.

We put together a strategy: Don't reboot the server, use an identical server as a template, and re-create the /etc directory. Now that I had a plan of action, the rest was easy. It was just a matter of running the right commands to bring the system back, avoiding a complete restore. And I managed to stop thinking about my mistake.

To be sure, I learned from that mistake. For the rest of my days as a systems administrator, I'd always check what directory I was in before running any destructive command, especially "rm". Learn from your past, but don't dwell on it.

Since then, whenever something goes wrong, I try to put together my "mistake strategy". What things do I need to do, to make things better? And that helps me to let go of the mistake. Sure, I may still think about it, but knowing my next steps allows me to just "Get over it".

Relative Importance in IT

Understanding the Relative Importance for different levels in an organization can help you communicate more effectively to those around you. Do you want to pitch a new technology to your manager? You may find that if you emphasize the technical differences, your presentation may not get very far. But if you instead focus on, say, the cost savings, you may make a greater impact.

I am interested in writing an article about the Relative Importance of various skills, at different levels in an IT organization. You can help me by responding to this simple survey. It should take 15 minutes or less of your time:

Please feel free to share this survey with others! I would like to hear from as many people as possible. Share it on Facebook, tweet it on Twitter, post it on your blog, email the URL.

The form is very easy. Just enter a number between 0-100 for the Relative Importance of each category: Technical, Strategic, Interpersonal, Finance. How important is each to your job? Think about the things that you do most often, the types of tasks that you tend to work on every day. Consider how important each area is to what you do. Some areas may be more important to you than others, or get more "exercise" in your day to day routine.

So I can get a baseline on Relative Importance, please make sure the total scores for all categories add up to 100. For example: 10, 25, 30, 35 = 100.

You may recognize this as similar to the Korn-Ferry study, but I am focusing on different skills than what they reported on.

I'll publish my results in a month or so. Look for a followup item here on my blog.

The next generation

In my last two weeks at OIT, I went out for a lot of lunches (and happy hour) with many of you. It was great to get together again for each sendoff and get caught up with everyone.

At one lunch in particular - a small group of us, who all started on the Webteam - I was pleased to realize I had hired many of them. I say "pleased" because, as a leader, it's a very good thing to see the people I've helped bring into OIT move up to bigger and better responsibilities. In the case of the Webteam lunch, I hired one person as a "build coordinator" (sort of what SWA does for migrating projects from Dev to Test, Test to Prod.) That person has since become a developer. Quite the career change, but one that better suited this person's interests and career goals.

Part of good leadership is providing mentoring and coaching to our staff. In working with the Operations and Infrastructure staff, I took advantage of opportunities for coaching, sometimes via coaching buttons. (It also works the other way, from staff to manager.)

I never expect, when I hire an individual into an organization, that the person will remain there for the rest of their career. If I provide good mentoring, and if that person has an interest to grow their skills and career, then it shouldn't be a surprise when they eventually leave the position to do something else. Ideally, they will stay within OIT, or at least the University. But as a leader, I'm happy to have supported them in their career, to prepare the next generation of leaders.

Wrapping that back to my earlier statement, it's been great to look across OIT and realize I helped so many of you step up to the next level. Whether you've moved into a leadership position or simply built up your current position, hope that you, too, will help prepare the next generation.

Looking back

| 1 Comment

Please feel free to join us for a Happy Hour at Town Hall, Tuesday 4:00, to celebrate almost 12 years with OIT!

I thought I'd take a moment to look back on my time here. Here are a few hightlights:

In 1998, I joined the University, working on the Web Team as a Web Production Manager. Our project of the day: build a new web-based registration system to interface with the forthcoming PeopleSoft Student Records system.

Other projects I worked on as part of the Web Team: OneStop web site, online paychecks, ePortfolio, and the Electronic Document Management System.

Around 2000 or 2001, I was asked to move into the Central Computing Operations group, under Nick Choban, to help manage the Unix systems administration teams. I continued to manage the Web Production team, as well. Many of you across OIT probably first met me while I was a manager in CCO.

As the Enterprise Financial System project prepared to go live, I was again asked to transition into a new role: Senior Manager for Operations and Infrastructure. As Senior Manager, I had the opportunity to work on many new projects, with different areas of OIT and with customer departments. The OIT Climate Survey Team was part of this, as were many other projets and initiatives.

I've really enjoyed the opportunity to work with so many of you in OIT - and I look forward to working with you as a customer from the Morris campus.

Free to good home: books

I'm starting to think about the move to Morris, and packing away all my stuff. I have a few O'Reilly books that aren't useful to me anymore. If you want any of them, I'll put them on the beige cabinets across the hallway from my office:

  • Managing NFS and NFS

  • TCP/IP Network Administration

  • Essential System Administration

  • Practical UNIX and Internet Security

  • Java in a Nutshell

  • Sendmail

  • DocBook

  • Motif Programming Manual 6A

  • Motif Programming Manual 6B

Those first 4 have my name written in the inside cover. Just mentioning it.

Understanding risk

A friend of mine from ITLP posted a great item about managing risk, in response to an op-ed essay, Drilling for Certainty, by David Brooks in the New York Times. Since we often discuss balancing risk with the need for change, I thought I'd repost Tom Holub's comments here:

Brooks makes six important points after observing "Humans are not great at measuring and responding to risk when placed in situations too complicated to understand."

As I read through this editorial my thinking was challenged by this list as well as Brooks' quote from Malcolm Gladwell's 1996 New Yorker essay: "We have constructed a world in which the potential for high-tech catastrophe is embedded in the fabric of day-to-day life." So. in the words Brooks uses to close the essay, "It's a challenge for people living in an imponderably complex technical society."

The haunting question for me is simply What are we going to do about it?

This article mentions Richard Feynman's scathing report on risk assessment related to the Challenger disaster; it certainly bears reading if you haven't seen it already:

It appears that there are enormous differences of opinion as to the probability of a failure with loss of vehicle and of human life. The estimates range from roughly 1 in 100 to 1 in 100,000. The higher figures [1:100] come from the working engineers, and the very low figures [1:100,000] from management. What are the causes and consequences of this lack of agreement? Since 1 part in 100,000 would imply that one could put a Shuttle up each day for 300 years expecting to lose only one, we could properly ask "What is the cause of management's fantastic faith in the machinery?"

We have also found that certification criteria used in Flight Readiness Reviews often develop a gradually decreasing strictness. The argument that the same risk was flown before without failure is often accepted as an argument for the safety of accepting it again. Because of this, obvious weaknesses are accepted again and again, sometimes without a sufficiently serious attempt to remedy them, or to delay a flight because of their continued presence.

Of course, Feynman's suggestion that the failure rate would be closer to 1 in 100 than 1 in 100,000 now seems correct; the Columbia disaster was 78 missions after Challenger, and various others have had dangerous complications.

I suppose the leadership lesson is that the courageous action may be to not soldier forward against all odds, but to advocate for caution when the risks have not been properly assessed. On the other hand, change-averse folks often use the existence of unquantified risks to advocate for the status quo, when the status quo often has high levels of known risk associated with it.

Scheduling time

We only have so much time during a week to get our work done. Sometimes, we get pulled in different directions, often involving meetings. Where meetings are in another building, it can become difficult to schedule time appropriately.

It's not often practiced, but defensive scheduling is an important part of managing and scheduling your time. Just block out areas on your calendar where you are unavailable, so people don't try to schedule meetings when you are not able to be there.

Say you've been invited to a meeting on campus, from 10:00-11:00. It would be unrealistic for someone to assume you can attend a meeting back to WBOB immediately at 11:00. So go ahead and reserve some time on your calendar to assume travel. In general, it takes between 20-30 minutes to travel between any two points on the Minneapolis campus - whether you walk, bike, or drive. Leave yourself that extra time to make your next meeting.

I use defensive scheduling on my own calendar. For meetings where I'm going to be on campus, I always put down 30 minutes on either side, marked "travel". If you look at my schedule, you may also notice that I've conveniently scheduled a "meeting" from noon-1:00 so I can do lunch without interruption. When I'm crunched for time to get a particular project finished, I'll book time on my calendar to be in my office, working on documentation.

Morris Campus IT Director

From the announcement:

Jim Hall has been named campus Information Technology Director at the University of Minnesota, Morris. He will begin his duties on Monday, July 12. The Director of Information Technology will:

  • Provide IT leadership to the Morris campus and in the broader University community.
  • Serve as a key member of the Morris campus and Univeristy of Minnesota technology leadership team, which formulates and implements local and institutional goals and initiatives.
  • Partner with the academic and administrative leadership across the Morris campus and University-wide to participate in the creation and implementation of strategic goals and IT initiatives.

Jim has been part of the University of Minnesota since 1998. Currently Senior Manager for IT Operations and Infrastructure at the University of Minnesota's Office of Information Technology, Jim works with IT groups from across the University, giving him a deep understanding of the challenges facing colleges, departments, and the campuses. Jim provides leadership in several key areas of technology, supporting enterprise initiatives that reach across the University system.

Additionally, Jim represents the University of Minnesota to the Committee on Institutional Cooperation, a collective of the Big Ten Universities that regularly meet to discuss technology issues. He currently serves on three subcommittees, exploring issues facing IT operations and infrastructure, storage cloud technology, and research computing.

Outside of the University, Jim is a recognized name in the open source software community. In 1994, he founded the FreeDOS Project, an open source implementation of MS-DOS. As the project coordinator, Jim built an active community of independent developers from across the world. FreeDOS is available as a pre-installed operating system option on certain computer models from HP and Dell.

Jim's appointment comes after a national search and extensive campus interviews. As campus IT Director, he will report to Lowell Rasmussen, Vice Chancellor for Finance and Facilities and the U of M Vice President for Information Technologies and CIO. Jim will support the University's mission through strategic development and implementation of academic, administrative and network technologies.

Jim holds a B.S. in Physics from the University of Wisconsin, River Falls.

We are pleased to welcome Jim to the campus community.

It's true

Yesterday, I held an all-team meeting to announce that I am leaving OIT. I have accepted a Campus IT Director position with the University of Minnesota- Morris. My last day with OIT will be Friday June 18. I'll have a few weeks for relocation, then I'll start at Morris on Monday July 12.

Patton Fast will move into my current role. Earlier this week, I met with Patton to discuss a transition plan. When I accepted the position at Morris, it was important for me to take 3 weeks to ensure a smooth handoff. I will transition my responsibilities to Patton over the next few weeks.

Even if you aren't part of my groups, you may have seen a copy of the email that went out to all OIA senior managers yesterday afternoon. Ann will send out an announcement with the OIT weekly email, either later this afternoon or tomorrow.

It's been a pleasure to be a part of OIT for almost 12 years, and to work with all of you. I have truly enjoyed my time here.

Web Hosting 2.0 update

OIT customers currently can choose from three web hosting services offered by three different teams. Each service is defined and accessed somewhat differently, but are essentially the same. Earlier this year, Arash, Dan, Kevin and I sponsored a project team (Chris A, Chris B and Patton) to align these services, provide current and potential customers with a clearer roadmap, and better define partnerships between customers and our staff.

The team has made initial recommendations that are currently being reviewed by senior management. They include the following.

  1. Create a centralized customer service and support structure. Customers will have a single point of contact for all communications, updates, and support.

  2. Consolidate the current web hosting services into one, and at the same time, create a single hosting infrastructure.

  3. Evaluate third-party solutions to enhance our web hosting options. Web hosting has become a low-cost commodity available from a multitude of providers that may enhance OIT's offering.

  4. Simplify OIT's web hosting environment. Several opportunities exist that will provide greater value to our customers. Some resulting changes will be visible to customers, and others on the support side will not be noticeable.

More information about the new consolidated hosting services will be published this summer.