Superficial Intelligence

Today is the last day of this week of research papers and comic strips. The paper I chose for today is very different from all the others we read in class. It was described by the authors during the conference it was presented at as the most controversial paper of such conference (CHI 2008). Just to give you a quick background, usability evaluations are a very useful type of experiment used extensively in HCI where interfaces and devices are tested with users to find out how “usable� they are. On to the paper then!

Paper: Usability Evaluation Considered Harmful (Some of the Time) (reference at the bottom of this post)

What in the world did they do?

Usability evaluation has become so important in the HCI field that lately it is very hard to publish any research if you don’t do a usability experiment. The authors of this paper argue that although usability evaluation is very useful, it “can be ineffective and even harmful if naively done ‘by rule’ rather than ‘by thought’�. According to the authors, sometimes usability just isn’t what you need.

What’s so cool about that?

This paper is basically a critique. It was called “controversial� because it is going against the currently accepted notions that good HCI research has to have usability experiments.

Did anything worthwhile come out of it?

They came up with various cases where usability evaluation can be harmful. One of these is when prototyping a really innovative idea. Sometimes, new ideas just aren’t ready for users at first. It takes many iterations to arrive at the final, polished version that really does make a difference in users’ lives. If usability testing is done on these early prototypes, a good idea could be quashed before it has a chance to mature. It’s up to the rest of the research community now to decide if they want to do a better assessment of whether this technique is helpful for their experiments or if they just want to keep going at it with usability evaluations, regardless of its usefulness.

Why should I care?

Although many readers of this blog probably won’t be doing usability evaluations in the near future, this affects all of us. The better HCI researchers do their stuff, the better and faster new technology reaches the rest of us.

We can even take this a bit further. How many times do we follow patterns and methods mindlessly just because they have worked before? Or because we are lazy? Or because it’s what others expect us to do? Take some time to assess why it is that you do things the way you do.

Now, on to today’s comic strip. (Click on the image to enlarge it)

Reference: Greenberg, S., Buxton, B., Usability Evaluation Considered Harmful (Some of the Time), Proceedings of the SIGCHI conference on Human Factors in computing systems, April 5-10, 2008, Florence, Italy