University of Minnesota Extension
http://www.extension.umn.edu/
612-624-1222
Menu Menu

Extension > Youth Development Insight > What makes an evaluation report compelling? (or not)

What makes an evaluation report compelling? (or not)

4 Comments

Samantha-Grant.jpgOkay, time for true confessions here. How many research or evaluation reports do you have sitting on your desk? You know there was blood, sweat, and tears put into the creation of those documents, but somehow you don't feel compelled to read them. Why not?

I'm willing to guess that the answer is either: A. you don't have time or B. the reports are way too boring. (By the way, reason A is just a disguise for reason B.)

The truth is that many evaluation reports are dull, but there are also great ways to spice them up by focusing the message and using pictures and stories to illustrate points. I have set a goal of unlearning some report-writing habits to make mine more interesting, and thus more likely to spur action.

pencil-chart.jpgA new learning experience that I'm taking part in is the American Evaluation Association's eStudy series: An Executive Summary is Not Enough: Effective Reporting Techniques for Evaluators. In it, Kylie Hutchinson sets out to teach practitioners to create meaningful reports.

One of the reasons for ineffective reports may be bad writing habits. In an editorial, Jane Davidson argues that our social science training has taught evaluators some behaviors that need to be unlearned. She notes four ways:

  1. Including models or theories without connection
  2. Leaping to measurement too quickly
  3. Reporting results separately by data source or type
  4. Organizing reports like a master's thesis

In my experience, I've noticed that we train evaluators and researchers to design reports without thinking about our audiences. But if the end user isn't involved in the creation, evaluation data will never be used for change. Davidson states, "Evaluations cannot produce useful answers unless they actually ask useful questions in the first place!" I wanted to stand up and cheer for that line. Asking useful questions requires that an evaluator is in tune with stakeholder needs.

In thinking about this, I realized that as educators, our formal training may have created some behaviors that need to be unlearned. Take for example the youth worker who has training in formal education. What works inside a classroom may not translate to an after-school program or a summer camp. Academic training is important, but we need experience to be good at our jobs.

How about you? Have you read an evaluation report that really drew you in? What was it about that report that made it compelling? Can you see the need for unlearning bad habits?

-- Samantha Grant, assistant Extension professor, program evaluation

You are welcome to comment on this blog post. We encourage civil discourse, including spirited disagreement. We will delete comments that contain profanity, pornography or hate speech--any remarks that attack or demean people because of their sex, race, ethnic group, etc.--as well as spam.

4 Comments

Kylie Hutchinson said:

Yeah! Another convert! Right on Samantha!

Ironically, reading your blog I can tell you're a great writer, so you probably write a pretty mean final report! But I'm sure an even better newsletter or two-pager!

Samantha Grant said:

Thanks Kylie. I think blogging is a great way to get used to writing in a shorter, more focused format that translates well to report writing.

Everyone, take a peek at Kylie's website: www.communitysolutions.ca
for some excellent resources on reporting.

A great tip that Kylie shared during a recent session was to shut your computer off to get creative. (So finish reading my post first, of course, and go get creative!)

Thanks, Samantha, for picking up on the "unlearning social scientist habits" article!

I'm not sure whether it's simply "not" thinking about audiences so much as assuming the audience is a peer reviewed journal rather than a non-academic manager. Either way, not good!

Your comment about dreary evaluation reports reminds me of an impromptu composition from the 2011 Aotearoa New Zealand Evaluation Association conference, sung to the tune of Pink Floyd's "Brick in the Wall", where the middle verse went like this ....

We don’t need no phonebook milestones***
We don’t need no weighty tomes
We don’t need no random diatribes
Hey, researchers, leave our work alone!
All in all it’s just a waste of time for us all

...
*** phonebook milestones – reference to telephone book-sized contract milestone reports that have been painstakingly written, largely unread, and seldom used

:)

The full three verses went up on the Genuine Evaluation blog as a Friday Funny: http://genuineevaluation.com/the-friday-funny-on-open-tenders/ - enjoy!

Jane Davidson

http://RealEvaluation.com
twitter: @ejanedavidson
LinkedIn: http://www.linkedin.com/in/realevaluation

Blogging with Patricia Rogers at
http://GenuineEvaluation.com

Samantha said:

Thanks Jane. Does someone else want to jump in with some song lyrics? I think this could be a first on the blog.

Leave a comment


Type the characters you see in the picture above.

  • © 2014 Regents of the University of Minnesota. All rights reserved.
  • The University of Minnesota is an equal opportunity educator and employer. Privacy