Building Extension's Public Value

An Extension program creates public value when its positive impact extends beyond the program participants to the greater community. Documenting that a program has created public value, therefore, requires measuring system- or community-level impacts. How often do evaluation studies measure the kinds of impacts that can be classified as public value? In an article in the most recent (April 2012) Journal of Extension, Jeffrey Workman and Scott Scheer try to answer that question. The article, titled "Evidence of Impact: Examination of Evaluation Studies Published in the Journal of Extension," examines program evaluations published in JOE to determine the levels of impact they reached. They considered two impact hierarchies. In Bennett's Hierarchy (from "Up the hierarchy," by C. Bennett in the March 1975 JOE) the highest of seven levels is "end results," which would include such community-level impacts as a stronger economy or improved environmental conditions. In the Logic Model, these kinds of changes in social, economic, civic, and environmental conditions are called "long-term outcomes." Workman and Scheer found that about 5.6 percent of the studies they surveyed reported impacts at these levels.

The authors conclude that "more higher-level evidence of impact is needed." They write that Extension's "ultimate goal is to remain relevant and of value to the public. The strongest method to demonstrate relevancy and public value is to document "true impact" (end results/long-term outcomes)."

Do the authors' findings ring true for you? Are only a small percentage of programs able to demonstrate public value-level impacts? Is it because few programs are achieving that level of impact? Or because the resources have not been available to measure programs' long-term impacts?

Posted by Laura Kalambokidis at 11:45 AM | Permalink | Comments (2)

Many of you use the University of Wisconsin Extension logic model to guide program development and evaluation. Below is my first attempt at mapping the elements of the logic model to a public value message.

The "short-term" or "learning outcomes" in the logic model are a means to achieving the behavior changes and outcomes contained in the public value message. These learning outcomes lead the way to public value--and we must identify and measure them--but they are not the focus of the public value message. A skeptical stakeholder is unlikely to be persuaded of a program's value be hearing that a participant learned or became aware of something. The stakeholder is concerned with what the participant actually did with that knowledge.

What I call "changes" in the public value message are called "intermediate" or "medium term outcomes" in the logic model. What I call "outcomes" are the logic model's "long-term outcomes" or changes in conditions.

It seems to me that public value typically arises from a program's long-term outcomes. In some cases, a program's logic model will already include the outcomes that a stakeholder cares about (public value). In other cases, the public value exercise will tell us which additional outcomes we need to monitor--how we should extend the logic model--in order to substantiate our public value messages.

I believe that the public value approach must work hand in hand with program evaluation: it is through good program evaluation that we are able to make credible statements about our programs' public value.

Posted by Laura Kalambokidis at 4:17 PM | Permalink | Comments (0)