University of Minnesota Extension
http://www.extension.umn.edu/
612-624-1222
Menu Menu

Extension > Food, Agricultural and Natural Resource Sciences Intranet > Thinking beyond surveys

Thinking beyond surveys

| Leave a comment

Surveys are not the only way to gather evaluation data for your program! While many of you use surveys as your primary data collection method, it's worth thinking about other options, such as interviews, observations, focus groups, secondary data, and document review. In fact, using more than one method to answer your evaluation questions not only will give you greater understanding of your program, but can also increase the validity of your conclusions. Ask Whitney (mered025@umn.edu) to help you determine how to design an evaluation using multiple methods.

Evaluation data collection methods

More than just surveys!
Method Advantages Disadvantages
Survey
  • Relatively inexpensive
  • Online surveys allow you to invite many people to participate at no additional cost
  • Online survey programs often analyze at least some of your data for you
  • Can gather the numbers you need for the Federal Report
  • Good items are harder to develop than you think!
  • Hard to understand participants' thought process and motives
  • Low response rate and response bias can decrease data quality
Interview
  • Can provide in-depth understanding
  • Researcher can clarify questions and ask for more detail when needed
  • Can collect stories that illustrate your survey data, which is needed for the Federal Report
  • Time consuming and relatively costly
  • Analyzing data can be subjective
  • Cultural norms and rapport with interviewer may prevent respondents from answering honestly
Observations
  • Relatively unobtrusive
  • Relatively objective
  • Hard to identify indicators you can observe
  • Participant behavior may be influenced by presence of observer
  • Can be time consuming
Focus groups
  • More cost efficient and time efficient than interviews
  • Can provide in-depth understanding
  • Researcher can clarify questions and ask for more detail when needed
  • Logistics may be challenging
  • Group setting may influence responses
  • Requires strong facilitation skills
  • Analyzing data is time consuming and can be subjective
Secondary data (e.g. census data)
  • Unobtrusive
  • Don't have to spend time collecting data
  • May be hard to access data or to find relevant data
  • Quality of data influenced by source
Document review
  • Unobtrusive
  • Relatively inexpensive
  • May be hard to gain access to needed documents

-Whitney Meredith, evaluation

Leave a comment

  • © 2014 Regents of the University of Minnesota. All rights reserved.
  • The University of Minnesota is an equal opportunity educator and employer. Privacy