University of Minnesota Extension
http://www.extension.umn.edu/
612-624-1222
Menu Menu

Extension > Youth Development Insight > Who can assess the quality of a youth program?

Who can assess the quality of a youth program?

12 Comments

Thumbnail image for Samantha-Grant.jpgCan youth and volunteers effectively assess program quality?
Does it matter if adult volunteers or 4-H staff are paired with youth to complete assessments?

Early results from our Minnesota 4-H Quality Improvement Study suggest that youth and volunteers can indeed assess quality and can work with local 4-H clubs to improve their programs. We have also learned that, for the most part, whether youth are paired with adult volunteers or staff made little difference.

Traditionally, program quality is assessed using reliable, trained assessors. Taking that model to scale in a youth organization with more than 130,000 youth would be impossible, so our study sought to discover how to measure quality in a youth and volunteer-led organization.

The study design called for 40 coaches (20 youth and 20 adults) receiving a two-day training on how to observe program quality using the Weikart Center's Youth Program Quality Assessment (YPQA). Participants were grouped into 20 pairs: half of them one youth and one adult volunteer; the other half one youth and one 4-H staff member. Each youth-adult team assessed at least two 4-H clubs for a total of 40 4-H clubs observed.

Source 4-HCover.JPGThe goal of the study was to investigate the nuances of using an alternative assessment approach. It was not designed to measure the actual extent of quality improvement attained or compare trained assessors with volunteer assessors. We viewed the process of engaging, as a member of the 4-H program, as a step toward building a quality improvement system.

Results revealed that:
• Youth and adults played an equal role in the observation and initial assessment scoring.
• Youth involvement in the data collection was seen as extremely valuable.
• Observed 4-H clubs were receptive to this opportunity and viewed the assessment process as a positive one.

We at the Extension Center for Youth Development are taking this work to scale in 4-H clubs around Minnesota. The project represents a fine expression of the University of Minnesota's land-grant mission: to bring research and practice expertise together to support local community needs. For more on program quality and how to measure it, visit our website's program quality research page.

Is your youth program finding ways to measure program quality? How are you doing it? 

Samantha Grant, assistant Extension professor, program evaluation

Read the Preliminary Minnesota 4-H Quality Improvement Study


12 Comments

Deborah Moore said:

Sam,

A really consice capture of the research. As a member of the research team, what are the next burning questions on quality assessment that you would like to explore?

Samantha Grant said:

Deborah, This is the start to practicing my elevator speech. Next time that you see me, test me!

As far as next questions...I always have so many! I really want to look further into the experiences that youth had in being engaged in the process and learn more about the strengths and weaknesses. I'm also interested in learning how to best bring this model to scale in the 4-H organization.

If you had a crystal ball, where would you see the Center for Youth Development going with quality in the next 10 years?

Joanna Tzenis said:

Hi Samantha,

Thank you for sharing this. I am currently working with the Minnesota CYFAR project and we are always seeking ways to improve not only the assessment of our programs, but also ways to foster a greater sense of ownership among the youth –especially as we work to sustain the CYFAR programs past the life cycle of the grant. We currently use youth surveys and focus groups to assess the quality of our programs. We just recently started having focus groups and found that it is not only a fantastic method to collect rich and meaningful data, but that the opportunity to engage in a conversation about program quality heightens youth’s enthusiasm about their program.

That said, could speak more of the benefits of having youth engaged in assessing their own programs—(elaborating on the research result that “Youth involvement in the data collection was seen as extremely valuable.”)? Your research report referred to youth taking on new responsibilities and a “forward momentum” that was created as a result of being engaged in quality assessment. How do we keep that momentum going?

Samantha Grant said:

Joanna,

Those are million dollar questions!

I would encourage you to read the full report, which will be published soon. Youth reported that they were equal partners in the assessment and action planning. Their adult counterparts commented on the new perspectives that youth brought to the process and the ways that 4-H clubs responded best to their advice. Youth were definitely viewed as experts.

We want to learn more about youth's experiences, and that is a proposed step two in the process. From the larger literature on youth action research and youth participatory evaluation, we know that youth gain new skills and feel increased connections to their community after engaging in these types of projects.

Keeping the momentum going is always difficult, but I think one of our best assets in 4-H (and many youth programs) is the central role that youth play in the development of our programs. When youth feel like they "own" the project, they are more likely to be the ones pushing for more.

Do you see the benefit of incorporating quality assessments with youth involvement in your CYFAR programs?

Anonymous said:

Hi Sam –

Thanks for the synopsis of the early results from this study.

Like the goal of this study, I too am interested in exploring the nuances of using alternative assessment approaches. "Creativity" combined with "rigor" in evaluation can lead to very compelling results and new understandings about program quality. I look forward to reading the full report.

You were a member of the research team. What nuances did you discover in the process of preparing youth to be quality coaches?

Sam Grant said:

I should probably defer to the other members of the team who were involved with the implementation of the training. Unfortunately, I've the one with the blog post, but I can say from our conversations that there were some nuances. One of the biggest ones was simply getting youth to the training, because our 4-H youth are so busy. Finding time to complete a two-day training can be daunting! Other than that, youth took to the material very well, and from my end on the research/data entry standpoint, assessments from youth had lots of great details (and often times more than their adult partners.)

Thanks for your comment! Are you building program quality in your program?

Dale A. Blyth Author Profile Page said:

Sam,
Thnaks for blogging about this important effort to experiment with deeper and richer involvement of youth in assessing quality. Great to see it reaching fruition and starting to explore the next questions to study. It is probably imporstn to note that youth assessors assessed clubs other than their own (by design) and yet also talked about bringing their learnings into their own clubs as well. One of my questions for the future is what happens when this appraoch gets in the "water" of a youth program and permeates the way it thinks about itself and how it does things? Taking it to scale, as Minnesota 4-H is doing opens up a world of important questions that could be addressed.
Dale

Pam Larson Nippolt said:

The results reported here by Sam Grant illustrate that youth are willing and effective in making contributions through youth development programs - in this case, through observing and assessing the quality of the program setting. The concept, and the values that fuel the concept, are intriguing and so relevant for the field right now. Sam brings a key issue for large organizations to the forefront - how can this important effort be taken to scale while keeping the momentum that comes from involving individual youth in important leadership roles?

My thinking goes toward technology on this - what is the best mix of "face to face/in person" with technology delivery in order to keep growing these quality practices - at the same time maintaining the quality of the trainings, the assessment process, the coaching? Another million dollar question and one that will be watched closely as Minnesota 4-H broadens these practices across the state.

Margo Herman said:

This is a terrific example of building long term capacity for quality in youth programs! In many realms we need youth to carry forth and improve upon what we know. This project provides youth with new knowledge and some skills for applying the knowledge in a concrete way. I also like the model of pairing a youth with an adult to assess programs as a tangible example of youth/adult partnerships. I find a great deal of optimism in how this is unfolding. Thanks for writing about the study and keep us informed about the results!

Samantha Grant said:

Dale, Pam, and Margo,

I'm pleased to see the positive reactions to our study of youth program quality. This tells me that it's the right time to be investing in building quality programs for youth. I hope the momentum in 4-H will spill over into other youth serving organizations. If the field was behind this movement, as I think they are moving in this direction, it would be beneficial for youth in all of their out of school time opportunities. What a fabulous way to demonstrate the value of the Land Grant University!

Jennifer A. Skuza, PhD Author Profile Page said:

Hi Sam -

For some reason I came up as "Anonymous". I think it is adjusted now. Anyway ... thanks for your response. I'd like to continue the conversation with you about involving youth in assessment. :-) It is important to uncover the nuances in preparing youth for assessment and even in collecting data from youth.

Josey Landrieu said:

Great blog post Sam! and insightful comments. I think one of the key elements of involving youth and adults into assessing quality is providing the opportunity to have ownership of the program and its benefits.

Allowing youth and even volunteers to be do something that they are often not asked to do is an opportunity to be invested in what they are doing day-to-day (running the program, participating in it, etc). Many of the comments I think talk about this...I thought I would put forth a term that might be useful for us as we conduct evaluation and research: How much ownership do program or study participants have in the process?

Best,
Josey

Leave a comment


Type the characters you see in the picture above.

  • © 2014 Regents of the University of Minnesota. All rights reserved.
  • The University of Minnesota is an equal opportunity educator and employer. Privacy