University of Minnesota Extension
http://www.extension.umn.edu/
612-624-1222
Menu Menu

Extension > Youth Development Insight > What does it mean to be driven by data?

What does it mean to be driven by data?

9 Comments

Dale-Blyth.jpg From evidence-based practice to data-driven decision making, the role of data in driving everything forward is becoming omnipresent. As a recovering quantitative sociologist this excites me. As a person devoted to building the field and making a difference in the lives of youth it raises both opportunities and concerns.

Like driving a car, youth work is a navigational sport filled with hundreds of decisions on a moment-by-moment basis. Whether it is the development of the field of youth work or the development of a young person, we process thousands of bits of data to make decisions.

But the data driving these decisions, like many others, are a real mix -- some conscious and quantitative and some unconscious; some rational and some emotional. We drive differently when we are angry than when we are happy.

If youth work is to become a data-driven field, we had better make sure we know what that means and take a strong role in shaping the data available and how they are used.

In driving the field of youth work there are decisions at many levels. Decisions at the policy level about what we fund and support, how and for whom. Decisions on the system level about what quality looks like and who is qualified to practice. Decisions at the program level about what we offer and how it's designed. Decisions at the offering or activity level as a youth worker plans and executes part of a program. And then there are the decisions by each youth, which shapes the experience for themselves and for others.

As several new books point out, from David Brooks's The Social Animal to Incognito: The secrets of the Brain by David Eagleman -- we are learning that more and more of the data driving our decisions are youth-road.jpgcollected and processed unconsciously -- not in some simple rational, conscious and largely cognitive ways.

Over the last few weeks I have had the pleasure of thinking about how we collect and use data on young people's learning, especially but not solely about non-formal learning in out-of-school-time opportunities. Data that can help us to drive decisions on what we do and how we do it with respect to the learning and development of young people.

Learning is about both the journey (the levels of quality in a program, a young person's engagement, and opportunities for youth to contribute) as well as how the journey helps youth get to critical destinations or outcomes.

What mix of data do you think should drive our field and the practice of youth work?

Dale Blyth, associate dean and director

9 Comments

Sam Grant said:

Dale,

I like that you bring up the fact that humans are not rational in their processing of information. I think this should be considered when we collect and present data to our audiences and decision makers. What kind of implications do you think that has for our field?

In past weeks discussions, Jen Skuza has brought up the idea of youth driving learning and the data collected in our system. I would agree with her that involving youth in these decisions is critical to collect data that has direct relevance to our target audience.

I worry that a tension will grow between collecting accountability or impact data and program improvement data and that improvement data will not be seen as important for programs. I believe that we need a healthy mix of both.

Thanks for starting discussion around this important topic.

Dale Blyth said:

Thanks for your comments Sam. Great points. It is at the core of what I was trying to get at in the blog. We see data in too narrow a set of terms and in too silo-ed ways.

I do believe our field could become divided by those who use data to improve practice and those who use data to judge it in terms of impact - from both inside and outside the field That would be unfortunate and is, I believe both unnecessary and unhealthy for the field.

The future of the field lies in finding ways to better collect, analyze and use data in both ways and for both ends -- but to do so in ways that are aligned and connected in both efficient and effective ways. We will not get better impacts if we do not improve and sustain strong practice. We may not even have practice if we do not show it makes a difference -- and not only for academic outcomes.

But beyond even these two ways of using data we need data we also need data on whether we are reaching youth with the types of learning opportunities beyond the classroom that are critical to their learning, growth and ability to contribute. Data on access and how communities, not just youth and programs, are assuring youth are engaged is needed.

In all three cases, data can be a critical tool assuring we are making progress along the journey as well as reaching important destinations.

The implications of all of this as well as the non-cognitive or unconscious elements in decision making for how we present data to our audiences is an intriguing question.

In some ways it reminds me of brain growth in adoelscencce. It is not just, or even primarily, that brains are developing a new ability to get to the right answer. Rather it is that the efficiency, effectiveness, and very nature of the processes are changing. If we are going to optimize the use of data in decision making in our field we have to help people understand data on both the outcomes and the process are important.

One implication is making sure we do not polarize or accept a polarization of what making a difference means in terms of just narrowly defined outcomes. We need to present data on whether and how youth are participating, to what extent they are engaged, why participation and engagement are important, and how they are linked to valued societal outcomes such as learning academically as well as socio-emotionally. We have to stop saying we know youth enjoyed it but not whether it made a difference. Setting enjoyment as an important, but not sufficient, part of whether the program will have an opportunity to make a difference is important.

Similarly, we have to not go so far to the other side and say we only and always have to have youth drive the data (not that that is what either you or Jennifer is saying). Data that is important for decisions needed to be seen from the perspective of the decision maker but informed by an understanding of the dynamics involved in the context. A decision maker may not want to know about non-formal learning's role in the achievement gap but if it is part of the context needed to understand and change it, we have an obligation to help inform the decision maker about the context using the best data we can.

What do others think about Sam's question?
Dale

Josey Landrieu said:

This was a great post and discussion thread. I echo Sam's question and I agree with Dale's response. After having read The Social Animal (I recommend this book to anyone out there) it became even more apparent to me that data must be "informed by an understanding of the dynamics involved in the context" (Dale).

I remember going to a workshop a few years back in which the two researchers presented to teachers about data interpretation and how even those of us interpreting the data must be aware of our own preconceived ideas, biases, unconscious processes that guide our decision making and meaning making when it comes to data. So this is a different question than Sam's and maybe a little off topic but since Dale you brought up The Social Animal and Incognito I can't help by wonder how do we keep our unconscious processes at the forefront of data-driven decision making? Some food for thought.

Thanks again for a great topic.

Joyce Walker said:

Now this is an interesting conversation! I have a couple reactions to your very thoughtful commentary.

Personally I resist the term "data driven." It elevates "data" to the force, the major factor, the driver. I prefer to view decision-making as "informed by data" -- as well as by political and financial realities, mission considerations, funder priorities, common sense, experiences with best practice, stakeholder interests, etc. Those working in our field know full well that there are many considerations that drive decision-making, but today "data" is perceived to have a force and power beyond all else. I say put the thoughtful and relevant data on the table with all the other considerations -- as a source of information -- and then do the tough work of deciding and prioritizing. Dale acknowledged this when he used words like "optimizing the use of data" and "data as a tool" (not the only tool or driver).

Without definition, people too often see "data" as numbers and mysteriously complex analysis. Data is just information that's been gathered in a deliberate way and analyzed or processed for use. it comes in lots of forms -- numbers, words, ideas, opinions, stories. The process for gathering must have integrity and the analysis rigorous. What's important is to have a variety of relevant information on the table when making decisions.

Josey's notion of keeping the/our unconscious processes at the forefront when gathering, analyzing or using data is the most important idea for me. Researchers or evaluators frame the issues by the words they choose, the questions they ask, and the people from whom they seek answers. We see it in political polls all the time. I often say to myself, " Well, if you ask it that way, I'd say yes, but it you ask it this way, I disagree!"

Final thought today: Any data put on the decision-making table needs to pass the test of "Why is this data on the table?" and "What important question does this data answer?" Data should stimulate us to ask more questions like "How is it relevant to our organization?" and "What does it fail to consider that's important to us?"

Framing data as a litmus test -- as the driver -- doesn't help much. Framing data as indispensable, solid, useful information makes sense. Data's an important thing, but not the only thing! Thought - provoking! Thanks.


jennifer skuza said:

There are so many compelling points and questions in this commentary. Great blog!

Here are some of my thoughts on data:

So often, the word “data” is narrowly understood as being “quantitative” – so only including numbers and involving statistical analysis. Like Joyce Walker mentioned in her post, data can take on many forms -- numbers, words, ideas, stories, themes and so forth. Along with that narrow view, in many cases, greater value is placed on “quantitative data” – such as when people say “do you have any hard data?” - they are typically referring to results with statistical significance. If someone comes forth with qualitative data, sadly it is often accompanied with an apology or the question “do you have any numbers to back that up?”

Qualitative and quantitative data each play an important role in “knowing”. Scientific rigor can be applied to the collection and analysis of both – as long as they are understood within their own epistemological context. So, I think it is important to make a shift toward valuing both qualitative and quantitative data. It opens up our possibilities of knowing more about life and informing decisions on what we do and how we do it in regard to learning and development of young people.

E.B. said:

Great conversation, but you assume the people making decisions even value data. If they don't trust science or research, they will ignore it and mistrust it, regardless of the integrity or rigorousness of the research. They will use their ideology to make decisions. The greatest data is of no use when it is ignored.

Dale Blyth said:

Great comments from Jossey, Jennifer, Joyce, and E.B!! Thanks for the thoughtful contributions. Sorry for a delayed response but I actually took a vacation and did no work. I have to say it was refreshing!!

Now onto some reflections on the comments received.

Jossey poses a great question about how we keep our unconsious processes at the forefront in our decision making. I am not sure we ever can fully - that is part of the process being unconsious I suspect. What I believe we can do is decrease the blackness of the box around youth development programs. We can, through various types of research and data, begin to make some of the most important unconsious processes more visible and explicit. Cars originally did not have speedometers to measure speed and it was left to other less conscious data to inform how fast we were going. As automotives became more complex and the way they interacted with other cars under different driving conditions evolved we added more consious tools for giving us data on speed, direction, temperature of the engine, etc. We need to seek to measure and understand more of the aspects of our work so that we can become more consious and intentional about them.

I agree with Jennifer and Joyce that as a field we need to have a broad and inclusive sense of different types of data (such as qualitative and quantitative, experiemntal and survey, etc.) and their respective strengths and limitations. That said, I believe our feild has a long way to go to strengthen both its quantiative and qualitive use of data for program improvement, system assessment, and impact evaluation work. Much of the decision making processes in these areas could greatly benefit from better use of data and better data - and yes, especially better quantitative data that carries additional weight with external stakeholders such as funders. Our field needs to recognize and capitalize on the strengths of quantitive data and appropriate and complex analysis in ways that are currently too rare. That should not be done at the expense of getting and using qualitative data.

This issue of data types and their strengths and limitations is going to be increasingly important if our field is going to be either driven or informed by others use of data about our work but also by our own use. I agree with Joyce that data should inform decisions and most decisions are not and should not be determined by data alone. I used the term data driven to reflect the notion that data is one of the forces shaping our field or driving it in different directions. I am not advocating for data to determine the decision and much appreciate and agree with the notion of data informed decisions.

E.B.'s point that many decision making processes and specific decisions ignore data or devalue its importance is both real and problematic. Data can be used as both a tool to improve decisions and a weapon of destruction. His points remind me of the work of Frameworks Institute (see www.frameworksinstitute.org) regarding the frames people use and how frames affect data. In simple terms, frames eat data as well as help shape them. Data that is inconsistent with a dominating frame is undervalued and data support a frame is more highly valued. Frames also change the data itself as Joyce refered to as he way questions are framed affecting how they are answered.

So where does all this leave us as a field? I believe we have got to get better at defining what matters, measuring it more fully, analyzing it more deeply, connecting it more usefully, and communicating it more effectively. We need to use data to inform the frames we use and understand and access the frames others use to evlauate and fund our work. We need to have better data on some of the basic fundamentals issues of our field such the number of providers and opportuities available in a community, what happens to accessibility when we change certain factors, how do opportunity rich communities differentially impact youth learning and development, how does quality relate to outcomes, how do different approaches to preparation and support of youth workers matter to how they practice, how does participation lead to engagement and engagement to learning in non-formal settings differently or similarly to that in formal settings. The list goes on.

For me the issue is no longer about whether data will inform or drive decisions in our field. That is a given. The real issue instead is the extent to which our field will learn about and from different forms of data and use that data effectively to build a stronger more impactful field moving forward. If data is going to become a driver then we as practitioners, program leaders, system builders, and funders better get some more driver's education when it comnes to data.

What do you think could be done now to improve our field's understanding and use of data?

Anonymous said:

How does youth development affect the economy

Power Point Presentations said:

For me the issue is no longer about whether data will inform or drive decisions in our field. That is a given. The real issue instead is the extent to which our field will learn about and from different forms of data and use that data effectively to build a stronger more impactful field moving forward. If data is going to become a driver then we as practitioners, program leaders, system builders, and funders better get some more driver's education when it comnes to data.

Leave a comment


Type the characters you see in the picture above.

  • © 2014 Regents of the University of Minnesota. All rights reserved.
  • The University of Minnesota is an equal opportunity educator and employer. Privacy