How Do We Evaluate A University's Performance?
Is the University of Minnesota an Under- or Over-Performer?
Although rankings of universities are certainly a matter for debate, the data available from some of these exercises, especially that of US News, can provide some valuable and thought provoking information for prospective students and their parents. University administrators would do well to think about them and use this information to make rational improvements in the way they operate.
Would you send your son or daughter to an institution that has a much better graduation rate for its students (say 20% better) than would be expected based on the characteristics of an incoming class or would you send them to one that underperformed and actually had a lower graduation rate (4% lower) than would be expected?
One little noticed feature of the 2009 version of US News rankings is an analysis of graduation rates for 2007. After all, it seems reasonable to ask: given an input set of students with certain credentials, how well does an institution do in graduating these students.
Institutions who admit very well-qualified students should have a high graduation rate and those whose standards are somewhat less rigorous might have lower rates.
Thus a new statistic is available described as Overperformance (+) and Underperformance (-).
So to take the example of Harvard: their predicted 2007 graduation rate is 94% and their actual rate is 97%. Their performance score is +3, because students actually graduate at a slightly higher rate than would be expected, based on their entrance characteristics.
According to USNews,
Graduation rate performance " … [is] an indicator of added value [that] shows the effect of the colleges programs and policies on the graduation rate of students after controlling for spending and student characteristics such as the proportion receiving Pell grants and test scores. We measure the difference between the schools six year graduation rate for the class that entered in 2001 and the rate we predict for that class. If the actual graduation rate is higher than the predicted rate, the college is enhancing achievement."
So how does this work out in practice?
Let's first look at the Big Ten Schools in order of the qualifications of incoming students.
Now Northwestern is an anomaly because it is the only private school. Also there is not a whole lot of headroom when your actual graduation rate is 90%+. It is noteworthy, however that the U of M is smack in the middle of the BigTen in terms of entrance qualifications.
So let's put Northwestern aside and redo the list based on performance. The results are quite surprising, I think.
So here we are, once again, in a familiar place at the bottom of the BigTen as far as this graduation statistic goes. Our next closest rival, 7% above us is the University of Iowa. It is noteworthy that there are five schools in the BigTen whose incoming class has a lower predicted graduation rate than Minnesota. So making noises - as our administration has been doing - about the ever increasing quality of our incoming students is clearly not the answer to the questions raised by the data above.
What is it about Penn State, Indiana, and Michigan State that makes them such overperformers compared to Minnesota? In fact all of the schools of the BigTen, except Minnesota, are overperforming in their graduation rates.
Why is this, President Bruininks and Provost Sullivan?