AIRUM Best Presentation: Linking Library Data and Student Success
Academic libraries, like other university departments, are increasingly asked to demonstrate their value to institutions. This study presents the results of analyses predicting the relationships between library usage and first-year students' retention, college experiences, and academic success at a large, public research university. Usage statistics were gathered across 13 different library access points and outcomes were derived institutionally (e.g., retention, grade point average) and aggregated through student surveys (e.g., academic engagement, scholarship activity).

Krista Soria (University of Minnesota-Twin Cities)
Shane Nackerud (University of Minnesota - Twin Cities)
Facilitator: Kevin Fosnacht

1. Uneven formal academic knowledge and skills.
2. Lack of informal knowledge about being a college student.
3. Inadequate development of self-regulation skills.
4. Impaired self-efficacy and resilience.
5. A mindset believing in fixed rather than flexible abilities.
6. Inability to delay gratification.
7. Impaired ethical judgment.
8. Disengagement from the university environment.
9. Lack of interest in courses.
10. Issues in academic trajectory.
11. Psychological issues.
12. Financial concerns.

I continue to be interested in how we can be more explicit in "lifting the veil" on academic research such as -- how to pick a topic (not just a list of ideas), how to get started on research assignment, how to deal with frustration, how to make mistakes in research, how to troubleshoot problems/deadends, ethics in research, get out of the mindset of looking for a perfect source, etc. I know I assume a lot of background knowledge in the guides I make or sessions I teach. Seems, to me, like #1, 2, 3, 4, 5, 6, 7, 8, 9, and 12 are issues the libraries (along with many other campus partners) could have a role in working on.

• Slides (PDF-1 per page)

Overall our session at ACRL was good. It was interesting to hear what our co-panelists have found and to see the different ways to demonstrate value. Please contact us with any questions.

Do or Do Not...There is No Try: The Quest for Library Value
Saturday 4/13, 9:45 AM - 10:45 AM
Venue: JW, Room: Grand Ballroom 7-8

Program Description:
Librarians today can establish academic library value by linking library services and resources to the missions, goals, and needs of their institutions. Statistical correlations between student library interactions and established learning and retention measures offer one path to demonstrable library value. This panel reveals research results from three libraries that successfully followed this path, shares best practices for "doing" value research, and embraces the quest to both prove and improve the value of academic libraries.

Outcomes:
Explain the concept of correlation and the role of correlation in establishing and communicating the value of academic libraries.
State several correlations between student interactions with the library and learning, retention, and achievement measures.
Form a plan for the investigation of correlations between library interactions and success measures at their own institutions.

Session Hashtag: #acrlcorrelation

In addition to Shane, Jan and Kate, we will be presenting with:
Graham Stone, Information Resources Manager, University of Huddersfield, Huddersfield
David Pattern, Library Systems Manager, University of Huddersfield, Huddersfield
Melissa Bowles-Terry, Instruction and Assessment Librarian, University of Wyoming, Laramie, WY
Megan Oakleaf, Associate Professor, Syracuse University

It will be interesting to hear more what other institutions are doing from the horse's mouth as it were (beyond reading what they are writing), what challenges they are facing and what our colleagues think of all this.

We will add a link to a presentation once it is finished (hopefully later today).

University of Minnesota Twin Cities, Library Data and Student Success Project - Updates

The articles are also linked in the "valueography" . Although the name of this made me roll my eyes a bit--I want to read just about every single article listed. It is amazing what great research is being done in this area from instruction to archives to value.

The Libraries' Role in the Success of First Year Students

Location: Room 326
It's standard advice for a college freshman: Use the library. But how many students take that advice? And what evidence exists that library use has a positive impact on student success? Which of the many services now provided by libraries are most important to students just beginning their undergraduate careers?

Working with the Office of Institutional Research (OIR), the University Libraries collected data on how people interacted with the library in Fall 2011 and Spring 2012. Although limited to access points where an Internet ID can be captured, the resulting dataset provides the most complete picture of library use currently available at any institution. Moving beyond the overall counts, demographic data from OIR allowed the Libraries to aggregate use across categories like college and status.

Armed with data describing library use by first year students, along with the demographic, college environment, and prior academic factors known to influence student success, OIR performed regression analyses to discover relationships among different types of library use and the student success factors of GPA and student retention. Our presentation outlines the observed relationships and details our investigations into how much and what types of library use help new students the most.

Libraries Role Success FYStudents handout.pdf

Certainly beyond academic student success it would be fascinating to explore a connection between library use and workplace success.

Here is a selection of major findings from PIL:

• Employers placed a high premium on graduates' abilities for searching online, finding information with tools other than search engines, and identifying the best solution from all the information they had gathered.
• Once they joined the workplace, many college hires demonstrated computer know-how that exceeded both the expectations and abilities of many of their employers. Yet we found these proficiencies also obscured the research techniques needed for solving information problems...
• Most college hires were prone to deliver the quickest answer they could find using a search engine, entering a few keywords, and scanning the first couple of pages of results...
• Students said they leveraged essential information competencies from college to help them gain an edge and save time at work when solving workplace information problems. Many of them applied techniques for evaluating the quality of content, close reading of texts, and synthesizing large quantities of content, usually found online.
• Overall, our findings suggest there is a distinct difference between today's graduates who demonstrated how quickly they found answers online and seasoned employers who needed college hires to use a combination of online and traditional methods to conduct comprehensive research.

Complete report:
"Learning Curve: How College Graduates Solve Information Problems Once They Join the Workplace," Alison J. Head, Project Information Literacy Research Report, October 15, 2012.

We have also posted the articles in the University of Minnesota institutional repository which means that you can view and read the articles now. Here are some links and abstracts.

Title: Analyzing Demographics: Assessing Library Use Across the Institution

Abstract: In Fall 2011, staff at the University of Minnesota Libraries-Twin Cities undertook a project to measure how often, and in what ways, students used the Libraries' services. Partnering with the University's Office of Institutional Research, the team investigated ways to match library service usage to individual accounts while retaining patron privacy to determine who was - and was not - using the library. With complete data sets, the group was able to determine overall usage rates for undergraduate and graduate students and compare how students in different colleges used library services. This article discusses data gathering techniques, analysis, and initial findings.

Title: Library Use and Undergraduate Student Outcomes: New Evidence for Students' Retention and Academic Success

Abstract: Academic libraries, like other university departments, are being asked to demonstrate their value to the institution. This study discusses the impact library usage has on the retention and academic success of first-time, first year undergraduate students at a large, public research university. Usage statistics were gathered at the University of Minnesota during the Fall 2011 semester for thirteen library access points. Analysis of the data suggests first-time, first-year undergraduate students who use the library have a higher GPA for their first semester and higher retention from fall to spring than non-library users.

As always, please let us know if you have any questions.

Using What You Already Collect: Library Data and Student Success

Please let us know if you have any questions about it! Thanks to the judges and attendees at the Library Assessment Conference!

The data we gather certainly does not discriminate towards any particular type of U of M Internet ID. We gather all of them and don't remove any based on whether they are a student or not. So, in other words, of the 59,722 unique Internet IDs we collected in the Spring semester 2012, how many of those represented faculty and staff? Can we get at that data?

Thanks to the U of M Office of Institutional Research, yes we can! The following table shows usage percentages for user populations other than students for Spring 2012:

Staff type	Usage percentage
Faculty	73%
Grad Assistants	93%
Civil Service	23%
Professional	51%
Administrative	34%

When sharing this data, people usually are amazed that we have registered such a high percentage of faculty use, or they are amazed that 100% of the faculty don't use the library. Some people are even disgusted that 100% of the faculty don't use the library. Keep in mind that we do not have a perfect count of people or Internet IDs that use the library. We can only analyze the Internet IDs we can successfully gather. There are some library services that do not require an Internet ID (reference desk transactions) and if you are using a campus IP address you can enter many of our resources through a bookmark or Google search. Having said that, we are reasonably confident that if you are a library user, at some point during the semester we are going to capture your Internet ID. However, if we had perfect usage tracking mechanisms that gathered every use for everyone at the University, these percentages would likely be higher. But I'm guessing not by much.

We can also determine what colleges and departments have the highest and lowest usage. For example, 90% of the School or Nursing faculty made use of the library during Spring 2012, while only 30% of the School of Dentistry did. The School of Dentistry's 30% number was by far the lowest percentage of use by faculty. No other school or department went below 69%. Other high percentage use for faculty was seen by the College of Liberal Arts (87%), the College of Science and Engineering (77%), and the College of Food, Agriculture, and Natural Sciences (76%).

That's all for now. We are having fun swimming in all this data!

So, in the library access areas we measured there were 146,126 Internet IDs (61,195 unique) captured at the frequency of 1,548,209 transactions. With this data, we found that 77% of undergrads made use of the library during the Fall 2011 semester and 85% of grad students.

Now let's take a look at Spring 2012:

In the Spring 2012 semester, we captured 1,646,836 transactions, almost 100,000 more than in the Fall 2011 semester. As might be expected, the increase in usage was mainly seen in website, database, and e-journal usage.

You'll also note that we attempted to count transactions in more library access categories such as students who take our introduction to libraries instruction through their Writing course, or students that had a consultation with our Archives and Special Collections department. These new library access points did not have a huge impact on our overall numbers, but they are nice to have to compare and to determine who uses these services the most.

One thing this chart does not list is the number of unique Internet IDs captured in the Spring 2012 semester. That number is 59,722. So, slightly less unique Internet IDs in the Spring 2012 semester than in the Fall. Some of that difference can certainly be attributed to student graduation and attrition. In fact, from Fall to Spring we saw 43,015 returning users and 16,707 new users. Some of those new users were most certainly new undergrads and grad students just starting at the U of M in January.

In the next few days we will write some entries regarding the demographics of this Spring 2012 data, namely who are these students and what colleges and majors are represented. But as a little teaser, what we have found out so far is that 78% of undergrads used the library in Spring 2012. That is one percentage point higher than Fall 2011. 87% of graduate students made use of the library in Spring 2012. That is two percentage points higher than Fall 2011.

More to come soon!

"Seeking Evidence of Impact (SEI) is a program lead by the ELI community to find current effective practices that would enable the collection of evidence to help faculty and administration make decisions about best practices to adopt, invest their time, effort and fiscal resources in....We hope to bring all types of higher education institutions and professional associations into a conversation on this theme. We envision an inclusive discussion that includes faculty members, instructional support professionals, librarians, students, and research experts in a collaborative exchange of insights and ideas."

http://www.educause.edu/eli/programs-and-resources

Have any libraries been part of this so far?

As I was reading this over I like the mention of "making decisions." Sometimes I feel our need to collect data makes us lose sight of the reasons and next steps. For my concrete brain...the data collection and analysis should hopefully should be driven by decision making. I also think this organization (EDUCAUSE/ELI) is doing some great work on behalf of Libraries. I think many users lump the Libraries into the broad category of "technology" even if that isn't necessarily how we see ourselves.

I have been thinking of ways to share our results with new students during our efforts at Orientation and this might hit the right note.

Read at: http://www.waldenlibrarynews.com/blog/2012/6/13/is-using-the-library-good-for-you-yes.html

Like many institutions, the University of Minnesota uses QuestionPoint for chat and e-mail reference. When a patron poses a question via chat, the inclusion of their e-mail address is not required, and using their designated University of Minnesota e-mail is not specified. However, at least we have some data to use. Many other institutions use services such as Meebo, which use anonymous chat patron handles, making gathering data much more difficult. QuestionPoint transactions are accessible for only 90 days, which means we need to go in and retrieve the data at numerous points during the semester.

How do we gather in-person reference transaction statistics without impeding privacy and maintaining a welcoming atmosphere? Please post suggestions in the comments below.

These are high-touch and potentially some of the highest impact services we provide but our data collections techniques are almost non-existant for these face to face interactions.

What we did in Fall....
For Reference we gathered email and chat transations from Questionpoint--the software we use to facilitate these services.
BUT...
-email addresses aren't required
-U of M email address aren't required
-no data on desk transactions in any location
-no data on consultations or small group sessions with librarians

For Instruction we gathered our workshop *registrations* and we gathered *class lists* of courses who had a one-short/guest lecture sort of session.
BUT...
-not matched up with actual attendance or any last minute drop-ins
-missing entire classes due to confusion over section, professor, etc.
-missing non-course session (e.g. dept. orientation sessions, clinical groups, etc.)
-challenging to make multiple sessions obvious in reports

Bottom line: We are missing tens of thousands of interactions and uses of our libraries.

So...I have been asking myself....

• Can we ask students to swipe with each session or consultation or question at the desk?
  
• How do we adequately reflect this high touch, nuanced interactions?
  
• How do demonstrate the range of activities in these categries (from 5 minute interaction to multiple session class time or hour long consultation)?
  

Do you have any ideas?