The Ubiquitous Librarian

And the most “satisfying” library is…

I’m working on a mega-LibQUAL+ project, essentially dumping all the data from large universities into an Excel file. This allows for benchmarking: 1) it lets me see who the top libraries are with specific questions (comfortable space, website, etc), and 2.) it let’s me comparatively track my library’s progress over time.

So far I have the undergraduate data for 75 libraries (mostly ARL with a few notable exceptions) and about half the faculty data. I am pulling from 2003 – 2007, entering the most current numbers available. Excel’s “paste special --–> transpose” is great and so is LibQUAL+ Analytics Tool.

LibQUAL+ is not a perfect measurement, but it’s all we’ve got in terms of national benchmarking. And user satisfaction is far more valuable than collection size. (No disrespect to the HYP.) What I like about my satisfaction tool is that it equalizes everything. It goes beyond simply comparing my perceptions against your perceptions, and instead takes all the data points into account.  It asks the more global question: “how full is your glass?” (How satisfied are your patrons?)

If you look through your LibQUAL+ results PDF, you’ll find barometer charts, like this:

Your goal is for that orange line to fill as much of that gray box as possible; the more the better.

And so, looking at all the undergraduate data, which library ranks highest? The home team: Texas A&M. They invented the tool and run it every year, so I have to discredit them, sorry Colleen.

So who is number two? Georgia Tech. I was shocked and had to triple check all the data. But indeed, in terms of pure satisfaction, our participants responded most favorably across the 22 questions collectively. In all sincerity, we got lucky. We conducted our survey just as we finished a major renovation and benefited from good buzz. By comparison, our pre-renovation numbers would have placed us 25 out of 75 libraries.

I’ll be writing more on this once the faculty and graduate student numbers are collected, but here are a few findings:

Overall Customer Service: Cornell
Overall Library as Place: U of Kansas (really it was Texas A&M, but like I said, home team)
Comfortable & Inviting Location: U of Washington
Website: Clemson

Questions about UK Libraries - how do they measure up with us?

I just finished a LibQUAL+ project comparing the Georgia Consortium libraries with other systems: OhioLINK, Network of Alabama Academic Libraries (NAAL), and the Wisconsin System. When looking undergraduates the numbers are pretty similar. There were some small differences here and there, but overall very marginal. However, the thing that was shocking was when I compared the US libraries with SCONUL, which is sort of like the ACRL of the UK.

I can’t post any numbers or graphics because ARL is touchy about that, but here is the gist.

The 2006 SCONUL results featured 20 academic libraries, including biggies like Cambridge and Leeds, and over 11,000 undergraduate responses.

Compared with the US they claim to use their libraries more often. When asked about using resources in the library, 86% indicate daily or weekly use, while the US is around 50%. When asked about using library web resources they were at 77% daily/weekly, while US was between 40-50%. So they use their libraries more frequently, but is that because of study halls or something? (I can’t think of UK libraries beyond Harry Potter, sorry.)

So far so good, I was hoping to use SCONUL has a benchmark of excellence—something for us to strive toward, but… that’s not the case. When you look at collections there is considerable difference between the level of satisfaction of US and UK libraries. With printed materials and journals the US consortiums all had a moderate level of satisfaction—not very high, but not too low either. The UK was just barely adequate, barely.

The physical space results are similar too. Looking at quiet space as well as group space, the US consortiums were consistently solid, whereas the UK undergraduates were much less satisfied. “Inspiring” space was also solid within the US, but was a negative in the UK.

How can this be? I’ve always assumed their libraries were superior to ours, at least in architecture and collections. Do they have closed stacks? Do they enforce the shhhh rule? Have they jumped on the café trend? Do they have as many databases and e-journal subscriptions as us? What ILL like? Just about everyone is in the library weekly, so what’s wrong? Why go if they don’t like it? Is it forced upon them?  Are the UK libraries too high-brow and unresponsive to the shift in patron perceptions?

When comparing public service, overall they did much better than with space and collections, but they are still a noticeably lower than the US groups.

I looked at a few of their websites and the vibe I got was one of tradition and convention. Cambridge’s strategic plan emphasizes remaining a world-class library which they seem to define by collections. Buried in the middle is “improve understanding of the needs of the user community” so they are at least recognizing patron perceptions. On the Glasgow University Library site they state: as a result of staffing shortages the Reading Room will close at 18.30 (Monday-Thursday only) from Monday 8 January until further notice-- that doesn’t seem too good.

And the University of Westminster as some interesting “code of behaviours” including: headphones are allowed in some libraries (Not all?) but there must be no leakage of sounds, quiet is to be observed in the library at all times, readers may normally borrow up to 15 items at a time (just 15?), there is a period of grace of ONE DAY ONLY during which you may return or renew overdue items without penalty and after you return overdue loans you will not be able to borrow or renew or request other items for the same number of days that your loans were overdue. For example, if you return a book five days late you won't be able to borrow, request or renew anything for the next five days. Harsh!

I guess I can understand why the undergraduates might be dissatisfied with their libraries. If anyone has experience or insight with UK libraries I would like to know more. Their results are fascinating to me. Different cultures.

A review of ARL's LibQUAL+ Analytics

I am very excited about the new Analytics tool that ARL is beta testing right now. It's far from complete, but hopefully it is an indication of the direction LibQUAL is heading. I think that offering a GUI to do some advanced analysis is great. The real power behind Analytics though is the ability to work with participant data. The PDF versions from other institutions have always been available, now we do something with it.

There are two tools:

Institution Explorer – allows you to look at participant data back to 2004 and to filter by user groups (faculty, grad, undergrad) as well as by discipline.

Here is an example. These charts show the difference between our engineering and social science faculty in terms of information control:

Two very different perspectives, however the social science faculty sample is only 4 people, so I think the data is exaggerated.

Longitudinal Analysis – allows you to look at an institution's data over time.

Here is an example of how our undergrads feel about our library as a place. The perception has risen, but so has the minimum expectation. When you give people nice stuff, they want more.

 

And that's how I feel about Analytics—it's a good start, but only the tip of the proverbial iceberg with the level of data we need.

Here are some things I'd like to be able to do:

  •  I hope that ARL takes it beyond the dimension level and gives us question by question access and comparability. For example, I want to take the three collections questions and see how the engineering faculty at GT match up with engineering faculty at other schools, as well as a composite average of all engineering faculty participants. This could allow me to say, here is where we stand amongst our peers, and here is where we stand nationally.

•  I'd like a true benchmarking tool. Take group study place for example. Our rating is pretty strong, although I feel we have a long way to go. I'd like to see the top 10 or 20 highest perceptions as well as adequacy means amongst all participants. If USC or Wake or whoever continually rates highly, then maybe we want to see what they are doing, what works for them.

•  It's great that they started to allow custom disciplines, but I want to be able to sort by other factors too—such as gender, class (freshmen vs seniors), as well as frequency of visits. Do the patrons who visit the library weekly/daily have different opinions than those who visit monthly? I have looked around at the raw data of a few schools and unscientifically I can claim that the more female freshmen you have, the better your satisfaction rating will be—but don't quote me on that. It's also very interesting to look at the change that occurs between sophomore and junior year.

•  I wish there was more control toward building dynamic charts and tables and the ability to create personalized templates, so that I could apply them to other schools or groups of schools.

There is a lot more, but this is for starters. Right now, we can use Excel and SPSS to generate these types of things, however we do not have access to the raw data files of other schools—therefore we cannot dig into the discipline level or really do anything very advanced. That's the power of Analytics, it opens the door, but right now it only allows us to place one foot inside.

The only negative thing I can say is that there seems to be some scripting problems with some schools. You get frustrating screens like this more often then you'd think:

   

But overall it's a great step toward the future of library service assessment. Thanks ARL.

Using LibQUAL+ to assess your niche

One of the features I like about the new LibQUAL+ is the ability to customize disciplines. 60% of our students are engineering—so having ‘engineering / computer science' as a base category doesn't really work too well.

This new option let's you narrow down your population- while keeping the total base together too. This has the potential to help us discover areas we should extend outreach and publicity. The next step for ARL is the ability to compare disciplines across all institutions. I would love to be able to compare my engineering patron perceptions with the engineering data from our peer institutions and beyond. Most schools are more rounded than us, so this would really let us see how we stack up—apples to apples, you know?

Discipline level customer service benchmarking-- that's what I'd like to see next.

Hacking Your LibQUAL+ Data: the satisfaction tool

What does it take to make the Cites & Insights Librarian Blog Census ? Perhaps the Ubiquitous Librarian needs to step it up with more posts and more content? What do you want Walt, a pic with me and Natalie Portman hanging out? I'll get to work on that one.

I'm going to step out of character here and actually give you folks something useful, although I guess that is debatable. I'm giving you my LibQUAL+ Satisfaction Tool (.xls)

ARL would probably try and charge you $25 for it and call it something like a Supplemental Patron Satisfaction Percentage Scale, and ACRL would charge you $50 to listen to a podcast about it or to participate in an online class—but it's all free here.

People seem to either love or hate LibQUAL+. I'm a statistics hack, so I dig it. Whenever I talk LibQUAL, there's always confusion about the numbers. My adequacy mean is .43, is that good? My answer: it depends.

Ok, so you have this great lump of data—make it meaningful. Enter your results into the spreadsheet and it calculates the percentage of how satisfied the patron group is with each question. (I guess I could pull together a user guide or something—maybe next week or the week after?). The gist of it is based on a talk I attended by Steve Hiller from UWash. He stated that their goal was to give users 50% of what they wanted. Long story short—this tool takes your data, establishes the zone of tolerance or the range of expectations, which is divided by the adequacy mean. The end result is a nice little percentage, which allows you to say something like, in terms of books, we give our undergraduates 60% of what want, while graduate students are 25% satisfied and faculty are only 5%. This allows you to quickly size up your institution, compare with peer groups, and measure progress over time.

I know this is kind of vague, but you don't have to do any math—just plug your numbers in and let the formulas do the rest.