3. Assessment
• Measurement of services & resources
• Student Learning Outcomes!!!!
Annual Reports
• Checkouts
• Purchases
• Instructions
• Use, etc.
Surveys
Types of
StatisticsLINSCHEIDLIBRARY
4. Began working with the University
Assessment Committee in 2006
• All about student learning outcomes
• Struggle for libraries
• We don’t see the end results
from our instructions
• We don’t grade papers
• ECU focused on reference &
instruction services, and high-
quality resources.
Assessment
LINSCHEIDLIBRARY
6. • Numbers made no sense
• No trend could be detected
• Focus was on big picture
• Entire collection numbers
• How does that fit into the
university?
Shell GameLINSCHEIDLIBRARY
geocaching.com
7. Moved to total Student Learning
Outcomes (SLO)
• SAILS – university buy in
• UNIV 1001 Freshman Seminar
• UNIV 3001 General Ed Seminar
• In-house Assessment tools
• SAILS 1.2 Developing appropriate
search terms
• SAILS 3.2 Articulating evaluation
criteria
http://ecok.libguides.com/assessment Tools
Assessment
RefocusedLINSCHEIDLIBRARY
8. Freshman compared to Juniors:
• Instruction numbers
• Can now help in decision making
• What SAILS criteria were weak – beef
up instruction in those skills
• What SAILS criteria improved – did
instructions during those two years
correlate?
• Provide results by discipline to academic
departments – lead-in to future
information literacy sessions
Results
LINSCHEIDLIBRARY
9. • More numbers!
• Still big picture with no correlation to
academic departments or the
university
• No structure among multiple library
departments
• Refocus was needed, desperately
Annual ReportsLINSCHEIDLIBRARY
10. First focus - to create a template
• Provost’s requirements
• Work plan items
• Program data & accomplishments
• Data in context
• Departmental Projects
• Personnel
• Summation
Template
LINSCHEIDLIBRARY
11. No more big picture
• Break down into academic
departments
• Group departments into colleges &
schools
ILS (Innovative)
• PCODE2 = Classification
• PCODE3 = Major (based on
Admission’s code)
• PTYPE = Type of Patron
Data in ContextLINSCHEIDLIBRARY
12. Master file in Excel of P codes & colors:
PCODE3LINSCHEIDLIBRARY
13. The big picture is now in smaller, more
digestible chunks.
No longer what the library has done, but
who and how it is being used.
• Provides the needed connection to
the university & individual academic
departments.
Now easier to compare these numbers to
external numbers, i.e., enrollment.
ResultsLINSCHEIDLIBRARY
14.
15. Web Access Management (WAM)
• Tracks connections to databases from
non-institutional networked computers
• Laptops
• Smart phones
• Home computers
Tracked since 2007-2008
• Never used data except to report
number of connects
• Does report by PCODE3
Unused DataLINSCHEIDLIBRARY
16. Vendor supplied COUNTER statistics
• Proves resources are being used
• Doesn’t tell you who uses them
• Takes a lot of time gathering them
Problem: How do connects (WAM)
intersect with COUNTER statistics?
Solution: Excel and percentages
Aggregate
Vendors
LINSCHEIDLIBRARY
17. Number of database connections by
major for AY12-13
• Majors listed with ‘total’ indicate more
than one degree available
• Connects not associated with a
college/school are segregated out
• Not included in totals/percentages
used in calculations
• Allows comparison of apples to
apples
Handout 1LINSCHEIDLIBRARY
18. Number of connections compared to full-text
use
• Eliminated any databases not providing full-
text
• Inserted two rows between databases
• First: divided each major’s total connects
by the total connects of all majors to
come up with a percentage of connects
• Second: took the percentage of connects
and multiplied by the total full-text
(COUNTER) for that database
• The total for the percentage row for all
majors = 100%
Handout 2LINSCHEIDLIBRARY
19. At-a-glance, you can see what databases are being used by each academic
department.
Reminder: these statistics only track connections made by non-institutional
networked computers.
For AY1213, total connections = 1,115,228 with 249,849, or 22.40% from
institution networked computers.
For AY1213, total connections by major = 762,921, or 68.41%. Remaining
connections were made by non-academic department entities.
20. Using the previously
mentioned master Excel
file of patron codes, I
can quickly insert
college/ school codes to
group academic
departments.
21. Connects all the pieces into one picture
• Enrollment numbers provided by
Academic Affairs
• All other numbers provided in annual
reports
• Findings:
• AY1213 – not all library statistics
gathered based on PCODE3, or
major
• AY1314 – steps have been
implemented to gather as much as
Master TableLINSCHEIDLIBRARY
22. At-a-glance, the correlation between size of college/school and parts of
the library.
• It’s no longer that the library had 9,000+ checkouts, but that CEP had
24% of the checkouts and they are 25% of the total enrollment.
• Easier to see where there are strengths and weaknesses.
23. Master Table Exploded:
• Each item is coded to a library department.
• All numbers entered come directly from departmental annual reports.
• Highlighted areas weren’t counted by major for AY1213.
• All areas are now being counted by major for AY1314 thru use of other
ILS codes or Excel functions.
Other Master Table – designed the same & includes all OTHER statistics
24. Libraries need to make internal statistics
correlate more to the university
• Refocused assessment to true
SLOs, providing individual results to
academic departments.
• Refocused annual reports to also
provide individual results to academic
departments.
Recap
LINSCHEIDLIBRARY
At ECU, there are three major reasons we keep statistics. Today, I’m going to talk about the first two.
This was our first venture into ‘connecting’ to the university. In other words, doing things that only academic departments did.We were the first support unit to undertake assessmentNow, all are in the process of beginningLibrarians are faculty – assessment further connects us to ‘teaching’ faculty
This is what our high quality resources looked like. After three years, we had to make some decision.
Who really cares that the library checked out 10,000+ items or had over 84,000 full-text uses.Even though we were doing university things, we still felt dis-connected.Our first UAC review proved that – they didn’t like anything we had done since there was no connection to SLOs.
University (Director of Assessment) agreed to pay for SAILS.Instructional Services Librarian maintains university account, along with distribution of tests and results.
Results are shared with academic departments to help market our instruction program.Teaching faculty have responded favorably to the shared results.Instructional Services continues to monitor trends in SAILS results and suggest curriculum to improve areas of weakness.
Dean approached me last summer to development guidelines.She was tired of cutting, pasting, and having to reformat for font and table size.Process took her more time than writing the actual report.
Our hopes were that now everything was connected to an academic department.Using Excel, it’s easy to group the departments into colleges/schools.We also report by classification and patron type, but I’m only going to talk about the Major code.
Each department and their code is listed.Each department given the color of their designated college/school.Easy to sort by college/school to group departments.
For example, the College of Education & Psychology make up 25.51% of ECU’s enrollment.24.04% of academic circulations22.03% of academic collections31.75% of electronic resource use (connections only – we’ll talk more about this in a minute)Only 13.39% of academic library instructions
Here are some examples of our graphs and charts.Top left example – other should be orange
WAM allows for access to resources from any internet connection.User does not have to setup the proxy.Based on IP.For now, only non-campus networked computers.Considering moving to all connectionsWould make patrons have to loginWould eliminate community patrons
Yet another spreadsheetMonthly gathering of statisticsWho else does this?
Just like in the master table, academics are compare against other academics. Faculty/staff not coded with academics – limitation of university systemGeneral Studies degree not connected to any academic departmentNon-student – special programs on campus (Upward Bound) with loginsNon-verified (all campus networked computer logins)
Only used full-text providing databasesDepartmental connections divided by total connections (for all academic departments) to get percentageMultiply percentage by total full-text for that particular databaseTotal calculations are sums of each row to ensure that they equal 100 and full-text total for that db
Goal: to increase number of verified connectionsPossibility: have login required from all computersCon: eliminates community patronsThe next handout is a collapsed version of handout 2 – quickly shows full-text percentages of connections by majorIntended to share with departmentsTotal numbers included in budget allocation formula
As you can see here, the School of Business’ (the smallest academic entity on campus) connections far exceed their enrollment.
There are 16 departmental annual reports, each with an appendix (raw data) fileInstead of having to remember which department does what at the spur of the moment (we can, but can your dean?), the numbers from the annual reports are transferred to the Master Table.Master Table is shared with the provost and all deans at the June Academic Affairs Council meetingAnother handout to share at departmental meetingsFor now, the Master Table is my College/SchoolCould easily be done by department, as wellEach liaison could adapt the Master Table to their own departmentsMaybe a Master Table for each College School showing the individual departmentsThe numbers are all in the annual reports
This is the collapse version of the Master Table.
Your next handout is the full version with defined items and ownership by library department.The numbers are also transferred to a 5 year spread table for strategic planningThe next handout is the same table but for all those OTHER numbers not attributable to an academic department.Goal: decrease the numbers in OTHER, increase the numbers for the main Master Table.
