I gave a one hour overview to librarians from NH about assessment. My approach to assessment focuses on collection of performance assessments, mapping session level outcomes to program outcomes, aggregating data by outcome, SHARING what you learn, and contributing to program level assessment. I plan for and organize assessment methods into “tiers” with tier one assessments capturing student development of information literacy from a variety of academic experiences, and tier two assessment methods capturing librarians contribution to students development of information literacy. One librarian asked me after the discussion: where should I begin, especially with limited access to students? My recommendation is always to start with what’s already being done. Where are students already being assessed? Look there and see what you can learn about the challenges students are having. Then create your plan, and “start small, but start” as Deb Gilchrist and other ACRL Immersion faculty always mantra.
https://docs.google.com/presentation/d/1pOxo0s29jsQw9PVr7fp1AA7HKeOL8T_YiupQjyZFpGM/edit?usp=sharing
1. Designing Information
Literacy Learning
Experiences
&
Classroom Assessment
Workshop for University of New Hampshire librarians
July 9, 2013
Elizabeth Dolinger, Assistant Professor, Information Literacy
Librarian.
Mason Library, Keene State College
edolinger@keene.edu (603)358-2749.
2. • Information literacy: Where are we coming
from?
• Learning theories overview
This morning (10:00-10:50)
After Break (11:00-12:00)
• Backwards Design/ Understanding by Design: Gilchrist model
• Small groups design activities, start lesson
plan using cycle worksheet
3. Where we are coming from...
Libraries are too
complex with
too many tools
for too many
purposes.
Bibliographic
Instruction
4. Where we are coming from...
From
research
tools to
information
concepts.
7. Reflection Card
Take a moment to reflect on your
experience as a learner.
Come up with a positive and a negative
example of a classroom experience.
• what is it?
• what did you learn?
• why is it memorable?
12. Zone of Proximal Development
& Vygotsky
internalization of knowledge/ how we "know"
something
• scaffolding
• zone of proximal development
What
student
can do
What
student
can not
yet do.
What
student
can do with
help.
ZPD
14. Note Check & Break until 11:00
Turn to a partner and discuss a few of your
take-away's so far.
• Compare notes on what was covered, and
summarize the most important parts.
• Clarify any sticking points or questions for
each other.
15. Backwards Design:
Understanding by Design
Identify desired
results of
learning:
Enduring
Understanding Determine
acceptable
evidence:
Assessment
Plan learning
experiences
Wiggins and McTight
(1998) Understanding by
Design
17. What do you want the student to be
able to do?
Students critically evaluate information sources to identify the
appropriate use of information (evaluative criteria should be
appropriate for the project).
Program/course level outcome:
Students will be able to
differentiate between
academic and non-academic
sources
Possible session level outcomes:
Students will be able to
distinguish primary from
secondary sources using a set
of criteria
18. What do you want the student to be
able to do?
Students develop appropriate methods and effective strategies to
search for and access information.
Program/course level outcome:
Students apply concepts, key
words and synonyms as
search terms to locate
appropriate information.
Possible session level outcomes:
Students will identify elements
of a citation in order to search
for the information source
19. What does the student need to
know?
Content:
• what "keywords" are
• how to identify/find keywords
• how to structure a search in a database
using keywords and phrases (use AND/OR)
20. What is the Learning Activity?
Pedagogy
• lecture
• group work
• problem-
based
• active learning
• hands-on
• demonstration
• etc.
How will you teach?
21. Putting it in context...
You are working with a sophomore level
anthropology class. The faculty member
has asked you to show students the
database JStor.
• What are questions we could ask the faculty
member?
• What is one outcome we could work on?
• What is an in class activity we could use to
work with this class?
22. Putting it in context:
Brainstorm tickets
On front side of ticket:
What is one outcome to work on?
On back side of ticket:
What is an in class activity that could be used
to work with this class?
23. Before lunch... starting the Cycle
• Small groups of 2 or 3
• Agree on one outcome (write/use your own
or select from the list)
• Think of 2 different activities that could be
done in class (one activity per lesson plan
cycle)
• start to fill in your assessment circle poster
25. Evaluation v. Assessment
Educational
Assessment:
Intentional gathering
of evidence in order
to judge the quality
of achievement
based on pre-
determined learning
outcomes and
improve learning
Evaluation:
reflective tool
that gauges
impression of
an experience
26. Why assess student
learning?
"the quality of student learning is directly,
although not exclusively, related to the quality
of teaching. Therefore one of the most
promising ways to improve learning is to
improve teaching." Angelo & Cross
27. comes from Latin 'ad sedere', which means to sit down
beside
Helps us to know what we are doing, why we
are doing it, what students are learning as a
result, and to make changes based on that
information. -Deb Gilchrist
"Assessment"
28. Summative
• part of the instructional
process
• helps determine next
steps
• student participates in
the assessment process
and learns from it
Formative
V.
How will you use
what you learn about
the student learning?
• designed to be
comprehensive
• most often occurs
at the end of the
learning/at the end
of an instructional
phase
• used to judge
mastery
29. Self Report
• Asks students to estimate their learning
• Can be used as a benchmark at start or end of class
• Can be used to get at outcomes that are challenging to
assess/that are subjective
Examples:
• Minute paper
• Muddiest Point
• Interview
• Focus group
• Journal/log
30. Tests/ Quizzes
• Often multiple choice
• Strive for objectivity
• Grounded in traditional behaviorist theory
31. Classroom Assessment
Techniques
• in context of learning
• formative
• elicit student feedback about
their learning: "feedback loop"
• usually quick
A Few Examples:
• one minute paper
• muddiest points
• performance assessments
32. How will the student demonstrate
learning?
Classroom Assessment
Performance
Assessment
Task/ Assignment:
Student will write a paragraph
stating the best search structure
and key words used, why it was the
best, and two places to look for key
words.
33. Re-group & Share
• If needed, re-work Learning Activity
• Work on: Assessment: How will the
students demonstrate the learning?
• Write what the learning activity is at the top
of the large sheet & hang up on wall.
34. How can the activity be altered so
students demonstrate learning and
provide assessment opportunity?
How can you improve the activity so
students show their learning/ability?
Gallery Walk: Add & Map
36. How do I know the student
has done well? What does good
performance look like? Acceptable?
Poor?
37. Criteria
Decide to create something with this fabric.
• What are you going to make?
• Which fabric will you use?
• Why this fabric?
• How did you decide?
M. Oakleaf, Assessment Immersion 2012 exercise (used with permission)
39. Outcome:
Apply backwards design to develop lesson plans for information
literacy that employ active learning methods and provide
opportunity for assessment
Content:
Overview of Learning Theories
Intro to backwards design
Intro to CATs/Performance Assessments
Learning Activities:
Lecture, Modeling, Reflection, Discussion, Brainstorm, Group work
Assessments:
Lesson plan cycle
Brainstorm Ticket
Today's workshop
Gallery walk
Four minute ticket
40. Thank you!
On front of card:
• Summarize the main points of the workshop and provide 3-
50 takeaways. These can be challenges you are
concerned about or points that were particularly insightful
in consideration of your practice.
On back of card:
• On a scale of 1-5(very) how likely are you to incorporate
some of these strategies as you plan lessons next year?
• What activities were particularly helpful?
• How could this workshop be improved for others?
4 minute ticket:
Elizabeth Dolinger
edolinger@keene.edu
42. Suggested Resources
Angelo, T. A., & Cross, K. P. (1993). Classroom assessment techniques: A handbook for college
teachers. San Francisco: Jossey-Bass Publishers.
Doyle, C. S. (1993). The Delphi method as a qualitative assessment tool for development of
outcome measures for information literacy. School Library Media Annual, 11, 132-144.
Gibson, C. (2008). History of information literacy. In C. N. Cox & E. Blakesley Lindsay (Eds.),
Information literacy instruction handbook (pp. 10-25). Chicago, IL: Association of College
and Research Libraries.
Holder, S. (2010). History and evolution of credit IL courses in higher education. In C. V.
Hollister (Ed.), Best practices for credit-bearing informtion literacy courses (pp. 1-9).
Chicago, IL: Association of College and Research Libraries.
Kuhlthau, C. C. (2004). Seeking meaning: A process approach to library and information services.
Westport, Conn: Libraries Unlimited.
Wiggins, G., & McTighe, J. (2005). Understanding by Design, Expanded 2nd Edition. Alexandria:
ASCD.
43. ImagesTools
By Tony Hall. May 26, 2009. http://www.flickr.com/photos/anotherphotograph/3571242832/
Compass
By Jeremy Beker. February 8, 2007. http://www.flickr.com/photos/confusticate/385786155/
Information Literacy Umbrella
By Dana Longley. March 29, 2010. http://www.flickr.com/photos/danahlongley/4472897115/
A Taxonomy of Literacies. By Lane WIlkinson. March 30, 2011. http://senseandreference.wordpress.com/2011/03/30/literacy-
sucks/
Dog getting clicker training. By Ellen Levy FInch. September 17, 2004.
http://commons.wikimedia.org/wiki/File:Dog_clicker_training.jpg
The image of Welsh conductor Grant Llewellyn
By Ncsymphony. September 18, 2007. http://commons.wikimedia.org/wiki/File:Grant_Llewellyn.jpg
Milk pitcher with lid
By Pitcherman. October 14, 2008. http://en.m.wikipedia.org/wiki/File:Milk_Pitcher_With_Lid.jpg
Filing Cabinet. By 401 (K) 2012. May 19, 2012. http://www.flickr.com/photos/68751915@N05/7249752654/
Digital glass blower. By Village 9991. June 8, 2009. http://www.flickr.com/photos/village9991/2901378820/
Human brain. From Gutenburg Encyclopedia. June 5, 2007. https://en.m.wikipedia.org/wiki/File:Cerebral_lobes.png
Rainbow Cake. By Time Ebbs. December 1, 2012. http://www.flickr.com/photos/ebbsphotography/8235949798/
Tie dye cake batter. By Selena N.B.H. September 2, 2011. http://www.flickr.com/photos/moonlightbulb/6113826338/
Social Media Marketing. By Paola peralta. September 25, 2005.
http://commons.wikimedia.org/wiki/File:Social_Media_Marketing.jpg
All images are
Creative Commons
eligible to share.
Editor's Notes
Information literacy instruction is based in the academic library's history of providing bibliographic instruction. Based in the idea of and the necessity for the librarian to act as "steward of the stacks" and "gate-keeper" to library information. discipline focused, as was the academy.
place bound "library assignment"
tools and library place focused
one shot although there have always been some courses that were primarily "about the book" and bibliography
not necessarily transferable across tools
term "information literacy" first used in 1974 by Paul Zurkowski, who was president of the Information Industry Association. in a proposal to the US National Commission on Libraries and Information Science.
By the 70's and 80's increased focus on "concepts" because there's too much information available for most students to figure out which is appropriate and how to use it. higher education begins to focus on shifting from teaching to learning.
1987 ALA Information Literacy group advocates student centered learning approaches and the National Forum on Information Literacy develops.
1989 "Information Literacy" given definition based on work of Christina Doyle's DELPHI METHOD. 1999 ACRL IL Standards for Higher Education.
by mid 1990's the World Wide Web dramatically increases access to information and focus shifts to end user searching. bibliographic instruction remains valuable but is limiting in its role in helping students to sustain and cope with complex information environment that is multi-dimensional.
Craig Gibson (lead imersion faculty member and assoc. director for research & education at ohio state) talks about needing to provide students a "compass" in order to navigate and make decisions within the information environment. the new "compass" for NetGen students is a bundle of abilities called information literacy
identified and integrated set of learning outcomes
thought of as "everyone's business" rather than "library territory"
builds upon previous work of librarians in bibliographic instruction
an understood need to involve faculty and increasingly work with other education professionals
focus on information concepts: finding, accessing, evaluating, using ethically
Challenges:
technology: student tech skills have outpaced their knowledge base
communication: what vocabulary do we need to use in order to communicate with others
collaboration: sharing of teaching and learning : we are in a strategic position across all disciplines of a campus
assessment: "proving" value to stakeholders
scalability of learning programs and services
legacy of bibliographic instruction: locked into collaborative models where we teach in others classes rather than develop and teach own course: this is a debated topic within and among information literacy professionals
definitional conundrum: information, technology, critical thinking, lifelong learning, 21st century skills, digital literacy, e-literacy, information fluency, PLUS pedagogies: evidence based, problem based, inquiry based, discovery learning,
Future we will see and are already seeing more integration with other skills outcomes.
new ACRL standards for IL will be more inclusive of sharing and producing in web environment.
ACRL Information Literacy Competency Standards for Higher Education Task Force is working on revising the IL standards toward blending competencies across multiple literacies in order to better address the social or participatory information environment, and the multi-dimensional information landscape.
Learning is defined as change in behavior
Checking student's work regularly and providing feedback as well as encouragement (reinforcement).
Focus is directly on the content to be taught. Often takes the material out of the context in which it will be used.
the learner is a clean slate to be filled
behavior is shaped by positive and negative reinforcement and the environment
Direct or "teacher centered" instruction: Lectures, tutorials, drills, demonstrations, and other forms of teacher controlled teaching
According to Piaget, the ability to learn a concept is related to a child’s stage of intellectual development. This focus on scaffolded early learning and sequential development of mental processes defines the Cognitivists' learning theory.
the individual learner is key
generate knowledge and meaning through development of cognitive abilities: recognize, recall, analyze, reflect, apply, create, understand, and evaluate.
memory system is an active organized processor of information
prior knowledge plays an important role in learning / new knowledge is built on prior knowledge
learning occurs as a result of disequilibration, the state of confusion that results when we interact with new phenomena, to which we must adapt by either assimilating the information into an existing scheme, altering an existing scheme to accommodate, or by creating a new scheme entirely.
people construct their own understanding and knowledge of the world, through experiencing things and reflecting on those experiences.
new information is linked to prior knowledge learning is constructed through social interaction
learning is an active process of constructing knowledge rather than acquiring it .
in this way, our knowledge is subjective. is it frosting? cake batter? glue? goo?
emphasizes the importance of active involvement of learners
Discovery, hands-on, experiential, collaborative, project-based, and task-based learning are a number of applications that base teaching and learning on constructivism.
activities are interactive and student-centered
students work primarily in groups and learning and knowledge are interactive and dynamic
focus on social collaboration and exchange of ideas
Role of teacher: modeling, scaffolding, coaching, facilitating
Lev Vygotsky = soviet psychologist 1920's
focused on internalization of knowledge,
through the assistance of a more capable person, a child is able to learn skills or aspects of a skill that go beyond the child’s actual developmental or maturational level. Therefore, development always follows the child’s potential to learn. In this sense, the ZPD provides a prospective view of cognitive development, as opposed to a retrospective view that characterizes development in terms of a child’s independent capabilities.
Scaffolding is changing the level of support to suit the cognitive potential of the child. Over the course of a teaching session, a more skilled person adjusts the amount of guidance to fit the child’s potential level of performance. More support is offered when a child is having difficulty with a particular task and, over time, less support is provided as the child makes gains on the task. Ideally, scaffolding works to maintain the child’s potential level of development in the ZPD.
emphasizes that two processes are occurring simultaneously: the process of information-seeking and the process of constructing meaning
Carol Collier Kuhlthau's Information Search Process (Book SEEKING MEANING)
based on 20 years of empirical research (first on grade school students, through experienced learners then into the workplace) ,
a six stage model of the experience in the process of information seeking.
Kuhlthau identifies three realms of experience: the affective (feelings), the cognitive (thoughts) and the physical (actions) common to each stage.
identifies "zones of intervention" based on Vygotsky's theory of proximal development: helping to identify the points at which students are most in need of and most likely to accept guidance
Using the Information Search Process model as a guide for the design of integrating information literacy development in a sustained research and writing course
helps to identify the information literacy skills and timing of the sessions so they are embedded at the point of need based on HOW THE FACULTY HAVE DESIGNED THEIR COURSE
helps faculty and librarians to communicate and focus on the development of skills that facilitate critical thinking about information, rather than the specific tools or creation of long bibliographic lists of resources
1. Forces you to think about the big picture goal of learning: what is the enduring understanding of the learning
2. How will we know that students really understand the identified big ideas? What will we accept as evidence of proficiency? The backward design orientation suggests that we think about our design in terms of the collected assessment evidence needed to document and validate that the desired results of Stage 1 have been achieved.
3. What will need to be taught and coached, and how should it
best be taught, in light of the performance goals?
So I often think of the "Enduring Understanding" and THE BIG IDEA... and it's often what can be found in PROGRAM LEVEL OUTCOMES... either for general education programs or for information literacy programs or departments, etc. These are often BIG IDEA level.
Deb Gilchrist, lead Immersion faculty and Dean of Library/Media Services for Pierce Colleges in Washington
At Keene, in a task force for one semester we worked through this process and identified common outcomes, content, and assignments = assessments for core foundation courses. The learning activities might be different, or how we teach, but ultimately the students need to end up at the same place and able to complete the assignment during the session.
Over the summer, in a teach-in, we worked through this cycle with all librarians learning about each element of the process of designing.
Backwards design, Gilchrist model
Understanding by Design, Wiggins & McTighe
Give me some examples of what might be an outcome in one session, that would reflect or tie into these outcomes? what does this look like in one of our sessions?
Outcomes need to be JUDGeABLE not necessarily measurable.
so what would a session level outcome look like? How do we already do this in a session?
Give me some examples of what might be an outcome in one session, that would reflect or tie into these outcomes? what does this look like in one of our sessions?
Outcomes need to be JUDGeABLE not necessarily measurable.
so what would a session level outcome look like? How do we already do this in a session?
so, What are some ways or activities you might use to teach these, or one of these during a session? WHat is the learning activity?
what are the students doing?
role play
debate
discussion
think-pair-share (small teams of students reflect on lesson and share learning and questions with group)
learning by teaching
problem based learning (identify and solve a problem)
small group/team work
What are questions we could ask the faculty member to make this more about developing info lit than demonstrating a tool?
10 minutes
have folks do at least 2-3 cards
then pass once more to next person
have person with card read off an idea that seems like a good one
Can you think of any examples?
WHY DO WE ASSESS STUDENT LEARNING?
TEACHERS
Assessment informs practice in the classroom in order to improve student learning (pedagogy and curriculum revision)
Assessment can increase student's learning awareness by knowing what learning is expected of them (opportunity for self assessment)
ADMIN REASONS
Program Improvement : Assessment provides information for programs to determine how they can be improved: SLO assessment allows programs to determine where redundancy occurs and where gaps exist in the curriculum
Stakeholder Communication : The results of program assessments should be communicated to stakeholders in order to demonstrate accountability of student learning
Resource Allocation
Accreditation Requirements : Participating in the ongoing process of assessment will demonstrate to accreditation bodies that the academic programs are achieving accreditation requirements
What are examples of outcomes that may be assessed through self report?
research process
anxiety, bordeom, self advocacy
familairity
What activities could we ask them to do that will help make the learning more evident?
by employing a number of classroom assessment techniques educators get feedback from students on their learning
Integrated performances that students engage in, that show learning, and result in an artifact or evidence
Simulate real life application of skills
Strive for authenticity
Grounded in constructivist theory & Assessment for Learning
Numerous methods, context and outcome specific
15 minutes
gallery walk, and map
go around room and read activity described
add below the activity a way to alter it so students demonstrate learning and provide assessment data
you can add onto others ideas by drawing lines and circles, like a concept map
A rubric can be as simple as a checklist.
It can be as complex or as simple as you need for it to be in order to apply your criteria for learning.
On an index card. First question on front, last four questions on back.