Presentation at WFATE Conference, 2021. Lorenzo, N. (2021). Transformative Evaluation Criteria. Evaluation Tools for Changing Processes Sixth Biennial Conference, WFATE, US.

1. 2
Transformative evaluation: Criteria
Evaluation tools for Changing Processes
Neus Lorenzo i Galés
Universitat d’Andorra (UdA)

2. Everybody has their own pace

3. Conditions are not the same for everyone
25% of the children in the world are living in extreme poverty
(2019)
Ref: https://www.wrmea.org/2019-june-july/the-scenario-of-a-million-palestinians-going-hungry-in-gaza.html

4. FEP Project : Plurilingualism in Catalan Schools (2014-2019)
(individual and group initial detection: 5-6 years old)

5. New orientation: 2021-2024
To include teachers attitudes,
beliefs, expectations
To include social priorities,
global challenges and
International frameworks.
SDGs

6. 21stC. Systemic Transformation: Changing dynamics
1. Investigate the world
2. Recognize Perspectives
3. Communicate Ideas
4. Take Action
Global Competence, OECD (2018)
Ref: http://ncee.org/wp-content/uploads/2020/10/GlobalCompStat.png

7. The Spiral of Inquiry:
Recursive Reflection
What’s going on
with our learners?
How do we know?
Why does it matter?
FOCUSING
DEVELOPING A HUNCH
LEARNING
TAKING ACTION
CHEQUING
SCANNING
Adapted from: https://www.educationalleaders.govt.nz/Pedagogy-and-assessment/Evidence-based-leadership/The-spiral-of-
By Helen Timperley,
Linda Kaser and
Judy Halbert

8. We are assessing all the time...
1- Your mother/partner/friend asks you to buy the best apples in the Store...
?

9. The best apple in the world...
Indicators versus descriptors
Indicators vs. Descriptors
Level 1 Level 2 level3
COLOR
SIZE
FLAVOUR
TASTE
PRISE
ÈTHICS

10. Building Rubrics
Shape Round Irregular Deteriorated
Color Red Yellow Green
Texture Fine Granular Damaged
Taste Very Sweet Sweet Bitter
Cost <2 Eu/Kg <4 Eu/Kg <5 Eu/Kg
Sostenibility Zero Km
(Vecinity)
From the region From far away
Indicator
descriptor
criteria

11. Is it the best apple?
General characteristics
Personal priorities
Measurable criteria
What for?
https://staticr1.blastingcdn.com/media/photogallery/2017/10/9/660x290/b_1200x675/a-beautiful-apple-image-credit-
skitterphotopexel_1622299.jpg

12. Can we avoid or minimize subjectivities?
I like it a lot It has 52 Kca. per 100gr

13. Rethinking needs, protagonists and techniques
• Context sensitive and inquirint mind
• Assessment for personal and organizational learning
• Assessment in teams, groups, and networks
• Improving processess, as much as achievement
• Participatory and inclusive assessment: flexible descriptors
(same criteria)
• Thinking globally, acting locally (personalization)

14. New Rubrics:
Personalized
Assessment

15. Typology of indicators:
From Quantum to Qualia
Example Data Tipology Process Operator Values
Nationality Discret data
No Order
Nominal Classification =  Nouns
(Qualitative)
Professions
according to
social
recognition
Discret data
Order
Order Order = 
 
Exclusively
Ordinal
Numbers
(Ordered)
Number of sons
and daughters
Discret data
Numeric
Cardinal Countable = 
 
+ −  
Cardinal
Numbers
(Quantity)
Age Continiuous
series
Cardinal Measurable = 
 
+ −  
Cardinal
Numbers
(Quantity)

16. Three teachers are asked
to prepare an activity
for improving orality in early levels...
How woud you assess their choices?

17. Phological consciousness to auditory memory
http://2.bp.blogspot.com/_5bnLqSYJmm0/TP0h-ReJuvI/AAAAAAAAMDk/4d9r7WBPD0M/s400/IMG_6231.JPG
K+Know

18. From meaningful vocabulary to
contextualized knowledge
https://s-media-cache-ak0.pinimg.com/564x/1c/d2/ba/1cd2bab0e0948dd3e0f2172b63a2a954.jpg
Understand and
interpret

19. From autonomous expresion
to cause-effect abstraction
https://s-media-cache-ak0.pinimg.com/564x/e0/96/a4/e096a4e0d1e38aa2599c6a1bc3ab2f0d.jpg
Observation,
explanation,
reflection...

20. • Personal and collective portfolios, pannels, infograms
• Written documents, texts, speaches, tasks, products...
• Records and anecdotes for the observation of reality.
• Personal portfolio for monitoring and collecting evidence.
• Questionnaires and forms: Knowledge and Prior Study Inventory (KPSI)
• Conceptual maps, and visual-data organizers and diagrams
• Rubrics: Themes, proceses, results, attitudes...
• Guidelines for action, sequences, protocols ...
• Collective productions: Conversation, direct dialoguea, debates ...
• Videos, audio recordings, webs, tweets, posts…
• Written tests, exams, reports…
• Projects, murals, dossiers, herbalists, collections…
• Dramatizations, performances…
• Models and maquettes…
• Exhibitions, organized productions, events, presentations…
• Digital productions, data analysis of interaction and participation
• Plans, graphs, representations, drawings…
• Quality of performances, abstract reflections, deductions...
Different evaluation resources
Tools
Strategies
Indicators

21. Tools for control and process
http://www.acquiredigital.com/uploaded/images/checklist.jpeg
Dimensions
Example: language learning
• Reading
• Speaking
• Writing
• Listening
• Interculturalitaty
• Negotiation

22. Measuring progress
Strategical
problem
solving
Self-assessment and self-knowledge:
(personal process, team, institutions...)
Ref: Neus Lorenzo & Ray Gallon, Transformation Society

23. Assessing the SDGs at
school
https://emeraldopenresearch.s3.amazonaws.com/manuscripts/14962/0a2981bd-a3e2-4c84-be68-f72f6bbdcab3_figure2.gif

24. Interested in the test?... (It’s in Catalan)
https://blogs.iec.cat/scp/lagenda-2030-i-els-ods/ https://sites.google.com/xtec.cat/ods-centres/formulari
Thanks to Vedrunes schools (Catalonia)

25. http://ugc-01.cafemomstatic.com/gen/constrain/500/500/80/2015/04/09/09/4y/re/po4ssptbsw.jpg
1-Learning
https://debrabell-tov6wyouooprzha1fzd.netdna-ssl.com/wp-
content/uploads/2013/08/iStock_000013895557Medium.jpg
2-Learning to Learn
h6ttp://d1lhri34tovdcj.cloudfront.net/prod/mom365/mom/libra/764357_59fb98b8df588ef91fba7cc6f64ad8d4-053d79e5-7a0f-4010-88f0-06b170f29cb2.jpg
3-Learning to act with others

26. Can we evaluate kindness,
interaction, communication?...
How can we get a common
understanding of these “líquid”
concepts?

27. Thanks for your attention!
More Information:
http://transformationsociety.net
ray@transformationsociety.net
neus@transformationsociety.net
Societat Catalana de Pedagogia
Institut d’Estudis Catalans