Indicators currently under development by KBHN's KT Core that measure impact of research and training. The KT Core's services are mapped onto David Phipps' Co-Produced Pathway to Impact diagram for KTEE (Knowledge and Technology Exchange and Exploitation).
DNV publication: China Energy Transition Outlook 2024
Indicators forMeasurement at each stage of Knowledge Translation: from research to impact
1. www.kidsbrainhealth.ca
Indicators for Measurement at each
Stage of Knowledge Translation:
from Research to Impact
Anneliese Poetz, PhD – Manager, KT Core
David Phipps, PhD, MBA – Lead, KT Core
Canadian Knowledge Mobilization Conference
Monday June 9, 3:30 – 4:00p (Room Cedar)
Saskatoon, Saskatchewan
2. What is KBHN?
• Federally funded
• NCE
• FASD, CP, ASD
• NI, NE, KT
• KTEE
• HQP
3. KT Core
“KT maximizes the impact of research and
training in neurodevelopmental disorders”
Services for researchers and trainees in KBHN
1. Knowledge Brokering
2. KT Events
3. KT Products
4. Evaluation
5. KT Planning
4. Role of Evaluation
How are the KT Core services helping KBHN
researchers & trainees?
How have we made a difference for children with
NDDs and their families, in Canada?
9. Anatomy of indicator
Definition components
1) Name/title of indicator
2) Definition (including N/D if
applicable)
3) Type
4) Rationale
5) Strengths/Limitations
6) Data Source(s)
7) Stage in CPPI
8) Date
9) Responsibility for collecting
data
Example
1) [CB] % increase in knowledge
2) Sum of scores/#attendees x 100
3) Impact
4) Measure knowledge gained
5) S: measures change/impact,
L: bias & missing data
6) Pre-Post questionnaires
7) All
8) May 28, 2014
9) KT Core
10. So what’s it used for?
Reporting (value/impact) of:
• services, events, products, etc.
Comparison
– Across services, events, products, programs, etc.
– Across NCEs (KTEE standards?)
Inform decision-making
Succession planning
Future (finish defn’s, Access)
What is NeuroDevNet? What we do, why we’re here (high level)
Federally funded NCE
~80 researchers/trainees (FASD, ASD, CP)
Knowledge and Technology Exchange and Exploitation
“…seek to shed light on the causes of neurological disorders, and to share this knowledge to health care professionals, policy makers, and communities of interest…
NeuroDevNet works across traditional disciplinary boundaries and sectors to ensure our findings are translated into tangible diagnostic, preventative, therapeutic,
social, economic, and health benefits for all.”
What is the role of the KT Core within NeuroDevNet?
We (KT Core) support NeuroDevNet’s HQP mandate through workshops like the KT/Social Media workshop.
What is the role of evaluation? To measure the impact the network has had on children with NDDs (and their families) in Canada.
Explain why do we do evaluation, why is it important.
Explain CPPI, what it is and why we are using it (important planning tool, also for monitoring progress toward impact – CPPI is best suited for a system of research or ‘set of projects’ rather than a single project that may never reach ‘impact’).
Re: the indicator that is circled – just drawing attention to how this indicator does not really measure impact of capacity building efforts. Still good to measure how many (for resource allocation decisions, or just to know how many events have been done) but doesn’t let you know what the impact of these events are.
Indicator Type (input, process, output, outcome, impact, performance, behaviour change, etc.)
Talk about an interesting thing about the indicator (value of a good indicator) such as the story about % increase in knowledge at the KT/Social Media workshop (e.g. 28% increase in knowledge – infographic on handout)
For Rationale – I actually got rid of some indicators because I couldn’t think of a good reason why we should collect data/measure it.
Why is thinking about indicators in this way, important?
So you can report measures of impact or value-for-the-money to your funder (NCE, CIHR, etc.)
Allow comparison across a) indicators for each service within NDN and b) across NCEs (a number of NCEs have self-identified as having KTEE within their NCEs, would be good to have a conversation about establishing standards for KTEE in NCEs. Having good clear definitions of indicators, this allows us to start to create some standards - does this form the basis for a standard for other NCEs, so comparisons can be made across Networks?)
**tell story about comparison between needle exchange programs (numerator/denominator)
inform decision-making (for KT Core services) – in other words, we are creating our own evidence-base for making evidence-informed decisions (**that the right indicators are informing the right decisions – we apply them in the right way to the right decision – we can be confident we are applying evidence in the right way because the definition tells us what it is defined as and used for).
As a management tool (e.g. for succession planning purposes) to make the transition easier for the next person (established data collection processes, data to learn from with regard to service provision)
Future: Finish the indicator definitions (~130) and continue to collect data on our services. These are ‘grouped’ according to the way they are mapped onto the CPPI. Some are other people’s responsibility to collect (through Valorization Committee, through established routes such as progress reports for each project/core etc.), evaluation data collected by projects/cores that do events and collect this data.
Migrate the system from Excel to Access, to make input of data more efficient.
Have this evaluation framework, and this set of services, needed to map the services onto the evaluation framework.
Have this evaluation framework, and this set of services, needed to map the services onto the evaluation framework.