Outcomes Methodologies in the Changing Landscape of CME

All Rights Reserved, Duke Medicine 2007This event is powered by BeaconLive Webinars | Webcasts | Online Events
Just like your CME, technology delivery is ever-evolving.
Pair your CME with BeaconLive’s cutting edge delivery methods.
BeaconLive | Webinars | Webcasts | Online Events Learn More: BeaconLive.com/Healthcare

Outcomes Methodologies in the Changing
Landscape of CME
Chitra Subramaniam, Ph.D
Assistant Dean and Director, Continuing Medical Education
Assistant Director, Center for Educational Excellence
This webinar is brought to you by
BeaconLive
Webinars | Webcasts | Online Events

All Rights Reserved, Duke Medicine 2007This event is powered by BeaconLive Webinars | Webcasts | Online EventsThis event is powered by BeaconLive
About Me
Education:
• Ph.D in Curriculum and Instruction with a focus on Instructional Technology; Cognate: Measurement and
Evaluation
• 10X10 AMIA training in Biomedical Informatics
• MS Clinical Biochemistry
Areas of Expertise and its Application:
Basic :Curriculum and Instruction
Education Evaluation and Measurement
Information Processing/The learning process
Knowledge Repositories and Instructional Design Systems
Data Visualization
Information/Interaction design; Usability
Applied: Design of Knowledge Systems for Outcomes based Education
Measurement and Evaluation methods; Impact of Learning
Instructional Design Systems; Instructional design strategies, teaching
Integration of Technology in Education
Information Design

Those who have been influenced!
• AO Foundation
• American College of Cardiology
• American Society for Clinical Pathology
• American College of Radiology
• CDC, Ministry of Health, Tanzania
• National Quality Laboratory Training Center, Tanzania
• Pearson Custom Solutions, Pearson Publishing
• McGraw Hill
• Cengage Learning
• Thomson/Delmar Publications
• Center for Medicare and Medicaid Services
• Council for Exceptional Children, Washington D.C
• University of South Florida
• University of Central Florida
• Lincoln Technical Schools
• US Education Corp
• City College
• Kaiser University
• Hernando County School Board
• Hillsborough County School Board
• IBM, AT&T
• Learning Mate, India
• Chartered Institute for Professionals in Financial Accounting, U.K
• American Society for Clinical Pathology

Polling Question
Rank the # 1 challenge in measurement and evaluation
you face today
1. Concepts are too abstract and cannot be
operationalized
2. Data has too much variability
3. Results are mostly subjective. Objective
measurements are a challenge
4. The system does not allow for a streamlined
approach to measurement
5. None to very few tools available for implementation
6. Accountability is too high- cannot meet expectation
7. Have little or no access to data; not within a health
system

My Perspectives
• Outcomes is an afterthought in most organizations both for
education and training.
• Strategic and tactical frameworks for achieving outcomes in
most organization is lacking
• Lack of clear mapping and alignment of the outcomes
methodology with the instructional design and delivery
strategies and learner engagement approaches.
• At a macro-level, the educational program goals do not align
with institutional goals.
• The process of learning and factors influencing the learning
(individual, group and system level) are not being considered.
• Protocols for programs that include QI/PI, research,
collaborative team based learning are not clearly defined.

A Paradigm Shift in Healthcare
From Here To Here
Individualistic Collaboration
Autonomous Integrated
Scholarly Practical
Expert Centered Focus Patient/Service Team
Hierarchical Distributed

Healthcare Environment…..
Patient Centered
Personalized Care
Diagnosis, treatment, and management- the process
Behavioral, cognitive, social, economic and cultural factors that impact the physical,
mental and social well being of a patient
Integrated system- Its about the experience
Micro and Macro level issues
Accountability/Standardization/health outcomes
Health care costs- Sustainability
Motivated, disciplined patient
Life Long Learners

The Challenge
Influencing Factors
Research
Quality Education

Performance Needs
Learning Needs
Business Needs
Learning Objectives
Application Objectives
Impact Objectives Impact and
Consequences
Application
Implementation
Learning and
Confidence
Begin with the End In Mind!

Business Needs
Job Performance
Needs
Learning
Needs
Design
Analyze DevelopEvaluation
Assessment
Needs
Assessments,
Evaluations
and survey
results,
Market
Research
New
developments
And trends,
Teaching
philosophy,
goals and
vision, situated
learning
Objectives/learning outcomes, content,
instructional design, assessment measures,
supplemental materials, resources, modalities
and technology
Formative/
Summative
evaluations,
resources,
support
services,
supplemental
materials
Implement
What does that mean?

SYSTEM NEEDS
Evidence Based
Medicine
Quality Improvement
Performance Metrics
Business practices
Delivery Models
Clinical Research
HEALTHCARE
DELIVERY
Healthcare Teams
Coordination of
Care
Healthcare
Disparities
Health Information
Technology
Pay for
Performance
Life Long
Management of
Diseases
CPD/CE
Healthcare Disparities, Episodic to life long treatment, Patient expectations and demands are
different, Patients are a part of the team
Learner Centered Education/System Driven Training
Quality
Research Education

Display
knowledge, comp
etence, performa
nce at predefined
level/benchmarks
Develop
contextual
competence, adap
t to
change, recognize
patterns in
problems within a
context
Adapt to change,
find, validate and
apply new
knowledge
Develop
competence further
through habits of
mind, behavior, and
wisdom
Continuously add to
knowledge, compet
ence and
performance
through practice
and reflection on
experience
Assess “the habitual and judicious use of
• Communication
• Knowledge
• Technical skills
• Clinical reasoning
• Emotions
• Values
• Reflection
in daily practice”.
Epstein, MR., Assessment in Medical Education, N. Engl. J Med Jan 25, 2007. 356;4
Teacher/Instructor
Facilitator
Coach/Curator

Polling Questions
Of the 5 stages discussed at which level do you think your CE activities are
designed and measured?
1. Display knowledge, competence, performance at predefined
level/benchmarks
2. Develop contextual competence, adapt to change, recognize patterns
in problems within a context
3. Adapt to change, find, validate and apply new knowledge
4. Develop competence further through habits of mind, behavior, and
wisdom
5. Continuously add to knowledge, competence and performance through
practice and reflection on experience

Performance
Knowledge
Competence
Moore’s Outcomes Levels
Bloom’s levels
Participation, Satisfaction
Performance
Learning- Declarative Procedural
Patient Health
Community
Health
Competence
Populatio
n Health
KNOWLEDGE
APPLICATION
ANALYSIS
CREA
TE
EVALUATIONLearner Needs
UNDERSTANDING
Learning Formats
Instructional
Strategies
M & E methods and
tools
Constructive Alignment

Pre-structural Uni-structural Multicultural Relational Extended Abstract
Quantitative Phase Qualitative Phase
Identify
So simple
procedures
Enumerate
Describe
List
Combine
Do Algorithms
Compare
Contrast
Explain
Relate
Analyze
Theorize
Generalize
Hypothesize
Reflect
The Hierarchy of Verbs
Deep Learning

Motivation, stress, time constraints, cognitive overload
Metacognition, critical thinking/reasoning, problem recognition
System/Business
Healthcare Consumers

In case there was a story to tell….
Member Engagement
• At the basic level, any product or service that meets
customer needs can derive growth in
sales, participation, brand identity and word of mouth.
• Consumption is a form of participation or engagement
• Can be measured in money, satisfaction, participation, idea
generation, retention, money invested by customers, number
of spin off products.
• Directly links to the professional advancement of the
member.
• Product/service level engagement
Educational Value
Activities’ influence on performance, habits and perhaps
even outcomes.
Activity integrates into professional practice
Activity-satisfaction
Activity addresses needs
Activity allows for knowledge and skills transfer
Format of the activity suitable, convenient and easy to
follow
Content relevant to practice and the needs
Return On Investment
A traditional financial measure based on historic data.
Trends from financial performance
Rate of increase/decrease in revenues
Marketing ROI
Retention and Acquisition rates
Strategic Effectiveness
Organizational strategy the association chooses to deploy
(product leader, most advanced, price leader, cheapest,
most customized) based on
o Strategic organizational goals
o Member value proposition

A systems approach to desired results
Framework for Program planning and Evaluation

How do you measure quality?
Improvement Tools
• Continuous quality improvement (CQI) –
opportunity for improvement exists in every
process on every occasion.
• CQI model emphasizes view healthcare as a
process and focuses on the system rather than the
individual when considering improvement
opportunities.

 The FADE Model
 FOCUS: Define and verify the
process to be improved
 ANALYZE: Collect and
analyze data to establish
baselines, identify root
causes and point toward
possible solutions
 DEVELOP: Based on the
data, develop action plans for
improvement, including
implementation,
communication, and
measuring/monitoring
 EXECUTE: Implement the
action plans, on a pilot basis
as indicated, and
 EVALUATE: Install an
ongoing
measuring/monitoring
(process control) system to
ensure success.

PDSA
PLAN: Plan a change or test of
how something works.
DO: Carry out the plan.
STUDY: Look at the results.
What did you find out?
ACT: Decide what actions
should be taken to improve.
Repeat as needed until the
desired goal is achieved

Six Sigma
• Statistical measurement methodology designed to reduce
cost, decrease process variation, and eliminate defects
• Sigma is a statistical unit reflecting the number of SDs a
given process is from perfection
• Steps
• 1. Define – creation of project charter which defines the
customer’s needs, project scope, goals
• 2. Measurement – data collection plan
• 3. Analyze – data analysis occurs, deviation from standards is
identified
• 4. Improve – create solutions and implementation plans
• 5. Control – process is controlled by implementing policies,
guidelines, and error-proofing strategies to make reverting to old
process impossible.

Things to consider
• Quality Improvement versus research
• QI objective address need of local situation
• Research seeks to address problems that will provide
generalizable results
• QI can be considered research if:
• the tested intervention involves a deviation from established
practices
• individual patients are subject
• randomization or blinding is conducted
• Participants are subjected to additional risks/burdens beyond
usual practice.

Quality of Care Measurement
 Costly to implement Quality Improvement programs
 Requires good data and evaluation of costs
Method Purpose Data Requirements
Cost-effectiveness
analysis
Comparison of costs and
health effects of a QI
program vs. usual care
Utilization data for
healthcare providers
Long-term health
benefits to patients
Cost-minimization
analysis
Comparison of costs of two
programs with identical
health benefits
Same as above
Cost-benefit
analysis
Comparison of program
costs and benefits
Health benefits of
program
Cost of programs

HealthCare Matrix
Bingham, J. and Doris, Q et al (2006). Using an
Healthcare Matrix to assess patient care in terms of
aims for improvement of core competencies. Journal of
Quality and Patient Safety, Vol 31 (2).

The Duke Outcomes Matrix
• Practice gap matrix
• This is where we started…….
• Everything in between……
• This is where we are now…….

Polling Question
Do you use tools to design, develop, implement and
measure CE activities?
Yes
No

Outcome- Increase pneumococcal vaccination rates in patients at
high risk (18- 64) and those 65 and greater
Physicians

Health Educators from the County
Just for us
• Reach out to all seniors and high risk groups through educational sessions
within Durham, Wake and Orange Counties
• Communicate the importance of pneumococcal vaccines and the
consequences of not obtaining one.
Outcome- Increase pneumococcal vaccination rates in patients at
high risk (18- 64) and those 65 and greater
• During home visits for seniors, screen for pneumococcal vaccinations.
• Document the screening
• Ensure vaccination of seniors who need one and document it.

Outcome- Improve Quality of Life for HIV patients by
empowering them and their care givers
Physicians, researchers, social workers/counselors, nurses and patients
Role Goals Measures
Physicians, Specialists Effectively communicate the need
to maintain the treatment
regimens.
Emphasize the need for mental,
emotional and physical well being
and empower the patient to
pursue the best treatment that
addresses their individual needs.
Work with the care giver team
members to support the
emotional well being of the
patient
Implement evidence based
strategies to communicate and
build trust and connect with
patient
During every patient encounter
evaluate patient’s adherence to
medication plans.
Assess the emotional, physical,
and mental well being of the
patient.
Document the assessment
Using coaching techniques, offer
suggestions and strategies to
enable patient self confidence
Present all available options
based on the results of the
assessment and discuss their
advantages and disadvantages.

Roles Goals Measures/Indicators
Social Workers Guide HIV patients regarding their
legal rights following disclosure of
their status.
Serve as an advocate for the
patient.
members to support the emotional
well being of the patient
strategies to communicate and build
trust and connect with patients.
Upon disclosure, inform and
document patient’s legal rights
using case examples.
List and share all community, state
and other resources available to the
patient.
Communicate to the physician and
other care givers if any changes in
the emotional, mental or socio-
economic condition is observed.
Follow up with patients on a regular
basis to ensure that the support
through resources are made
available.

Nurses Follow all protocols during patient
encounters
Provide guidance and support
related to treatment regiments and
their management
members to support the emotional
well being of the patient
strategies to communicate and build
trust and connect with patients.
Document all patient vital signs,
HIV health factors that are
evaluated.
Assess medication adherence and
communicate to the care team
appropriately.
Share with the care team any
changes observed in the emotional,
and physical well being of the
patient.
Communicate with patients,
physicians and specialists to ensure
proper coordination of care.

Patients Be well informed about the
disease, its diagnosis, treatment
and management
Contribute to, engage and learn
from the broader community
Ask questions of all care givers
and communicate concerns and
challenges.
Stay motivated and empowered
to pursue the best treatment that
addresses their individual needs
Participate in educational
opportunities offered by the care
givers.
Engage in online communities
and share/exchange knowledge,
experiences and resources.
During visits, clarify concerns and
understanding of treatment
options.
Participate in activities that lead
to empowerment and motivation
of oneself to lead a quality life.

Performance Dashboard

Questions and Next Steps
View these on-demand webinars at
BeaconLive.com/healthcare-webinar-series
Involve Me! Reflection of Adult Learning Principals
Healthcare Reform 2.0: What it Means for CME and
Medical Communication Professionals
Automating your CME Enterprise with Learning
Management Technologies
BeaconLive’s Healthcare Webinar Series
On-Demand Webinars

Questions and Next Steps
➔ Full Service Webinars
➔ Onsite and Remote Webcasts
➔ On-Demand Solutions
➔ Continuing Education Platform
➔ Integrated Registration
Contact BeaconLive today
to learn how we work with the healthcare industry
John Craine | National Sales Manager, Healthcare Services Division
jcraine@beaconlive.com | 434.927.4122
BeaconLive’s Healthcare Webinar Series

Outcomes Methodologies in the Changing Landscape of CME

Recommended

Recommended

More Related Content

Recently uploaded

Recently uploaded (20)

Featured

Featured (20)

Outcomes Methodologies in the Changing Landscape of CME