This presentation was given by Peters to the International Health Economics Association Conference 2009 in Beijing. It is research conducted as part of the Future Health Systems Research Programme Consortium www.futurehealthsystems.org.

1. Evidence & Implementation of Strategies to Strengthen Health Services David H. Peters, Johns Hopkins University Beijing, IHEA July 13, 2009

2. The Quest To find the “best evidence” to strengthen health services 2

3. Health Interventions Are “Known” To Be Cost-Effective in Developing Countries 3 Laxminarayan, Chow, Shahid-Salles (2006). Intervention Cost-Effectiveness: Overview and Main Messages. DCPP II

4. But Implementation is Variable: Country-Specific Changes in DPT3 Coverage (All LMICs) Source: Matsubayashi, Peters & Rahman (2009) Multi-level models, each line represents one country

5. Do Countries Follow a Common Pathway to Expanding Health Services? Trends in Skilled Birth Attendance Source: Matsubayashi, Peters & Rahman (2009) Multi-level models, each line represents one country

6. Do Countries Follow a Common Trajectory to Reach the MDG for Reducing Child Mortality Rates by Two-Thirds? Source: Matsubayashi & Peters (unpublished) Multi-level models, each line represents one country

7. What Evidence Do We Want About Strategies to Strengthen Health Services? Adapted from Habicht et al (1999)

8. 8 Evidence Before Economics or Epidemiology Sutra of Buddha Aristotle Confucius Adi Shankara Ibn Rushd JS Mills David Hume

9. Systematic Reviews On What Works in Developing Countries 9 Peters, El-Saharty, Siadat et al (2009). Improving Health Service Delivery in Developing Countries: From Evidence to Action

10. Types of Strategies 10

11. Types of Studies with “Adequate” Designs for Systematic Review Randomized controlled before-and-after trials Non-randomized controlled before-and-after trials Randomized controlled post-only trials Interrupted time series designs with at least 3 data points before and after the intervention Case-control study with prospective data collection 11

12. Improving Health Service Delivery: Common Findings Implementation faults are very common Many different types of strategies can succeed, but are not replicable in much detail Strategies with the same label do very different things Strategies produce many unintended consequences, not predictable in detail Policy makers define strategies, but often have limited influence on how they are implemented Institutions involved and how implementation occurs matter greatly Not nearly enough attention has been paid to demonstrating how to improve services for the poor 12

13. Characteristics of Strategies that Strengthen Services for the Poor Intention for benefits to reach the poor Regular measurement of impact on the poor Oversight to ensure that the poor benefit 13

14. Strengthening Health Services: Successful Approaches

15. Community Empowerment Strategies that Work 90 percent of all studies using community empowerment approaches had a positive primary outcome: Promoting communication and collective action by communities Supporting community ownership and management of services Providing training opportunities for local health workers Holding service providers, officials, and private organizations accountable 15

16. Context Also Matters: Factors that Really Make Community Empowerment Strategies Work 16

17. Main Limitations of Systematic Reviews on Health Systems Strategies Weak ability to generalize findings because of: Pooling of data on widely different interventions (often with the same label or name) and on multiple outcomes Many very different and changing factors that influence outcomes Unsuitability of many large-scale strategies for controlled designs Little information on HOW implementation occurred Plausibility and limited probability inferences only Publication bias 17

18. Key Lessons for Implementation There is NO BLUEPRINT for successful implementation How a strategy is implemented is at least as important as what strategy is pursued Insufficient assessment of experience in involving poor in design, implementation, or assessment of results A “learning and doing” approach underlies successful implementation of many different strategies 18

19. Strategies Institutional Support Customers & Beneficiaries Provider Organizations Flexible, Capable Management Public Participation Iterative Learning Structured Learning & Doing: What’s Involved 19

20. Implications for the Future Stop requirements for blueprints of what to do Identify and engage key local institutions Apply “Learning & Doing” approaches: Ask difficult questions about leaders, laggards, and lessons Use information intelligently: look for intended results, unintended consequences, explanations, connections Measure and disclose how programs affect the poor 20

21. Evidence to Support a Learning Approach Explanatory and Plausibility/Probability Interferences: Mixed qualitative-quantitative designs Longitudinal tracking of: Quality and scale of implementation How information is used, learning processes Perspectives and roles played by critical players Results for beneficiaries, stakeholders, organizations Comparison groups where possible 21

22. Future Health Systems Example: Chakoria Bangladesh Large informal markets and provider organizations are found to be the main source of outpatient care for the poor. But quality of medical practice often poor or dangerous. Information was used to support new partnerships (with public, private, beneficiaries, and research organizations) and innovations in services service delivery by informal providers. Integrated management of childhood illness (IMCI) protocols used to train informal providers, disclose performance, use local “Health Watch”, and reform payment systems (subsidies for services to poor). Results: increased coverage of safe deliveries linked to incentives, reduced use of inappropriate drugs, ongoing need shown for supportive institutions to back up training and education of public. 22

23. Afghanistan: Using A Balanced Scorecard to Improve Services Annual nation-wide assessment of health services by 3rd party used to support national policy to promote basic health package and benefits to the poor through service contracts with NGOs and government. The demonstration of improved coverage and quality of health services, and falling infant and child mortality, has helped strengthen management, policy-making (e.g. user fees policy determined by partner’s research), accountability, and new institutions in a post-conflict state. 23

24. A Bright Future for Health Systems Many strategies work Use data and disclose information Involve all key stakeholders Focus on needs of disadvantaged Use opportunities for learning 24