What it is
What others have done
What to ask
How to succeed

What is innovation and learning and why is it important?

Health systems and the environments in which they operate are constantly in flux as populations and political economies shift. At the same time, the fields of medicine and public health are dynamic, with new information, guidelines, and best practices emerging frequently. Innovation and learning are capacities of the health system to adapt to and learn from these external forces to ensure that the health system is evolving to effectively and equitably meet population health needs with high-quality care.

Innovation and learning is a multidirectional and dynamic process that can occur at any level in a health system, including outside of the traditional health sector.1 While countries will develop their own ways of innovating and learning, it is essential to ensure there is a systematic, structured, and reliable method in place for evaluating evidence and operationalizing these learnings into changes at scale. 4 By exploring  the factors that underpin innovation and learning in a health system and sampling the tools and frameworks commonly used to drive innovation and improvement across different contexts, this module can provide inspiration for stakeholders to begin innovating and making improvements in their own health system.

  • In order to achieve a PHC-oriented health system, stakeholders must identify and develop innovative ways of responding to complex challenges and catalyzing improvements. Innovation and learning is an important force for building a health system’s capacity to develop context-specific, evidence-based solutions to improve PHC performance.5 Cultivating a system for identifying and implementing improvements that are in service of high-quality PHC (care that is continuous, coordinated, comprehensive, and person-centered with PHC as the first point of contact) is an important step toward achieving universal health coverage and building resilient health systems that can effectively adapt and respond to changing population health needs and demands.6 To support the adoption and creation of innovations that support the goals of high-quality PHC for all, it is important to establish norms and values that fulfill the values of PHC such as inclusiveness and effectiveness.7 

  • This section will discuss the role of innovation capacity and culture for innovation in health systems. These terms are defined below: 

    The culture for innovation describes the specific values, behaviors, and processes that an organization or system takes to promote continual innovation.89 Organizations or institutions should strive to create a culture that readily identifies and implements innovations, taking into account the norms and values of a given context. Seven key dimensions of culture that distinguish highly innovative organizations include: risk-taking, resources (in the form of time and money), knowledge, strategic goals, rewards, tools (deliberate structures and processes that support innovation), and relationships (people are exposed to a diverse range of thinking and people). 10 The culture for innovation in an organization is a strong factor in determining whether a change (whether small, medium, or large) can occur. It is closely linked with organizational readiness, or the organizational commitment to and efficacy in implementing an innovation, 11 and spread, which dictates the overall impact of an innovation.10 Users can read more about creating a culture for innovation in healthcare here.

    Culture for innovation is a component of innovation capacity which refers to a country’s capabilities for adopting innovative solutions developed elsewhere and for creating innovations themselves, and supporting these innovations from ideation to scale.112 Innovation capacity comprises two specific functions: knowledge creation and knowledge adaptation. Knowledge creation is the process of generating new knowledge and technologies while knowledge adaptation involves creating new value by accessing, anchoring, and spreading existing global and local knowledge. 14 Developing innovation capacity at the national level relies on a commitment to innovation and learning as a priority investment through integration in national strategies, regulatory structures, dedicated budgets, and multisectoral stakeholder engagement. 4132 Potential indicators a country might use to assess its innovation capacity include: investment in innovation (including human, material, and technical resources for innovations), technological infrastructure for innovation (such as internet usage for health), the sources for soliciting ideas for innovation (including user driven and open innovation), the culture for innovation, human capital and education (i.e. tertiary education), technology transfer and participation in globalization, and gender equity. Additionally, the following considerations offer a helpful framework for understanding determinants that strengthen a country’s innovation capacity:13 

    • Awareness: providing information to policymakers, the public, and the media that stimulates innovation and learning. 
    • Performance: measuring progress and results of changes that have been implemented against public policy objectives and the market factors (i.e. supply and demand)
    • Signaling and Monitoring: calling attention to important innovation trends and growth opportunities.
    • Accountability and evaluation: supporting research and development budgets and innovation policies, in addition to compliance with regulatory structures and public interests. 
    • Consensus building: developing and implementing more effective innovation policies and strategies that strengthen a country’s national innovation capacity.

    Users can find more information on and frameworks for innovation capacity and creating a culture for innovation in the upcoming tools and frameworks section.

  • While every country will go about innovation and learning differently depending on their context, it is essential to ensure there is a systematic, structured, and reliable method in place for evaluating evidence and operationalizing these learnings into changes at scale.4 This requires mechanisms to recognize and evaluate innovations as well as the operational capacity to scale identified interventions and programs. Recognition, evaluation, and scale are described as:

    • Recognition: innovators are able to recognize where opportunities for innovation and learning exist.  This includes the identification of information, practices, processes, structures, or other products that could serve as sources of inspiration for new ideas or tools. The relevance of different information or technologies depends on user needs, the policy and market environment, trends and future projections, and the strengths and competencies of an organization.14 There are many sources of information and technology that innovators can use to recognize opportunities for innovation and learning such as technology databases, research and utility models from health and non-health industries, publications, local and international universities and research institutes, and evidence from the field.14
    • Evaluation: Monitoring and evaluation systems are in place to track how interventions are being implemented and if adjustment and innovation is needed, either to restructure the intervention if it is not achieving its goals or spread it if it is. Evaluation of an innovation should assess the impact on performance outcomes, suitability within a policy environment, and alignment with the norms and values of a given context. If the innovation is being adopted or adapted from elsewhere, this assessment should also take into consideration lessons learned in other contexts, or, if an incremental change is being made, lessons learned from past iterations of the intervention. It is important that innovations are evaluated before decisions to scale are made, rather than making such decisions based on internal or external lobbying. Standards and criteria used to assess if an innovation is successful should be clearly defined and transparent and applied consistently.2 The methods and structure for recognizing and evaluating and ultimately identifying and deciding to scale the “successful” innovations should be widely understood and used consistently as innovations arise to support an inclusive and participatory evaluation process. The evaluation phase should be aligned with the priority setting process to support the adoption and scale of relevant/impactful innovations. Users can learn more in the module on Priority Setting. 
    • Scale: Through the evaluation process, systems should be in place to identify if and when to scale an innovation.15 Scale refers to the process of spreading successful innovations within or among organizations to achieve a greater level of impact. While scale is an important component of implementing an innovation in a system, not all innovations will be applicable at all scales. The value and impact of an innovation determines the strategy for scale. For example, innovators may decide between increasing surveillance functions of a particular disease using a novel technology nationwide, or decide on district-by-district increases depending on what is most appropriate in their country-context. Various implementation factors such as the target population or organization's receptiveness to change, access barriers (financial and geographic), ease of use, quality, convenience, and acceptability will influence the strategy for scale.113 While countries will likely have their own unique innovation processes that lead to different innovation outputs, scaling up innovations require initiatives built around diverse stakeholder participation, financially sustainable institutions, and supportive governance structures that support their integration into the health system. Before scaling an innovation, innovators should ensure the intervention has undergone multiple iterations at the pilot stage to determine its value, performance, and application to the context/level/location in which it will be implemented, including whether the environment the innovation will be implemented in is sufficiently ready for change (thi relates to organizational readiness). More information can be found on Results for Development’s webpage on Scaling Innovations.

    Additional frameworks and metrics for evaluating innovations and a country’s national innovation capacity are included in the section What are some commonly used tools and frameworks for innovation and learning? below.

  • Innovation and learning is a multi-directional process that relies on shared learning and input from stakeholders across all levels of the system.4 Encouraging broader social participation in the decision making process (including community-based representation) helps to strengthen accountability across sectors and forge collaborative partnerships for equitable and sustainable initiatives.16 Community engagement helps to empower community-level stakeholders to collaborate and share important lessons from the field, such as on the successes and failures of past interventions, to promote the adoption of innovations in their communities. Users can read more about scaling up inclusive innovations here.

    The broad involvement of stakeholders supports an overall culture for innovation, and clearly defined roles help ensure that mechanisms for recognizing success and subsequent scaling are not dependent on specific individuals or relationships between individuals, but formalized into a system. Taken together, broad stakeholder engagement paired with social accountability helps to ensure the benefits of innovations are realized and sustained throughout the broader system.1 It is important that these decision-making processes are aligned with national strategy planning and budgetary cycles and take into consideration local norms, needs, and values in order to incorporate innovations in a sustainable and acceptable manner.3 More information on multi and intersectoral engagement and social accountability can be found in the WHO report on multi-sectoral and intersectoral action for health and well-being for all and in the Social Accountability module.

  • This section includes a sampling of tools and frameworks commonly used to drive innovation and improvement in an organization or system. The table below describes these three models and their purpose and users. 

    These tools provide actionable ways for stakeholders to begin identifying areas for improvement in their health system and determining the best evidence-based practices to drive and sustain improvements in their country. The application of the different components of these tools and frameworks will depend on user needs and context. 

    IHI Model for improvement

    The Institute for Healthcare Improvement (IHI) Model for Improvement is a tool designed to help accelerate improvements in an organization.17 It provides stakeholders or improvement teams with a methodology for understanding the essential elements of system improvement, including the cycle of innovation and ways to develop, test, and implement changes to drive improvement in a system. The model is not intended to replace already existing change models but to drive improvements in different health care processes and ultimately improve outcomes.17 This model is applicable to a wide array of contexts, for example users might leverage the elements of this framework to help policymakers create a strategic plan at the national level or a care team improve a specific service delivery activity at the facility level. 

    The model includes two parts:

    • Three fundamental questions, organized by aim, measures, and changes - these can be answered in any order
    • Plan-Do-Study-Act Cycle (PDSA), designed to help users test whether changes result in improvements in real work settings

    Aim: What are we trying to accomplish?

    Improvement teams should set aims that are time-specific, measurable, and defined to a specific target population or system. Users can find more information on setting aims here.

    Measure: How will we know that a change is an improvement?

    Improvement teams should select quantitative measures that will determine if a specific change leads to an improvement. Users can find more information on establishing measures here.

    Changes: What changes can we make that will result in improvement?

    Improvement teams should identify areas for change that are guided by the experiences of those who have worked in the system or have implemented successful improvements. Users can find more information on selecting changes here.

    PDSA cycle: Improvement teams use the Plan, Do, Study, Act (PDSA) cycle to test changes as they are being implemented and adjust accordingly. Testing the impact, feasibility, and acceptability of a change involves planning the change, trying it, and studying the impacts of change, then subsequently acting on what is learned from the results. PDSA cycles can and often are repeated multiple times. Users can find more information on testing changes and action-oriented learning here.

    Implementing and spreading changes: Implementation involves making a permanent change to a process, such as the way work is done, in a manner that embeds it in the broader organization’s infrastructure and processes, such as through written policies and trainings. Before implementing a change at scale, improvement teams should test a change on a small scale and learn from each iteration through multiple PDSA cycles. After the successful implementation of a change on a broader scale - such as for an entire pilot population or unit - the improvement team can begin to spread the change to other relevant populations. Spread processes should also undergo multiple PDSA cycles to ensure that the change is adaptable and drives predicted improvements in all relevant settings. Users can learn more about  enhancing innovation and learning in a system and implementing and spreading changes in the following resources:

    Exploration, preparation, implementation, and sustainment (EPIS) framework

    The EPIS Framework guides users through the complex process of identifying areas for improvement and through four phases: exploration, preparation, implementation, and sustainment. 

    Four phases of the implementation process 1819

    • Exploration phase: Potential implementers such as a service system, organization, research group, or other stakeholders, identify what evidence-based practices best address the emerging or existing health needs of the target population. Implementers then decide whether to adopt the identified evidence-based practice(s) depending on their potential to address these needs.
    • Preparation phase: Upon deciding to adopt one or more evidence-based practices identified in the exploration phase, implementers identify potential barriers and facilitators of implementation as well as needs for adaptation in preparation for integrating the evidence-based practice into the existing system.
    • Implementation phase: The adopted evidence-based practice is integrated into the system. Implementers should ensure that ongoing monitoring is taking place and adjust implementation strategies to support implementation.
    • Sustainment phase: The evidence-based practice is ingrained in the system, with ongoing monitoring and/or quality assurance processes and adaptations as necessary, to ensure the evidence-based practice is sustained and health impacts are realized.

    Potential factors affecting implementation, such as the innovation characteristics and intra-organizational characteristics, are included in the EPIS framework at each phase at the individual, organization, and system level here. The framework helps stakeholders determine the complex needs of users and the factors driving the successful implementation of evidence-based practices, taking into account the complex dynamics between the different factors affecting implementation. The EPIS framework is a useful tool for designing evidence-based practices or adapting existing ones to operate within the context of a given health system. Stakeholders can find more information on this implementation tool here and learn about the application of this framework to public sector service systems in this research paper.

    Consolidated Framework for Implementation Research (CFIR)

    The Consolidated Framework for Implementation Research (CFIR) is a tool for identifying implementation factors across contexts and guiding a rapid cycle evaluation of the implementation of evidence-based practices. The CFIR features five major domains: 20

    • Intervention characteristics refers to the attributes of an intervention that might affect implementation. 
    • Inner setting includes the qualities of the implementing organization or system that will potentially influence implementation. 
    • Outer setting includes features related to the external context or information that may affect implementation. 
    • Characteristics of the involved individuals refers to the qualities of the individuals involved in implementation. 
    • Implementation process involves strategies or tactics that may influence implementation. 

    There are predefined constructs within each of these five domains that may affect the implementation of an intervention.2021 These constructs can be found here. Each construct analyzes barriers and facilitators to implementing an evidence-based practice in the process of change. 

    The  CFIR approach helps facilitate an understanding of how an evidence-based practice is implemented and what constructs support or pose barriers to the process of change. CFIR can be conducted at any stage of the implementation process. CFIR can help produce actionable findings intended to improve the implementation of interventions to achieve a better outcome, including what adjustments and refinements need to be made to an intervention. 2021

PHCPI is a partnership dedicated to transforming the global state of primary health care, beginning with better measurement. While the content on this website represents the position of the partnership a whole, it does not necessarily reflect the official policy or position of any partner organization.

What has been done elsewhere that demonstrates innovation and learning?

  • The 1990s marked a crucial turning point in the organization and delivery of PHC in Costa Rica. Since this time, Costa Rica has undertaken a series of targeted reforms in pursuit of an integrated primary health care system that delivers comprehensive, coordinated, continuous, and person-centered care for all. The successful implementation of Costa Rica’s reforms was supported by strong measurement and management systems, including management contracts, that have been continuously adapted and refined to align with country values and norms. 

    The Costa Rican health system is divided into distinct geographic health regions, or health areas, responsible for a geographically empaneled population.2223 In the contract management system’s current form, the Costa Rica’s Direccion de Compra de Servicios de Salud (DCSS), or the Department of Purchasing Health Care Services, assesses the health outcomes of each Health Area’s empaneled population as determined by a series of targets that promote universal access to high-quality PHC.(23)

    Over time, the DCSS made iterative improvements to this system to better align with Costa Rican values and achieve healthcare goals. One major change involved the removal of pay-for-performance incentives and financial penalties from management contracts, as this system was considered to be at odds with the values of Costa Rican health care professionals and not cost-effective enough to maintain. To incentivize the delivery of quality care and adapt targets to local values, Costa Rica introduced innovative targets related to quality of care and removed many of the process and performance-based targets. The process-based targets that remained were made more specific and measurable to promote quality of care and best practices.24 Even though management contracts were not linked to financial incentives for the providers or facilities,  they have proved to be an important performance management tool of the Costa Rican reform and driver of continuous learning and improvement. This was in part due to an annual negotiation of targets between the Health Area and the DCSS that served to assess whether goals were meaningful and appropriate to the local context. 25 

    In 2014, Costa Rica introduced another reform, called the Evaluación de la Prestación de Servicios de Salud (EPSS), to create a standardized way to compare the relative performance of different Health Areas.23 The EPSS established a set of national targets and indicators for a five-year period along the dimensions of access, continuity, effectiveness, efficiency, and user satisfaction.26 To promote continuous improvements, the EPSS targets increase annually by small, predetermined increments. Health Areas that perform in the bottom 20% must create an official remediation plan with the DCSS to improve their performance against established targets. 23 

    Costa Rica’s iterative approach to reforming their performance measurement and management system is an important example of how embedding innovation and learning activities within reforms can help to improve overall health system performance. Costa Rica’s attention to local values and context as well as their commitment to continuous improvement and refinement have supported the implementation of sustainable and relevant reforms. Users can find detailed information about Costa Rica’s health system reform and system for measurement and monitoring for improvement here.

  • Two states in the United States, Arkansas and Oregon, are applying different reforms to address the same goal of reducing health care costs without compromising the quality of care or shifting the financial burden to patients. In 2012, Oregon launched the Medicaid Coordinated Care Organizations (CCOs) model, a program designed to improve the coordination of care through accountable care organizations. CCOs refer to a network of diverse health providers who agree to collectively work to serve a community of people with coverage under Medicaid and are held accountable for care quality and population health outcomes. Oregon’s CCO model promotes the coordination of comprehensive care by allowing some flexibility to invest in non-medical services that target social determinants of health.272829 More information on Oregon’s CCOs model is available here.

    Also in 2012, Arkansas initiated the Arkansas Health Care Payment Improvement Initiative (AHCPII), a statewide payment reform model comprising most of the state’s insurance payers, such as Medicaid, private insurers, and large employers.30 The AHCPII model pays providers based on a patient’s entire episode of care rather than for each individual service. This bundled payment model was implemented with the aim to incentivize quality of care and shift away from a system where providers are paid based on the number of services they provide. Alongside this payment reform, the AHCPII reorganized the traditional PHC model into a patient-centered medical home model where the primary care provider acts as the main provider and coordinator of a patient’s care for the majority of their health needs.3132 Users can find more information on the AHCPII here.

    Both reforms have shown promising results in reducing costs and improving the quality of care. Oregon’s CCO model met its growth targets each year since its launch and generated approximately $2 billion in state and federal savings.272829 Arkansas’ multipayer bundled payments and patient-centered medical home models have also reduced costs and demonstrated positive impacts in some quality and outcome metrics.33 These examples illustrate the importance of allowing experimentation and flexibility in order to make reforms work in the specific context of a health system and acknowledging that different contexts may require different approaches to the same problem.34 The co-existence of various reforms in different contexts also creates opportunities for shared learning from the successes and failures of reforms across different settings. To determine the relevance of a reform to a particular context and its capacity for spread, stakeholders will need to assess the intervention through an iterative evaluation and implementation process. Users can read more about evaluating and scaling innovations in the Tools and Frameworks section of this Innovation and Learning module. 

PHCPI is a partnership dedicated to transforming the global state of primary health care, beginning with better measurement. While the content on this website represents the position of the partnership a whole, it does not necessarily reflect the official policy or position of any partner organization.

What questions should be considered to begin improvements?

The questions below may be a useful starting place for assessing Innovation and Learning in your context and determining whether it is an appropriate area of focus and how one might begin to plan and enact reforms:

How reliable and consistent is the state of mechanisms to recognize, evaluate, and scale successful innovations?

In order to drive change, newly created or existing innovations need to be successfully scaled beyond the pilot level. There methods and structure for recognizing and evaluating success and deciding to scale innovations should be formalized, systematic, and transparent. If these mechanisms are in place but successful innovations are not being scaled or sustained in the system, stakeholders might consider several factors to better understand gaps in performance. Are mechanisms in place to routinely incorporate new evidence from research, data, or reviews? Are these mechanisms formalized into the system and widely understood? Are these used consistently as innovations arise? If not, are gaps due to problems with governance, policies, regulatory and legal structures, communication channels or other factors?

How are stakeholders engaged in innovation and learning?

Stakeholders at all levels of the health system (both private and public) should be engaged in innovation and learning to strengthen your country’s capacity for innovation and learning. Potential innovators should routinely assess the progress and challenges in new evidence from research or data and its relevance to your context. The lessons derived from these discussions should be used to predict and/or improve response to future health threats or changing health needs. If stakeholders are not being appropriately engaged, you might consider several factors such as:

  • Are stakeholders’ roles and responsibilities clearly defined and communicated?

  • What is the culture for innovation and learning? How does this culture take form at different levels of the health system and across different organizational contexts?

  • What processes are in place to engage and coordinate the interests and actions of stakeholders from diverse contexts (including the private and non-health sectors)? 

  • If stakeholder engagement is not transparent and systematic, are gaps due to issues with policies, communication channels, social accountability mechanisms, or other factors?

How often and reliably are innovations scaled beyond the pilot level are learnings being operationalized into the system?

Successful innovations should be consistently scaled beyond the pilot level, with structures in place for operationalizing and formalizing these learnings into the system. This requires mechanisms to recognize and evaluate innovations as well as the operational capacity to scale successful interventions and programs. If innovations are not being appropriately scaled, you might consider several factors such as: 

  • Are ‘successful’ innovations scaled as a result of a legitimate and systematic evaluation process and not as a result of internal or external lobbying?

  • Are standards and criteria for successful innovations clearly defined and transparent, with standards applied consistently?

  • How are the decisions made to adopt and diffuse innovations? Do these involve a broad array of stakeholders such as committees, departments and/or working group or rather, a small number of key players?

  • Are decision making processes aligned with national strategic planning and budgetary cycles?

  • If innovations are not being appropriately scaled, are gaps due to social accountability mechanisms, governance structures, policies, or other factors?

PHCPI is a partnership dedicated to transforming the global state of primary health care, beginning with better measurement. While the content on this website represents the position of the partnership a whole, it does not necessarily reflect the official policy or position of any partner organization.

What elements should be in place to support effective improvements?

In order for interventions aimed at improving Innovation and Learning to be most successful, the following elements of the PHCPI Conceptual Framework should be in place or pursued simultaneously:

  • Innovation and learning is a multi-directional process that relies on shared learning and input from stakeholders across all levels of the system. Therefore, stakeholder engagement is an important tool for making innovations responsive to existing and emerging social concerns and priorities relevant to internal and external stakeholders.4 To ensure a participatory and inclusive process for stakeholder engagement and support a system-wide culture of innovation, robust institutional frameworks for community engagement and social accountability should be in place. Innovation and learning should leverage community engagement strategies to empower and encourage communities to hold stakeholders accountable. 

    Learn more in the Social Accountability and Community Engagement Improvement Strategies modules.

  • High-level commitment to innovation and learning is necessary for developing and sustaining a high-performing system, organization, and culture to innovate and learn. 4132 This may be demonstrated through dedicated national strategies, budgets, and research and development capacity. Making innovation and learning a priority investment as a part of the priority setting process is important for ensuring innovation activities are in alignment with priority objectives at the national level and will be prioritized and ingrained into the broader health system.

    Learn more in the Priority Setting, PHC Policies, and Health Financing Improvement Strategies modules.

  • In order to recognize, evaluate, and determine the scale, direction, and success of innovations (knowledge, skills, tools, resources, and services) decision-makers must have access to reliable information reflective of emerging and existing population health needs and health system performance. This process relies on the interoperability of information systems that collect, analyze, and share critical information (including surveillance data) and feedback to all relevant stakeholders and facilitate continuous learning.

    Learn more in the Surveillance and Information Systems Improvement Strategies modules.

  • When a given intervention is relevant to the facility-level, strong facility organization and management is an important enabler of the innovation and learning process. Some of the elements of an effectively-run facility include appropriate information systems use, performance measurement and management, and facility management capability and leadership. Effective facility organization and management also support the adoption and adaption of novel and ongoing quality improvement initiatives for innovation and learning activities at the facility level. Additionally, testing innovations in effectively run facilities can be a useful strategy for evaluating the implementation and feasibility of spreading these innovations. 

    The effective implementation of a change at the facility level is dependent on frontline staff buy-in. To support a culture for innovation and learning that promotes continuous learning and improvement, relevant training (i.e. data use) and support should be initiated in pre-service education.

    Learn more in the Facility Organization and Management and Workforce Improvement Strategies modules.

PHCPI is a partnership dedicated to transforming the global state of primary health care, beginning with better measurement. While the content on this website represents the position of the partnership a whole, it does not necessarily reflect the official policy or position of any partner organization.

References:

  1. Oliveira T, Barrenho E, Autio A, Barlo J. Developing a Global Healthcare Innovation Index. Policy Innovation Research Unit; 2017.
  2. OECD. Towards a Measurement agenda  for Innovation. OECD; 2010.
  3. PHCPI. Primary Health Care Progression Model. Primary Health Care Performance Initative; 2019.
  4. Mahroum S, Al-Saleh Y. Towards a functional framework for measuring national innovation efficacy. Technovation. 2013 Oct;33(10–11):320–32.
  5. World Health Organization. Accelerator 5. R&D, Innovation and Access. World Health Organization; 2018 Dec.
  6. Chalkidou K, Glassman A, Marten R, Vega J, Teerawattananon Y, Tritasavit N, et al. Priority-setting for achieving universal health coverage. Bull World Health Organ. 2016 Jun 1;94(6):462–7.
  7. World Health Organization. Digital technologies: shaping the future of primary health care. World Health Organization; 2018.
  8. Emprechtinger F. What is innovation culture? [Internet]. LEAD Innovation Management. [cited 2019 Apr 2]. Available from: https://www.lead-innovation.com/english-blog/what-is-innovation-culture
  9. Watkins M. What Is Organizational Culture? And Why Should We Care? [Internet]. 2013 [cited 2019 Jun 14]. Available from: https://hbr.org/2013/05/what-is-organizational-culture
  10. Maher L, Plsek P, Price J, Mugglestone M. Creating the culture for innovation: a Practical Guide for Users. NHS; 2010.
  11. Robert G, Greenhalgh T, MacFarlane F. Organisational factors influencing technology adoption and assimilation in the NHS: a systematic literature review. Report for the National …. 2009;
  12. The International Development Innovation Alliance. Insights on Scaling Innovation. International Development Innovation Alliance; 2017.
  13. Milbergs E. Measuring Innovation for  National Prosperity. Center for Accelerating Innovation; 2004.
  14. LEAD Innovation Management. Identification of innovation potential - 6 important triggers for innovation [Internet]. [cited 2019 May 30]. Available from: https://www.lead-innovation.com/english-blog/identification-of-innovation-potential-6-important-triggers-for-innovation
  15. Seelos C, Mair J. Innovate and Scale: A Tough Balancing Act [Internet]. Stanford Social Innovation Review. 2013 [cited 2019 Jun 14]. Available from: https://ssir.org/articles/entry/innovate_and_scale_a_tough_balancing_act
  16. World Health Organization G. Multisectoral and intersectoral action for improved health and well-being for all: mapping of the WHO European Region. World Health Organization; 2018.
  17. Institute for Healthcare Improvement. Science of Improvement: How to Improve [Internet]. 2019 [cited 2019 May 28]. Available from: http://www.ihi.org/resources/Pages/HowtoImprove/ScienceofImprovementHowtoImprove.aspx
  18. Aarons GA, Hurlburt M, Horwitz SM. Advancing a conceptual model of evidence-based practice implementation in public service sectors. Adm Policy Ment Health. 2011 Jan;38(1):4–23.
  19. Moullin JC, Dickson KS, Stadnick NA, Rabin B, Aarons GA. Systematic review of the Exploration, Preparation, Implementation, Sustainment (EPIS) framework. Implement Sci. 2019 Jan 5;14(1):1.
  20. Keith RE, Crosson JC, O’Malley AS, Cromp D, Taylor EF. Using the Consolidated Framework for Implementation Research (CFIR) to produce actionable findings: a rapid-cycle evaluation approach to improving implementation. Implement Sci. 2017 Feb 10;12(1):15.
  21. Kirk MA, Kelley C, Yankey N, Birken SA, Abadie B, Damschroder L. A systematic review of the use of the Consolidated Framework for Implementation Research. Implement Sci. 2016 May 17;11:72.
  22. Morgan LM. Community participation in health: the politics of primary care in Costa Rica. New York: Cambridge University Press; 1993.
  23. Pesec M, Ratcliffe H, Bitton A, Ratcliffe Msc H, Director C. Building a Thriving Primary Health Care System: the Story of Costa Rica. 2017;
  24. de Compra de Servicios de Salud D. Table de Indicadores 2010-2011. 2010;
  25. Vásquez Evangelisti C. In Person Interview. 2017 Jun;
  26. de Servicios de Salud DC. Informe de resultados de la evaluación de la prestación de servicios de salud 2016. 2017 Jul;
  27. McConnell KJ, Charlesworth CJ, Meath THA, George RM, Kim H. Oregon’s emphasis on equity shows signs of early success for black and american indian medicaid enrollees. Health Aff (Millwood). 2018;37(3):386–93.
  28. Stecker EC. The Oregon ACO experiment--bold design, challenging execution. N Engl J Med. 2013 Mar 14;368(11):982–5.
  29. McConnell KJ, Renfro S, Lindrooth RC, Cohen DJ, Wallace NT, Chernew ME. Oregon’s medicaid reform and transition to global budgets were associated with reductions in expenditures. Health Aff (Millwood). 2017 Mar 1;36(3):451–9.
  30. Arkansas Center for Health Improvement. Arkansas Health Care Payment Improvement Initiative - ACHI [Internet]. [cited 2019 Jun 6]. Available from: https://achi.net/our-initiatives/ahcpii/
  31. American College of Physicians. What is the Patient-Centered Medical Home? (PCMH) | ACP [Internet]. [cited 2019 Jun 6]. Available from: https://www.acponline.org/practice-resources/business-resources/payment/delivery-and-payment-models/patient-centered-medical-home/understanding-the-patient-centered-medical-home/what-is-the-patient-centered-medical-home
  32. Golden W, Thompson J, Olson S, Hill R, Fendrick A, Mathis C, et al. Patient-Centered Medical Homes In Arkansas | Health Affairs. Health Affairs. 2014 May 20;
  33. Centers for Medicare & Medicaid Services. State Innovation Models Initiative: Model Test Awards Round One | Center for Medicare & Medicaid Innovation [Internet]. [cited 2019 Jun 6]. Available from: https://innovation.cms.gov/initiatives/state-innovations-model-testing/
  34. Huelskoetter T. Evaluating State Innovations to Reduce Health Care Costs [Internet]. Center for American Progress. 2018 [cited 2019 Jun 6]. Available from: https://www.americanprogress.org/issues/healthcare/reports/2018/04/06/448912/evaluating-state-innovations-reduce-health-care-costs/
How to use this Model

Brief Overview
Each section includes core principles and relevant background on the topic to help stakeholders understand how this topic relates to other parts of the health system and how to begin improvements.

In-depth exploration
Each section links to various external resources including research papers, toolkits, and implementation guides for readers who would like to learn more.

Diagnostic 
Countries that have a Vital Signs Profile can use it to identify gaps in their performance and find improvement strategies that are specifically relevant to their context. 

Sharing experience and learning from others 
Throughout the model, there are case studies that provide examples of how other countries have implemented reforms and the factors that supported implementation. Additionally, if you think others can learn from improvements in your country, you can submit your experience using the "share your experience" tab. 

Learn more in the Improvement Strategies user guide.