Policy implementation tools

Nineteen studies applied qualitative methods e. Online supplementary information Additional file 3 summarizes the studies including policy implementation measures, key features of methods and tools used to assess barriers and facilitators to policy implementation, and the overall quality rating of each tool.

Most often, food environment policies were assessed together with either nutrition education or physical activity policy, or both [ 42 , 43 , 45 , 47 , 49 , 52 , 53 , 55 , 59 , 60 , 62 , 68 — 77 ].

Fourteen studies focused specifically at national or federal-level policy [ 35 , 36 , 45 , 47 , 49 , 52 , 55 , 59 , 60 , 68 , 71 , 74 , 76 , 79 ], seven focused at state level [ 33 , 53 , 73 , 77 , 78 , 80 , 81 ] and five focused at subnational level, i. One study examined specifically at community level [ 45 ]. Some studies did not specify a particular setting [ 36 , 75 , 77 , 79 , 81 ]. The identified studies mainly investigated the factors barriers and facilitators impacting policy implementation.

Out of 28 studies, 24 focused on the investigation of barriers and facilitators of policy implementation while four focussed on specific issues or problems, such as infrastructure and resource support, stakeholder engagement, and monitoring and evaluation mechanism for implementation [ 42 , 53 , 71 , 77 ].

Other influential factors identified include organizational capacity [ 35 , 55 ], governance [ 42 ] and leadership [ 60 , 70 ]. Of all the studies, 19 studies were conducted using qualitative methods, while four studies used quantitative and five used mixed methods. Semi-structured interviews were most commonly used with a list of open-ended questions to facilitate and guide the interview. Most of the tools were originally developed for use in particular countries.

In several cases, one or more types of qualitative methods were used in one single study. In-depth interviews were most commonly used as primary sources of data [ 33 , 42 , 43 , 45 , 53 , 59 , 68 — 70 , 72 — 75 , 78 — 81 ]. Either policy implementers or both policy implementers and other relevant stakeholders were often recruited as informants for in-depth and focus group interviews.

Other qualitative methods used include document review, field observation and expert review [ 35 , 36 , 53 , 62 , 71 , 78 , 79 ]. Some studies used either a quantitative survey [ 47 , 49 , 52 , 77 ] or mixed methods [ 35 , 55 , 60 , 62 , 73 ] for evaluation.

Twenty-one qualitative tools and eight quantitative tools were reported for evaluating the policy implementation process. Among the qualitative tools used were interview guides, which varied from highly to loosely structured. In some cases, the tools were adapted from existing tools. For example, McDonnell et al.

In several cases, the studies developed their own tools such as a thematic matrix [ 42 ], interview and focus group guides [ 35 , 68 , 72 , 74 , 75 ] and lists of open-ended questions or issues to be explored [ 33 , 43 , 53 , 55 , 59 , 60 , 62 , 69 , 70 , 73 , 78 — 81 ]. Among the tools used, seven qualitative tools were presented data in a form of narrative report while three quantitative tools were based on numerical scores with different forms of data presentation, i. Three studies provided insufficient information for the assessment [ 62 , 71 , 77 ].

There were some broad similarities in the assessed aspects measured by the study and the methods and tools used. Studies evaluating policy implementation processes mainly sought information about barriers and facilitators of policy implementation, particularly infrastructure support and resources, stakeholder engagement, leadership, and available monitoring and evaluation systems, which were the most commonly identified factors which impacted the policy implementation process.

There are no common standard methods and tools used to measure the policy implementation or to assess the policy implementation process. This may be due to the differing contexts and the needs or interests of assessors using these methods.

The three tools that were rated as high quality i. However, there may be scope to include aspects of other tools as part of assessment methods, depending on context-specific requirements and the particular focus required. For example, the Report Card on Healthy Food Environments and Nutrition for Children in Canada included, combined or adapted indicators of several tools used for measuring progress in creating healthy food environments for obesity prevention to fit its purpose and scope and Canadian context [ 41 ].

Consideration should be given to harmonization of the use of methods and tools in this area. While it will always be important to apply tools and methods that are appropriate to the specific context in which they are to be implemented, the use of similar tools in different contexts will allow comparisons across countries and settings and over time.

This will also facilitate effective benchmarking of performance which can help contribute to increasing accountability of governments for their actions to improve the healthiness of food environments. The global impetus to assess policies for changing food environments is relatively new, and the development of appropriate tools for assessing implementation progress in this area is relatively under-developed.

In contrast, in other public health policy areas such as tobacco, alcohol and breast milk, tools are relatively more advanced and have been used for assessing changes over time in a range of countries [ 83 — 88 ].

These approaches share commonalities in terms of types of methods used for assessing policy implementation and provide useful means for the development of healthy food environments. This includes assessment of the strengths and weaknesses of their related policies and programmes. The assessment is conducted every 3—5 years, and the findings and recommendations are actively fed back to policymakers in each country. The main strength of this study is that it is a comprehensive review based on a thorough and systematic search of the literature for policy assessment and evaluation.

To our knowledge, it is the first time such a review has been conducted. The study rated the quality of each tool, and the methods used to conduct the quality assessment could be applied elsewhere. However, this study has several limitations. Firstly, the search was restricted to English-language publications. This may have resulted in the exclusion of important non-English publications. Moreover, studies assessing policy implementation were predominantly from high-income countries rather than low- or middle-income countries.

This may be due to literature search being limited to peer-reviewed studies or grey publications published in English only. It may have missed some relevant documents published in languages other than English, especially documents from countries where English is not the main language.

Furthermore, the studies identified were conducted in different contexts with different focuses, so they may be difficult to compare. The degree to which an approach used in one context is applicable to other contexts is uncertain. However, our findings are consistent with one recent paper identifying that there is little monitoring for accountability globally in this area [ 13 ]. Although there is a growing concern about the impact of unhealthy food environments on the prevalence and severity of obesity and diet-related NCDs globally and nationally, and some governments have implemented policies to improve the healthiness of food environments, a relatively small proportion of the implementation of these policies and actions is being assessed and evaluated.

This review investigated methods and tools used to assess and evaluate the implementation of government policies to create healthy food environments for preventing obesity and diet-related NCDs. It provides a shortlist of high-quality tools and methods for assessing the implementation of such policies.

Harmonization of the use of these high-quality methods and tools is needed to ensure that assessment of government policy implementation can be compared across different countries and settings and over time. The findings from the review are timely in that they provide insights for informing policy implementation and strengthening accountability mechanisms in the context of the increasing prevalence of obesity and diet-related NCDs in low-, middle- and high-income countries.

The global obesity pandemic: shaped by global drivers and local environments. Article PubMed Google Scholar. Dietary energy density is associated with energy intake and weight status in US adults.

Am J Clin Nutr. Dietary energy density and body weight in adults and children: a systematic review. J Acad Nutr Diet. World Health Organization. Global status report on noncommunicable diseases Geneva: World Health Organization; Google Scholar. Monitoring and benchmarking government policies and actions to improve the healthiness of food environments: a proposed government healthy food environment policy index.

Obes Rev. Creating healthy food and eating environments: policy and environmental approaches. Annu Rev Public Health. Global action plan for the prevention and control of noncommunicable diseases — Global nutrition policy review: what does it take to scale up nutrition action? World Cancer Research Fund International. Accessed 6 Nov An accountability framework to promote healthy food environments. Public Health Nutr. Article Google Scholar. Independent global accountability for NCDs.

Strengthening of accountability systems to create healthy food environments and reduce global obesity. Basic epidemiology. Bowling A. Research methods in health: investigating health and health services. Maidenhead: McGraw-Hill Education; Hagger-Johnson G.

Introduction to research methods and data analysis in the health sciences. Hoboken: Taylor and Francis; Hammond M, Wellington J. Research methods: the key concepts. Lawrence M, Worsley A. Public health nutrition: from principles to practice. Nutrition research methodologies. Hoboken: Wiley; Book Google Scholar. Hill R. An electronic journal for the 21st century. Isaac S, Michael WB. Handbook in research and evaluation. Elements of research in nursing. Louis, MO: Mosby; Hertzog MA.

Considerations in determining sample size for pilot studies. Res Nurs Health. Connelly LM. Pilot studies. Medsurg Nurs.

PubMed Google Scholar. Chapter 2. Research method. A systematic review of rural development research: characteristics, design quality and engagement with sustainability.

Cham: Springer; Organisation for Economic Co-operation and Development. Chapter 3. Assessment of learning outcomes and teaching quality in Kazakhstan. Reviews of national policies for education: secondary education in Kazakhstan.

This publication is licensed under the terms of the Open Government Licence v3. To view this licence, visit nationalarchives. Where we have identified any third party copyright information you will need to obtain permission from the copyright holders concerned. We have a lack of distributed capability around delivery across government…much of what I would call the delivery process, or execution process…is not second nature to many people in government.

For implementation to be successful, it must be considered early in policy design. Too often, policies and projects are kicked into action without proper delivery planning, which can lead to a good policy idea failing to achieve the desired outcomes.

The Implementation Unit in the Cabinet Office has developed tools and techniques based on their experience of what makes good implementation, tested with policy experts across Whitehall and further afield. These are shared here for use in your own policy design and delivery. The Implementation Insights are a simple set of steps that, when considered fully during policy design and implementation, provide the foundation for achieving outcomes.

The Insights offer an evidence based approach and tools, tested by policy experts, to help you implement effectively. The full set of questions will help you plan or analyse policies which achieve real world impact. Priorities and actions need to be clear and during the policy development process, to ensure that diverse stakeholders can interpret and implement them consistently at a national level. The context for policy implementation is crucial.

When implementing policy, many other implementation efforts may be occurring at the same time in different parts of the system. Coherence can help to create a climate for implementation success. Legislation may be required before a policy can be fully implemented, or there may be a need to ensure coherence with existing domestic and international legislation.

There may be synergies with, or divergence from other government policies or strategies. Policies may interact with each other, producing new, unplanned and sometimes unintended consequences. More complex governance and accountability arrangements are required to oversee policy implementation. There are many similarities between implementing policies and other types of interventions. Implementation enablers are important for policy implementation.

Examples include leadership, communication and feedback mechanisms. Leadership is needed at all levels of the system for policy implementation. From a political perspective, the appropriate level of leadership is needed to reshape mandates, resources, structures and programmes.

Consistency in leadership has also been suggested as an enabler of implementation, such as fixed-term positions for senior government department officials, to ensure continuity and strengthen relationships. Communication plays an important role in facilitating successful implementation of a policy and should be a core part of policy development from the beginning and throughout the stages of implementation.

Systematic communications are important to share information and feedback on how implementation is progressing across sites, and to share wins and important policy milestones to maintain buy-in and motivation of stakeholders. Feedback mechanisms should be established between policymakers and front-line practitioners once implementation has begun, to ensure the policy is being implemented as intended, unplanned consequences are addressed efficiently and to support the learning capacity of the system.

Feedback mechanisms can include regular, standardised reporting arrangements from front-line services to oversight structures for the policy, and policy reviews carried out at key points in the policy life cycle, for example mid-term reviews.

Implementation of programme and policy initiatives: Making implementation matter Australian National Audit Office, is a better practice guide for public service managers, covering themes such as implementation planning and monitoring and review. Click here. There are a range of terms used when writing or talking about implementation and in implementation science.

This glossary provides a short definition for each of these terms. Elements of an intervention which may be tailored to local settings during implementation without undermining the integrity of the intervention itself. Factors which hinder the implementation process and reduce the probability of successful implementation.

The ability or power to do, understand or absorb something. This can apply to an individual, a team, an organisation or a whole system. A formal, typically short-term, arrangement between a coach and an individual focused on developing work-related skills or behaviours. A group of people living in an area or having characteristics in common e. The action or process of formally discussing something with stakeholders, generally asking stakeholders a relevant question and receiving answers to that question.

While the views of stakeholders may then be used to influence decisions, there is no commitment or requirement to do so. The set of circumstances or unique factors in which implementation takes place. This can refer to both the wider, systemic context, as well as the specific setting in which a specific intervention will be implemented.

Ongoing use of emerging data and evidence on outcomes and implementation, and using that information to learn from experience, inform future implementation and improve outcomes. Progress is, therefore, achieved in an incremental manner over time. Indispensable elements of an intervention or implementation plan, which cannot be changed without undermining it.

All core components should be delivered with fidelity. Using processes for collecting and analysing different types of data to guide decisions with the aim of improving outcomes on an ongoing basis.

A process by which an intervention is communicated through certain channels over time. The spread of ideas through diffusion is generally a passive process, following an unpredictable, unprogrammed, emergent and self-organising path, e. An active, negotiated and influenced means of spreading an intervention or information about an intervention to relevant target groups.

A planned investigation of a project, programme, or policy used to answer specific questions. It can be related to design, implementation, results, and outcomes cause and effect of an intervention. Practices, programmes, policies, strategies or other activities that have been empirically shown through scientific research and evaluation processes to improve outcomes to some degree.

A structure, overview, outline, or system consisting of various descriptive categories and the presumed relationships between them. The carrying out of planned, intentional activities that aim to turn evidence and ideas into policies and practices that work for people in the real world. A plan outlining the key activities, responsibilities, timelines and other important information required to achieve the desired outcomes from implementing an intervention.

The scientific study of how interventions are incorporated into service settings.