Modified Policy-Delphi study for exploring obesity prevention priorities

Abstract

Introduction

Until now, industry and government stakeholders have dominated public discourse about policy options for obesity. While consumer involvement in health service delivery and research has been embraced, methods which engage consumers in health policy development are lacking. Conflicting priorities have generated ethical concern around obesity policy. The concept of ‘intrusiveness’ has been applied to policy decisions in the UK, whereby ethical implications are considered through level of intrusiveness to choice; however, the concept has also been used to avert government regulation to address obesity. The concept of intrusiveness has not been explored from a stakeholder's perspective. The aim is to investigate the relevance of intrusiveness and autonomy to health policy development, and to explore consensus on obesity policy priorities of under-represented stakeholders.

Methods and analysis

The Policy-Delphi technique will be modified using the James Lind Alliance approach to collaborative priority setting. A total of 60 participants will be recruited to represent three stakeholder groups in the Australian context: consumers, public health practitioners and policymakers. A three-round online Policy-Delphi survey will be undertaken. Participants will prioritise options informed by submissions to the 2009 Australian Government Inquiry into Obesity, and rate the intrusiveness of those proposed. An additional round will use qualitative methods in a face-to-face discussion group to explore stakeholder perceptions of the intrusiveness of options. The novelty of this methodology will redress the balance by bringing the consumer voice forward to identify ethically acceptable obesity policy options.

Ethics and dissemination

Ethical approval was granted by the Bond University Health Research Ethics Committee. The findings will inform development of a conceptual framework for analysing and prioritising obesity policy options, which will be relevant internationally and to ethical considerations of wider public health issues. The findings will be disseminated through peer-reviewed publications, conference presentations and collaborative platforms of policy and science.

Correspondence to Emily Haynes; [email protected]

Strengths and limitations of this study

• The novelty of this method brings the under-represented voice forward to identify ethically acceptable obesity policy options.

• The findings will provide a shared understanding of the ethical concepts currently acting as barriers to policy implementation, to encourage the development of counteractive strategies.

• Participants will not represent the full spectrum of stakeholder perspectives in obesity; however, this study aims to redress the balance where certain groups currently dominate the obesity policy debate.

• The Delphi method relies on participant retention between rounds. To address this limitation, time-efficient surveys will be developed, and the initial options will be provided for the participants.

Introduction

Obesity prevalence continues to rise; no country has been successful in reversing the trend in the past 30 years.¹ The rising financial and societal cost of obesity and associated non-communicable disease has led to urgent calls to develop an effective preventative strategy at a global level, with the WHO advocating for cohesive implementation led by federal governments.^{2 3}

There is a lack of empirical evidence to support policy decisions for population wide, complex public health issues such as obesity^{4 5} and ethical concern around regulating individual choice in the context of obesity prevention strategies.^6–9 The concern of developing a ‘nanny state’ by restricting an individual's freedom has shifted focus towards individual responsibility;^7–9 however, the government's role in creating accountability for health promoting environments is recognised as integral to address the epidemic.^{7 10 11}

Evidence-based policy in public health is known to be difficult to develop due to the practicalities of obtaining the ‘high quality’ evidence as traditionally valued in evidence-based medicine.¹² Where evidence for effective intervention is inadequate, stakeholders' opinions are highly valued by experts and may be a useful adjunct to inform policy decisions. Research supports the feasibility of involving a diverse range of stakeholders' perspectives in complex policy decisions;^{13 14} however, in the context of obesity, some perspectives are more dominant than others.¹⁵ Vested interests within some stakeholder groups, in particular those of industry, have been suggested to stimulate conflicting priorities.¹⁶

In the absence of a common tool to guide obesity policy decisions, combining or adapting constructs of existing relevant frameworks may be appropriate to develop appraisal tools.¹⁷ Valuable efforts are underway to monitor the progress of obesity-related policy implementation at national and international levels;^{11 18} however, ‘sophisticating’ investigations of obesity interventions and policy processes, and exploring novel platforms for analysing obesity policy options, have been proposed as integral to accelerating action.^{12 17 19 20} Mapping mutual components of feasible, acceptable and sustainable interventions may therefore be valuable for successful policymaking and implementation by government.

Stakeholder engagement in research methods

Consensus and appraisals methods, such as the Delphi technique, have been successfully applied to explore priorities for public health issues, where evidence for effective policy is inconclusive.^21–23 The Delphi technique, in its original form, intends to gain consensus among ‘experts’ on strategic priorities where there is a lack of empirical evidence.²⁴ The technique traditionally uses a rank or rate approach to assess a variety of options. These options are delivered in consecutive rounds of survey style questions and feedback, and reassessment is encouraged until consensus is gained; however, modifications of the technique have enabled application to a variety of situations and topics.

In the context of obesity, the Delphi technique has been successful in identifying priorities from a solo perspective of ‘experts’,²² but in the light of the diversity of stakeholders involved, there is a possibility to broaden the scope of ‘expertise’ to share opinion across diverse perspectives including local communities.^{23 25} Anonymous sharing of group opinion allows participants to ‘benchmark themselves’ against peer responses,²⁶ and share opinion without potentially destructive group dynamics.²⁷ However, the diversity of priorities, shaped by vested interests, exposure, experience and knowledge, is extensive, and therefore achieving consensus on priorities between stakeholders for obesity may be unrealistic.²³

One modification is the Policy-Delphi technique; this variation explores consensus and dissent, rather than aiming to achieve consensus,²⁸ and provides flexibility over the classic Delphi technique to enable diverse application to various situations.^{26 29} The approach can be used to map overlapping priorities from different perspectives and identify mutual priorities across stakeholder groups and therefore is a valuable exercise for investigating complex public health issues such as obesity.^{26 28 30} The technique facilitates an in-depth investigation which may detect limitations, considerations and consequences of policy options which may enhance the value and success of policy implementation.^{26 31} The diversity of stakeholders involved makes reaching consensus on priorities less feasible;²³ however, mapping perspectives may identify mutual concepts behind the most agreeable options to inform future research and practice. The technique provides an opportunity for participants to contribute equally, and offers additional options and comments throughout; in this respect, it gives all participants, including consumers, a voice in the complex debate.²⁶

A consumer-involvement movement

The public are underexploited in policy advocacy and the decision-making process;³² however, experts recognise the value of the ‘consumer voice’ in ensuring that acceptable, relevant decisions are made both in primary care and the wider political environment. Indeed, public advocacy is required to mobilise policy action and support existing proposals which have been made in the interest of public health.^32–34 Therefore, a growing proportion of health research is engaging patients to identify priorities for research and practice and inform decisions, particularly towards medical treatment.^{13 23 35–40}

All members of society are influenced to some extent by the physical, social and political environment, and therefore subject to the outcome of obesity policy implementation. The voice of industry and academia are suggested as particularly powerful in the obesity debate. In public health, the voice of consumers is rapidly becoming a more integral component to effective research on the priorities for action;^{16 39 41} however, the translation of the findings into practice remains inadequate.

The James Lind Alliance advocates the value of patient-centred practice for identifying research gaps regarding treatment for health conditions. Their approach, termed ‘Priority Setting Partnerships’(PSP),³⁵ was developed to bring the perspectives of the patient, carer and practitioner together, in isolation of vested interests, through transparent methodology, to identify treatment uncertainties which are important to both groups. The underlying principles of the PSP method, such as enabling transparency, enhancing consumer voice and reducing the influence of industry in decision-making, are relevant to the development of a framework to prioritise obesity policy in Australia.

Frameworks for policy development

Ethical frameworks have been proposed as a way to classify and prioritise policy options to government, particularly where there is disagreement between stakeholders. In the UK, the Nuffield Council on Bioethics' ‘Ladder of Intervention’⁴² (box 1) has been used by policymakers as an ethical framework to guide decisions on obesity policy through the concept of ‘intrusiveness’.⁴³ The concept is based on the effect of policy to individuals’ ‘freedom’,⁸ and recent reviews and authors' unpublished observations suggest an association between the level of intrusiveness to choice and the effectiveness of intervention.⁴ Further research proposes that ‘intrusiveness’ and the notion of influencing ‘freedom’ can be better described through the term ‘autonomy’ whereby interventions can enhance or diminish an individual's autonomy in decisions concerning their health (table 1).⁴⁴ Public health interventions which enhance autonomy are generally more acceptable;⁴⁵ however, individual perspective may be governed by how one construes this concept around the original definition of libertarianism.^46–48 Box 1

The Nuffield Council on Bioethics' Ladder of Intervention*³⁸

• Eliminate choice: Regulate in such a way as to entirely eliminate choice, for example, through compulsory isolation of patients with infectious diseases.

• Restrict choice: Regulate in such a way as to restrict the options available to people with the aim of protecting them, for example, removing unhealthy ingredients from foods, or unhealthy foods from shops or restaurants.

• Guide choice through disincentive: Fiscal and other disincentives can be put in place to influence people not to pursue certain activities, for example, through taxes on cigarettes, or by discouraging the use of cars in inner cities through charging schemes or limitations of parking spaces.

• Guide choice through incentive: Regulations can be offered that guide choices by fiscal and other incentives, for example, offering tax-breaks for the purchase of bicycles that are used as a means of travelling to work.

• Guide choice through changing the default policy: For example, in a restaurant, instead of providing chips as a standard side dish (with healthier options available), menus could be changed to provide a more healthy option as standard (with chips as an option available).

• Enable choice: Enable individuals to change their behaviours, for example, by offering participation in an NHS ‘stop smoking’ programme, building cycle lanes or providing free fruit in schools.

• Inform choice: Inform and educate the public, for example, as a part of campaigns to encourage people to walk more or eat five portions of fruit and vegetables per day.

  Do nothing or simply monitor the situation

  *The Ladder illustrates that public health interventions can be classified by a spectrum of levels intrusiveness. These range from the lowest (doing nothing) to the highest (eliminating choice) level of intrusiveness to individual choice.

Table 1

A balanced intervention Ladder*⁴⁰

+5?	Collective self-binding: For example, a decision by a community, after debate and democratic decision-making, to ban the local sale of alcohol.
+4	Enable choice: Enable individuals to change their behaviours, for example by offering participation in an NHS ‘stop smoking’ programme, building cycle lanes or providing free fruit in schools.
+3	Ensure choice is available: For instance, by requiring that menus contain items that someone seeking to maintain health would be likely to choose.
+2	Educate for autonomy: For example through a media studies curriculum which shows children how to recognise the techniques used to manipulate choice through marketing or by banning marketing primarily targeted at children.
+1	Provide information: Inform and educate the public, for example, as part of campaigns which inform people of the health benefits of specific behaviours.
0	Guide choice through changing the default policy: For example, in a restaurant, instead of providing chips as a standard side dish (with healthier options available), menus could be changed to provide a more healthy option as standard (with chips as an option available).
0	Do nothing or simply monitor the situation
−1	Guide choice through incentive: Regulations can be offered that guide choices by fiscal and other incentives, for example, offering tax-breaks for the purchase of bicycles that are used as a means of travelling to work.
−2	Guide choice through disincentive: Fiscal and other disincentives can be put in place to influence people not to pursue certain activities, for example, through taxes on cigarettes, or by discouraging the use of cars in inner cities through charging schemes or limitations of parking spaces.
−3	Restrict choice: Regulate in such a way as to restrict the options available to people with the aim of protecting them, for example, removing unhealthy ingredients from foods, or unhealthy foods from shops or restaurants.
−4	Eliminate choice: Regulate in such a way as to entirely eliminate choice, for example, through compulsory isolation of patients with infectious diseases.

*The Balanced Ladder suggests that public health interventions can be classified across a spectrum of levels according to their influence on autonomy. These levels range from autonomy-diminishing (eliminate choice), to autonomy-enhancing (enable choice).

The question of whether there is an association between intrusiveness and effectiveness is under explored. In spite of general assumptions that societal resistance prevents policymakers from implementing intrusive interventions, the extent to which intrusiveness alters stakeholder perception of policy priorities is unknown. Furthermore, the concepts themselves may be interpreted with variable meaning when applied to complex public health interventions.^47–49 There is no common understanding among stakeholder groups to define ‘intrusiveness’ or ‘autonomy’ in the context of obesity prevention, nor indication of its relevance in the policymaking process. Further insight is required into how stakeholders perceive the intrusiveness of policy options for obesity and whether the concept is, or should be, relevant to policy prioritisation.

Methods and analysis

Design and objectives

This research employs the Policy-Delphi methodology,²⁸ modified and informed by the underlying principles of the James Lind Alliance approach to collaborative priority setting.^{13 35 40 50 51} It will employ quantitative and qualitative methods of analysis to explore stakeholders' understanding of intrusiveness and autonomy, and to gain insight into their perspectives about the relevance of these concepts when considering obesity policy options.

The overarching aim of this study is to explore consensus on stakeholder priorities for obesity prevention policy in Australia, through the concepts of intrusion and autonomy.

The objectives are to:

• Identify the perceived intrusiveness and the cost to autonomy of policy options prioritised by consumers, practitioners and policymakers.

• Identify how stakeholders define concepts of intrusiveness and autonomy in the context of obesity policy, and the levels proposed by the Nuffield Council⁴² and Griffiths.⁴⁴

• Determine the feasibility of using the modified Delphi methodology to prioritise and gain consensus, between stakeholder groups, on policy options for obesity prevention in Australia.

• Identify the extent to which perceived intrusiveness, cost to autonomy and effectiveness govern prioritisation of policy options by stakeholders (box 2).

Box 2

Study outcomes

Primary outcomes:

• Obesity-related policy priorities from public interest stakeholders.

• Intrusiveness and cost to autonomy of stakeholder recommendations.

• A definition or shared understanding of ‘intrusiveness’ and ‘autonomy’ to inform future research.

• Feasibility of conducting a modified-Policy Delphi study for obesity policy research.

• Feasibility of gaining consensus across multiple stakeholder groups.

Primary target for dissemination:

• Public health practitioners

• Policymakers (governmental and non-governmental)

• Research

• Consumers

Participants and recruitment

This study will recruit participants to represent three perspectives: consumers, public health practitioners and policymakers. Strict inclusion criteria will be applied to recruitment (box 3); stakeholders with any commercial conflict of interest, and academics who are not defined as public health practitioners or policymakers, will be excluded from participating. The novelty of this method lies in exploring under-represented perspectives, aligned with values of the James Lind Alliance and the need for strong, impartial evidence and guidance.^{10 35} Industry's influence on obesity policy progress is recognised,^{10 52} and the academic voice is commonly represented in government advisory groups and funded research. The perspective of consumers and public health practitioners is disproportionally represented in the policy process, but is valued as a way to progress policy through public mobilisation: an enabler to political will.⁵² Additionally, the value of including policymakers in the research process is underpinned by their integral role in successful research translation and dissemination.^{53 54} Box 3

Participant eligibility criteria

Inclusion criteria

1. Adults over 18 years of age.

2. Australian resident (we will aim to recruit representation across states).

3. English speaking.

4. Able to provide voluntary consent.

5. Access to a computer, tablet or electronic device and an internet connection to enable completion of the online survey.

6. Must exclusively meet one of the following group inclusion criteria:

a. Public health practitioners: Individuals must be employed by an organisation recognised as relevant in obesity (ie, NGO, health professional).

b. Policymakers (including representatives from government departments, or non-government organisations): Individuals must be employed by a local, state or federal government level department and preferably hold a position concerning policy development, or employed by a non-government organisation and hold a position concerning policy development.

c. Consumers: Individuals must not meet any of the inclusion criteria for groups (a) and (b). They may represent the general community, and will include, for example, parents, workplace managers/staff and teachers.

Exclusion criteria

1. Individuals affiliated with industry through: employment; publicly declared competing interest; in receipt of funding which may influence their contribution; other recognised association.³⁵

2. Academics: defined as those employed in a research community who are not also public health practitioners or policymakers.³¹

3. Any individual in receipt of funding which may influence their contribution to the prioritisation process.³⁵

4. People with a cognitive impairment that prevents them from providing informed consent and understanding the nature of the study.

Relevant individuals will be identified, first, through a review of submissions to the Government Inquiry into Obesity (2009). The study details will be further distributed through social media advertisement and established professional networks of the researchers. A purposive sampling and ‘snowballing’ technique will be used to recruit an information-rich sample of 60 interested participants for the first online survey, including a minimum of 20 from each of the three stakeholder perspectives. There is no consensus on the optimal number of participants required for a Delphi; however, existing research suggests that a purposive sample of this size is sufficient to explore group perspectives and encourage participant retention between rounds.^{22 26 55–57} In accordance with a previously successful Delphi study design,^{22 58} a subsequent smaller sample will participate in face-to-face discussion (n=12–30; from the original 60 recruited for the online survey); this enables prioritised options to be informed by a diverse sample, while also ensuring that the environment is conducive to uninhibited participation during group discussion.^{35 59 60} The Delphi sampling methodology is purposive and inherently biased, as the technique aims to capture rich cases of those with an interest, experience or investment in obesity policy.

Delphi procedure

This modified Policy-Delphi study is structured as a three-round Delphi survey conducted online, followed by a one-day face-to-face discussion workshop, as illustrated in figure 1.

Phase 1: online survey

A list of 30 relevant policy options will be informed by submissions made to the Australian Government Inquiry into Obesity (2009), and nationally relevant food policy recommendations, as identified by the INFORMAS framework.¹⁸ The list will be translated to a survey-style format, and coded under key domains according to setting and target behaviour. The list will represent policy options of various levels of intrusiveness to choice, and cost to the individual's autonomy, as defined by the two previously mentioned ethical frameworks for public health policy (the ‘Ladder of Intervention’⁴² and Balanced Ladder⁴⁴). Each option will be sufficiently detailed to enable categorisation.

Survey Monkey (SurveyMonkey, California, USA) software will be used to develop and distribute each round. Participants will be emailed a link to the survey and invited to complete the first round within 3 weeks; a reminder will be sent if no response is obtained after 14 days.

In round 1 (R1), participants will be advised to read a list of 30 policy options (figure 1). They will be invited to rate each policy option using five-point Likert scales, through four constructs defined in table 2: Priority, Intrusiveness, Cost to Autonomy and Predicted Effectiveness. Participants will be invited to add their own option(s) if those provided do not relate to their preference.

Table 2

Definition of commonly used terms

TermDefinition

Priority	Ranked importance when compared against other options. High priority: Most relevant option. Must be implemented. Priority: Significant importance. Second-order. Low priority: Little importance. Not determining factor to major issue. Unimportant: No relevance. Not for consideration.55
Intrusiveness	The level of intrusion or interference on one's choice to consume healthy or unhealthy food; engage in physical activity or sedentary behaviours; participate in another behaviour which directly affects energy balance, weight gain, loss or maintenance at a given time, within the implemented setting.
Cost to autonomy	The extent to which an option influences one's capacity to self-rule or regulate.
Predicted effectiveness	The perceived, comparative success of a policy option in reducing obesity prevalence, if fully implemented.
Policy option	Any federal, state or local government-led policy action.

We anticipate some diversity between each group’s abilities to prioritise effectively and discriminate between options²³ which will be reported with the study results. To enhance the usability of the data, we will encourage all participants to use the full scale provided, and consider their choice as rankings as well as ratings.²³

Responses to R1 will be pooled with others from the respective stakeholder group. The collective median and IQR of each rating will be calculated independently for each stakeholder group. The median scores for each option will be redistributed in the second round (R2) and participants invited to rerate the 30 options in the light of their peers' response (figure 1). Their individual rating from R1 will be provided as the default, and each option will be colour coded according to the median score for intrusiveness (4–5 red, 3 yellow, 1–2 green).²³

Responses from R2 will be pooled by stakeholder group, and the median and IQR recalculated for each. In R3, the options will be redistributed and displayed in order of intrusiveness, as ranked by the median scores. Participants will be invited to rerate for a final time (figure 1).

The responses to R3 will be totalled to provide a sample of high priority options, the level of consensus within groups (defined by the IQR) and the median rating of intrusiveness, predicted effectiveness and cost to autonomy for each option. Intergroup similarities and differences will be analysed and reported.

Phase 2: discussion group

A subgroup of participants who complete all three rounds will be invited to attend the final phase of the study: a face-to-face discussion group. Purposive sampling will be employed as previously described.

A full day will be allocated and the entire session will be audio recorded. A six-part programme will be delivered during the discussion as detailed in supplementary information (see online supplementary additional file 1). In short, the lead investigator will initiate discussions and group activities designed to elicit the reasoning for the priorities will be identified through the surveys. This will include the rating and relevance of intrusiveness, cost to autonomy and evidence for effectiveness for the options, and participants' interpretation of the concepts of ‘intrusiveness’ and ‘autonomy’. The day will conclude with a final consensus building exercise on the relevance of the concepts discussed, to the identified priorities for implementation.

Data analysis

Quantitative data will be collected from the surveys which will be analysed using basic descriptive statistical tests; frequency, median and IQR.^{22 23 54} The quantitative summary of the combined responses from each stakeholder group will be calculated and distributed to participants in each round. Comparative analysis of similarities and disparities between stakeholder groups will also be undertaken, but not made available to participants in subsequent rounds of the survey. An IQR<1 will be used to indicate consensus for each of the four constructs, and a median priority score between 4 and 5 will define an option as high priority. The median scores for intrusiveness, the cost to autonomy and likely effectiveness will score the option as very low (1), low (2), moderate (3), high (4) or very high (5).

Qualitative data will be collected from the discussion group and will include an audio recording of the full day and photographs of any visual representations provided by the participants (ie, white board work). The recording will be transcribed verbatim and all data will be managed in NvivoV.10 software. Thematic analysis will be conducted using a framework approach, as recommended and commonly employed by qualitative research with similar objectives.^{59 61} The transcript will be read and open-coded by one researcher. The text will be reread, and the codes refined. All coded data will be subsequently clustered into categories to create themes. A constant comparative approach will be used to ensure consistency,⁶² and effort will be made to identify dominant, marginalised or disconfirming data. The data will be charted to provide samples and direct quotes as descriptive examples for each provisional theme. A second researcher will independently analyse the discussion transcript using the same approach and the researchers will come together to verify the key themes.

From the transcripts and derived themes, the researchers will attempt to develop shared understandings of the key constructs (intrusiveness, autonomy) that represent the views of the participants. If consensus on priority options is obtained, these will be included in the final results; however, this is not the primary objective of the study.

The Delphi method has been modified previously to suit the purpose and context of different research questions. The proposed modification facilitates anonymous and face-to-face interaction between participants, to provide quantitative and qualitative data to explore the relevance of the concepts to key perspectives, as supported by existing methods in policy research.^{4 63–66}

Ethics and dissemination

We aim to use the findings to inform a conceptual framework for analysing and prioritising obesity policy options, which may be applied to strengthen proposed frameworks for obesity policy implementation.^{33 34 67} The framework will be relevant internationally and to the ethical considerations of wider public health issues. The findings of this study are particularly relevant to the recent movement towards consumer engagement in health research and policy development, which suggests that all members of society may hold expertise in the acceptability of public policy implementation, through knowledge, experience or simply exposure to the lived environment.^{13 16 23 36 39 41 66 68 69} Furthermore, involving policymakers is considered integral to the successful translation of the findings to practice, and therefore dissemination of the results to those who participated will be considered a priority.

The findings will be disseminated through peer-reviewed publications, conference presentations and collaborative platforms of policy and science. They will provide a novel insight into the perspectives of those under-represented in the obesity debate, on the concept of government intrusion to individual choice: a recognised barrier to government-led implementation of obesity prevention policies,^{9 47 70} to encourage the development of counteractive strategies. Furthermore, where the value of health research in policy process is gaining interest,^{53 71} this research investigates potential research methods for informing policy in public health.

Delphi study status

The list of options and first round survey have been developed and will be piloted internally. The first round survey will be disseminated in September 2016 and the final discussion group is scheduled for November 2016. A paper reporting the results of the Policy-Delphi is anticipated for submission in December 2016.

The authors would like to acknowledge the supervisory support of Professor Paul Glasziou and Professor Roger Hughes in the conception of this study within a broader research project.

References

¹ Ng M, Fleming T, Robinson M, et al. Global, regional, and national prevalence of overweight and obesity in children and adults during 1980–2013: a systematic analysis for the Global Burden of Disease Study 2013. Lancet 2014; 384: 766–81. doi:10.1016/S0140-6736(14)60460-8

² World Health Organisation Executive Board Members. World Health Assembly 2016: a historic breakthrough for child nutrition. 17 December 2015. http://www.aso.org.uk/world-health-assembly-2016-a-historic-breakthrough-for-child-nutrition/ (accessed 29 Dec 2015).

³ Chan M. WHO Director-General addresses health promotion conference. 8th Global Conference on Health Promotion; Helsinki, Finland, 10 June 2013. http://www.who.int/dg/speeches/2013/health_promotion_20130610/en/ (accessed 10 Jan 2016).

⁴ Mayne SL, Auchincloss AH, Michael YL, et al. Impact of policy and built environment changes on obesity-related outcomes: a systematic review of naturally occurring experiments. Obes Rev 2015; 16: 362–75. doi:10.1111/obr.12269

⁵ Sacks G, Swinburn B, Lawrence M. Obesity Policy Action framework and analysis grids for a comprehensive policy approach to reducing obesity. Obes Rev 2009; 10: 76–86. doi:10.1111/j.1467-789X.2008.00524.x

⁶ Swinburn B. Obesity prevention: the role of policies, laws and regulations. Aust New Zealand Health Policy 2008; 5: 12. doi:10.1186/1743-8462-5-12

⁷ Jochelson K. Nanny or steward? The role of the government in public health. Public Health 2006; 120: 1149–55. doi:10.1016/j.puhe.2006.10.009

⁸ Calman L. Beyond the ‘nanny state’: stewardship and public health. Public Health 2009; 123: e6–10. doi:10.1016/j.puhe.2008.10.025

⁹ Crampton P, Hoek J, Beaglehole R. Leadership for health: developing a canny nanny state. N Z Med J 2011; 124: 66–72.

¹⁰ Swinburn B, Kraak V, Rutter H, et al. Strengthening of accountability systems to create healthy food environments and reduce global obesity. Lancet 2015; 385: 2534–45. doi:10.1016/S0140-6736(14)61747-5

¹¹ Vandevijvere S, Swinburn B, International Network for Food and Obesity/non-communicable diseases (NCD's) Research, Monitoring and Action Support (INFORMAS). Towards global benchmarking of food environments and policies to reduce obesity and diet-related non-communicable disease: design and methods for nationwide surveys. BMJ Open 2014; 4: e005339. doi:10.1136/bmjopen-2014-005339

¹² Oliver K, Lorenc T, Innvaer S. New directions in evidence-based policy research: a critical analysis of the literature. Health Res Policy Syst 2014; 12: 34. doi:10.1186/1478-4505-12-34

¹³ Boivin A, Lehoux P, Lacombe R, et al. Involving patients in setting priorities for healthcare improvement: a cluster randomised trial. Implement Sci 2014; 9: 24. doi:10.1186/1748-5908-9-24

¹⁴ Church J, Saunders D, Wanke M, et al. Citizen participation in health decision-making: past experience and future prospects. J Public Health Policy 2002; 23: 12–32. doi:10.2307/3343116

¹⁵ Swinburn B, Wood A. Progress on obesity prevention over 20 years in Australia and New Zealand. Obes Rev 2013; 14 (Suppl 2): 60–8. doi:10.1111/obr.12103

¹⁶ Millstone E, Lobstein T. The PorGrow project: overall cross-national results, comparisons and implications. Obes Rev 2007; 8: 29–36. doi:10.1111/j.1467-789X.2007.00356.x

¹⁷ Phulkard S, Lawrence M, Vandevijvere S, et al. A review of methods and tools to assess the implementation of government policies to create healthy food environments for preventing obesity and diet-related non-communicable diseases. Implement Sci 2016; 11: 15. doi:10.1186/s13012-016-0379-5

¹⁸ Swinburn B, Sacks G, Vandevijvere S, et al., INFORMAS. INFORMAS (International Network for Food and Obesity/non-communicable diseases Research, Monitoring and Action Support): overview and key principles. Obes Rev 2013; 14: 1–12. doi:10.1111/obr.12087

¹⁹ Must A, Barish EE, Bandini LG. Modificable risk factors in relation to changes in BMI and fatness: what have we learnt from prospective studies of school-aged children? Int J Obes (Lond) 2009; 33: 705–15. doi:10.1038/ijo.2009.60

²⁰ Soderberg E, Wikstrom E. The policy process for health promotion. Scand J Public Health 2015; 43: 606–14. doi:10.1177/1403494815586327

²¹ Stebler N, Schuepbach-Regula G, Braam P, et al. Use of a modified Delphi panel to identify and weight criteria for prioritization of zoonotic diseases in Switzerland. Prev Vet Med 2015; 121: 165–9. doi:10.1016/j.prevetmed.2015.05.006

²² Faulkner GEJ, Grootendorst P, Hai Nguyen V, et al. Economic instruments for obesity prevention: results of a scoping review and modified Delphi survey. Int J Behav Nutr Phys Act 2011; 8: 109. doi:10.1186/1479-5868-8-109

²³ Owens C, Ley A, Aitken P. Do different stakeholder groups share mental health research priorities? A four-arm Delphi study. Health Expect 2008; 11: 418–31. doi:10.1111/j.1369-7625.2008.00492.x

²⁴ Dalkey NC. Delphi. The RAND Corporation. P-3704. http://www.rand.org/pubs/papers/P3704.html (accessed Nov 2015).

²⁵ Rideout C, Gil R, Browne R, et al. Using the Delphi and snow card techniques to build consensus among diverse community and academic stakeholders. Prog Community Health Partnersh 2013; 7: 331–9. doi:10.1353/cpr.2013.0033

²⁶ Meskell P, Murphy K, Shaw DG, et al. Insights into the use and complexities of the Policy Delphi technique. Nurse Res 2013; 21: 32–9. doi:10.7748/nr2014.01.21.3.32.e342

²⁷ Murphy MK, Black NA, Lamping DL, et al. Consensus development methods, and their use in clinical guidelines development. Health Technol Assess 1998; 2: i–iv, 1–88. http://www.journalslibrary.nihr.ac.uk/__data/assets/pdf_file/0004/59512/ExecutiveSummary-hta2030.pdf (accessed Dec 2015).

²⁸ Turoff M. The design of a policy Delphi. Technological Forecasting and Social Change 1970; 2: 149–71.

²⁹ Williams PL, Webb C. The Delphi technique: a methodological discussion. J Adv Nurs 1994; 19: 180–6.

³⁰ von der Gracht H. Consensus measurement in Delphi studies. Review and implications for future quality assurance. Technological Forecasting and Social Change 2012; 79: 1525–36. doi:10.1016/j.techfore.2012.04.013

³¹ Pratt J. A Popperian approach to policy research. In: Swann J, Pratt J, eds. Educational research in practice: making sense of methodology. London: Continuum, 2003: 51–66.

³² Huang TT, Cawley J, Ashe M, et al. Mobilisation of public support for policy actions to prevent obesity. Lancet 2015; 385: 2422–31. doi:10.1016/S0140-6736(14)61743-8

³³ World Health Organisation. Global action plan for the prevention and control of non-communicable disease 2013–2020. Geneva: World Health Organisation, 2013.

³⁴ World Health Organisation. Report of the commission on ending childhood obesity. Geneva: World Health Organisation, 2016.

³⁵ Cowen K, Oliver S. The James Lind Alliance Guidebook (version 5). 2013. http://www.jlaguidebook.org/ (accessed Oct 2015).

³⁶ Hanley B, Bradburn J, Barnes M, et al. Involving the public in the NHS, public health and social care research: briefing notes for researchers. 2nd edn. Involve, 2004. http://www.twocanassociates.co.uk/perch/resources/files/Briefing%20Note%20Final_dat(1).pdf (accessed Nov 2015).

³⁷ Oliver S, Clarke-Jones L, Rees R, et al. Involving consumers in research and development agenda setting for the NHS: developing an evidence-based approach. Health Technol Assess 2004; 8: 1–148, III–IV. doi:10.1017/S026646230400073X

³⁸ Oliver S, Gray J. A bibliography of research reports about patients’, clinicians’ and researchers’ priorities for new research. London: James Lind Alliance, December 2006.

³⁹ VicHealth. Citizens Jury on Obesity. Report of the Jury. 2015. https://www.vichealth.vic.gov.au/programs-and-projects/victorias-citizens-jury-on-obesity (accessed Dec 2015).

⁴⁰ Wilson MG, Lavis JN. Rapid synthesis: engaging in priority setting about primary and integrated healthcare innovations in Canada. Hamilton, Canada: McMaster Health Forum, 31 March 2014. https://www.mcmasterhealthforum.org/docs/default-source/Product-Documents/rapid-responses/engaging-in-priority-setting-about-primary-and-integrated-healthcare-innovations-in-canada.pdf?sfvrsn=2 (accessed Nov 2015).

⁴¹ Queensland Government. Engaging Queenslanders: community engagement in the business of government (EQ). 2005. http://www.qld.gov.au/web/community-engagement/guides-factsheets/business-government/government-policy.html (accessed 12 Jan 2016).

⁴² Nuffield Council on Bioethics. Public Health: ethical issues. London, UK, 2007. http://nuffieldbioethics.org/project/public-health/ (accessed 20 Nov 2015).

⁴³ Jebb SA, Aveyard PN, Hawkes C. The evolution of policy and actions to tackle obesity in England. Obes Rev 2013; 14: 42–59. doi:10.1111/obr.12093

⁴⁴ Griffiths PE, West C. A balanced intervention ladder: promoting autonomy through public health action. Public Health 2015; 129: 1092–8. doi:10.1016/j.puhe.2015.08.007

⁴⁵ Diepeveen S, Ling T, Suhrcke M, et al. Public acceptability of government intervention to change health-related behaviours: a systematic review and narrative synthesis. BMC Public Health 2013; 13: 756. doi:10.1186/1471-2458-13-756

⁴⁶ Mill JS. On liberty. 2nd edn. Mineola, NY: Dover Publication, 1859.

⁴⁷ Barnhill A, King KF. Ethical agreement and disagreement about obesity prevention policy in the United States. Int J Health Policy Manag 2013; 1: 117–20. doi:10.15171/ijhpm.2013.21

⁴⁸ Buchanan D. Ethical standards to guide the development of obesity policies and programs; Comment on “Ethical agreement and disagreement about obesity prevention policy in the United States”. Int J Health Policy Manag 2013; 1: 313–15. doi:10.15171/ijhpm.2013.63

⁴⁹ The National Public Health Partnership's (NPHP). Planning Framework for Public Health Practice Deciding and Specifying an Intervention Portfolio. Public Health Planning and Practice Improvement sets out a systematic approach to planning the provision of public health interventions. 2000. http://www.health.nsw.gov.au/research/Documents/planning-framework.pdf (accessed Dec 2014).

⁵⁰ Nielsen ES, Myrhaug HT, Johansen M, et al. Methods of consumer involvement in developing healthcare policy research, clinical practice guidelines and patient information material. Cochrane Database Syst Rev 2006; (3): CD004563.

⁵¹ Florin D, Dixon J. Public involvement in healthcare. BMJ 2004; 328: 159–61. doi:10.1136/bmj.328.7432.159

⁵² Cullerton K, Donnet T, Lee A, et al. Playing the policy game: a review of the barriers to and enablers of nutrition policy change. Public Health Nutr 2016: 1–11. http://journals.cambridge.org/action/displayAbstract?fromPage=online&aid=10263715&fulltextType=RA&fileId=S1368980016000677 (accessed 4 May 2016). doi:10.1017/S1368980016000677

⁵³ Tricco AC, Cardoso R, Thomas SM. Barriers and facilitators to uptake if systematic reviews by policymakers and healthcare managers: a scoping review. Implement Sci 2016; 11: 4. doi:10.1186/s13012-016-0370-1

⁵⁴ Choi B, Li L, Lu Y, et al. Bridging the gap between science and policy: an international survey of scientists and policymakers in China and Canada. Implement Sci 2016; 11: 16. doi:10.1186/s13012-016-0377-7

⁵⁵ Linstone HA, Turoff M. The Delphi Method: techniques and applications. 2002. http://is.njit.edu/pubs/delphibook/delphibook.pdf (accessed Oct 2015).

⁵⁶ Okoli C, Pawlowski SD. The Delphi Method as a research tool: an example, design considerations and applications. Inform Manag 2004; 42: 15–29. doi:10.1016/j.intramuscular.2003.11.002

⁵⁷ Slade SC, Dionne CE, Underwood M, et al. Standardised method for reporting exercise programmes: protocol for a modified Delphi study. BMJ Open 2014; 4: e006682. doi:10.1136/bmjopen-2014-006682

⁵⁸ Hoffmann T, Glasziou P, Boutron I, et al. Better reporting of interventions: template for intervention description and replication (TIDieR) checklist and guide. BMJ 2014; 348: g1687. doi:10.1136/bmj.g1687

⁵⁹ Brown I, Gould J, et al. Qualitative studies of obesity: a review of methodology. Health 2013; 5: 69–80. doi:10.4236/health.2013.58A3010

⁶⁰ Tong A, Sainsbury P, Craig J. Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups. Int J Qual Healthcare 2007; 19: 349–57. doi:10.1093/intqhc/mzm042

⁶¹ Ritchie J, Spencer L, O'Connor W. Carrying out qualitative analysis. In: Ritchie J, Lewis J, eds. Qualitative research practice. 3rd edn. London: Sage Publications, 2010: 237.

⁶² Glaser BG. The constant comparative method of qualitative analysis. Grounded Theory Rev 2008; 7: 1–10.

⁶³ Turnpenny J, Radaelli CM, Jordan A, et al. The Policy and politics of policy appraisal: emerging trends and new directions. J Eur Public Policy 2009; 16: 640–53. doi:10.1080/13501760902872783

⁶⁴ Sharma T, Littlejohns P, Choudhury M, et al. Evidence informed decision-making: the use of ‘colloquial evidence’ at NICE. Int J Technol Assess Healthcare 2015; 31: 138–46. doi:10.1017/S0266462314000749

⁶⁵ Degeling C, Carter SM, Rychetnik L. Which public and why deliberate? A scoping review of public deliberation in public health and health policy research. Soc Sci Med 2015; 131: 114–21. doi:10.1016/j.socscimed.2015.03.009

⁶⁶ Paul C, Nicholls R, Preist P, et al. Making policy decisions about population screening for breast cancer: the role of citizens’ deliberation. Health Policy 2008; 85: 314–20. doi:10.1016/j.healthpol.2007.08.007

⁶⁷ Centers for Disease Control and Prevention. Using evaluation to inform CDC's policy process. Atlanta, GA: Centers for Disease Control and Prevention, US Department of Health and Human Services, 2014 (cited July 2016). http://www.cdc.gov/policy/analysis/process/docs/usingevaluationtoinformcdcspolicyprocess.pdf

⁶⁸ Uhm S, Liabo K, Stewart R, et al. Patient and public perspectives shaping scientific and medical research: panels for data, discussions and decisions. Patient Intelligence 2012; 4: 1–10.

⁶⁹ Rychetnik L, Doust J, Thomas R, et al. A Community Jury on PSA screening: what do well informed men want the government to do about prostate cancer screening—a qualitative analysis. BMJ Open 2014; 4: e004682. doi:10.1136/bmjopen-2013-004682

⁷⁰ Nestle M. Food Politics: how the food industry influences nutrition and health. University of California Press, 2013.

⁷¹ Kite J, Indig D, Mihrshahi S, et al. Assessing the usefulness of systematic reviews for policymakers in public health: a case study of overweight and obesity prevention interventions. Prev Med 2015; 81: 99–107. doi:10.1016/j.ypmed.2015.08.012