Legitimacy in Policing: A Systematic Review

Abstract

BACKGROUND

Police require voluntary cooperation from the general public to be effective in controlling crime and maintaining order. Research shows that citizens are more likely to comply and cooperate with police and obey the law when they view the police as legitimate. The most common pathway that the police use to increase citizen perceptions of legitimacy is through the use of procedural justice. Procedural justice, as described in the literature, comprises four essential components. These components are citizen participation in the proceedings prior to an authority reaching a decision (or voice), perceived neutrality of the authority in making the decision, whether or not the authority showed dignity and respect toward citizens throughout the interaction, and whether or not the authority conveyed trustworthy motives.

Police departments throughout the world are implicitly and explicitly weaving the dialogue of these four principles of procedural justice (treating people with dignity and respect, giving citizens "voice" during encounters, being neutral in decision making, and conveying trustworthy motives) into their operational policing programs and interventions.

OBJECTIVES

This review synthesizes published and unpublished empirical evidence on the impact of interventions led by the public police to enhance citizen perceptions of police legitimacy. Our objective is to provide a systematic review of the direct and indirect benefits of policing approaches that foster legitimacy in policing that either report an explicit statement that the intervention sought to increase legitimacy or report that there was an application of at least one of the principles of procedural justice: participation, neutrality, dignity/respect, and trustworthy motives.

SEARCH STRATEGY

Studies were identified using six electronic databases (CSA, Informit, Ingenta Connect, Ovid, Proquest and Web of Knowledge) and two library catalogues (National Police Library and the Cambridge University Library and dependent libraries). We also searched the reference list of each eligible study, and reviewed the biographies and publication lists of influential authors in the field of procedural justice and police legitimacy, to determine if there were any relevant studies not retrieved in the original search.

SELECTION CRITERIA

Studies were included if they described any type of public police intervention (e.g. routine patrols, traffic stops, community policing, reassurance policing, problem-oriented policing, conferencing) that either explicitly stated that the intervention was aimed at improving police legitimacy (through either a directive, training or organizational innovation) or explicitly used at least one of the principles of procedural justice. Studies had to include at least one direct outcome, such as citizen compliance, cooperation, or satisfaction with police, aimed at improving legitimacy, and could also include indirect outcomes, such as reduction in reoffending, or crime and social disorder. We included only studies that evaluated interventions if they were led by public police from any level of government (i.e., local, state and federal law enforcement officers). To be included in the systematic review, studies must have used one of the following research designs: an experimental (randomized) design involving at least two conditions, with one condition being the intervention and the other a control condition; a quasi-experimental (non-randomized) design involving at least two conditions, with one condition being the intervention and the other a comparison condition; a quasi-experimental interrupted time-series design that involved measurement of an aggregate outcome, such as crime rate, in equally spaced time intervals prior to and following the initiation of the police-led intervention.

DATA COLLECTION AND ANALYSIS

The systematic search identified 963 unique studies on police legitimacy and/or procedural justice and policing, of which 933 were obtained. Of those, 163 studies reported on police-led interventions. A final set of 30 studies, containing 41 independent evaluations, was eligible for meta-analysis. Data analysis was conducted using Comprehensive Meta-Analysis 2.0 (CMA), a statistical meta-analysis software package. We conducted separate meta-analyses using random effects models for each outcome of policing interventions that had been measured by at least two evaluations. The outcomes analyzed were: Direct - legitimacy, procedural justice, cooperation/compliance, and satisfaction/confidence; Indirect - revictimization or reoffending. We obtained or calculated a single effect size per study per outcome, either a standardized mean difference (g) for a continuous outcome, or an odds ratios for outcomes reported as dichotomous.

We also explored possible moderators of policing legitimacy including intervention type, research design, respondent type, crime type, year of publication, and country of publication, using analogs to the ANOVA implemented via subgroup analyses in CMA. In addition, we conducted a series of sensitivity analyses to test the robustness of the results to the following: inclusion of studies where data was imputed, inclusion of poor quality studies (e.g. lack of treatment integrity), and we inspected possible sources of bias in the data, including publication bias and small-study effects.

RESULTS

There were 41 independent evaluations available for meta-analysis: 7 assessed legitimacy as an outcome, 14 assessed procedural justice, 8 assessed compliance/cooperation, 29 assessed satisfaction/confidence, and 26 assessed reoffending. The direct outcome satisfaction/confidence showed the highest overall effect that was statistically significant (OR 1.75, 95% confidence limits 1.54, 1.99), followed by compliance/cooperation (OR 1.62, 95% confidence limits 1.13, 2.32), and procedural justice (OR 1.47, 95% confidence limits 1.16, 1.86). The estimated effect size for the direct outcome legitimacy (OR 1.58, 95% confidence limits 0.85, 2.95), while quite large, has a wide confidence interval, indicating a high degree of uncertainty around the estimate. Interventions showed a marginal effect on reoffending as an indirect outcome measure (g = -0.07, 95% confidence limits -0.14, 0.00). When reoffending was broken down by measurement method, studies that measured reoffending using official police data and self-reported reoffending showed no effect (g = 0.03, 95% confidence limits -0.05, 0.11); however, studies that measured self-reported victimization showed a large decrease in revictimization as a result of the interventions (g = -0.13, 95% confidence limits -0.23, -0.05).

AUTHORS' CONCLUSIONS

The main finding of this review is that the effects of legitimacy policing interventions on each direct outcome measure are in a positive direction. For all but the legitimacy outcome, the results were statistically significant. We note that there is a clear lack of randomized experiments in the international research literature that specifically seek to isolate and test the component parts of a legitimacy policing intervention. Notwithstanding the variability in the mode in which legitimacy policing is delivered (i.e., the study intervention) and the complexities around measurement of legitimacy outcomes, our review shows that the dialogue component of front-line police-led interventions is important for promoting citizen satisfaction, confidence, compliance and cooperation with the police, and for enhancing perceptions of procedural justice. In practical terms, this means that police can achieve positive changes in citizen attitudes to police through adopting procedurally justice dialogue as a component part of any type of police intervention. We conclude that the type of police intervention (the vehicle for delivering a procedurally just encounter) is secondary to the procedurally just dialogue that underpins the intervention.

Campbell Systematic Reviews

2013:1

First published: 02 January, 2013 Last updated: July, 2012 Search executed: 01 April, 2009

Legitimacy in Policing: A Systematic Review

Lorraine Mazerolle, Sarah Bennett, Jacqueline Davis, Elise Sargeant and Matthew Manning

Colophon

Title Legitimacy in Policing: A Systematic Review

Institution The Campbell Collaboration

Authors Mazerolle, Lorraine Bennett, Sarah Davis, Jacqueline Sargeant, Elise Manning, Matthew

DOI 10.4073/csr.2013.1

No. of pages 146

Last updated April, 2012

Citation Mazerolle L, Bennett S, Davis J, Sargeant E, Manning M. Legitimacy in Policing: A Systematic Review. Campbell Systematic Reviews 2013:1 DOI: 10.4073/csr.2013.1

ISSN 1891-1803

Copyright Mazerolle et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Contributions Not specified

Editors for this review

Editor: David Wilson Managing editor: Charlotte Gill

Support/funding Financial support for the systematic search was provided by a grant from the National Policing Improvement Agency (NPIA). Financial support for the systematic review was provided by George Mason University (sub-award no. E2216645). Additional financial support was provided by the Institute for Social Science Research at the University of Queensland and the ARC Centre of Excellence in Policing and Security (CEPS).

Potential conflicts of interest

Professor Lorraine Mazerolle, Dr Sarah Bennett, Jacqueline Davis, Elise Sargeant and Dr Matthew Manning have no financial interest in any of the existing research being conducted in this area of research (legitimacy in policing). The authors have no vested interest in the treatments that are the subject of this review or in the outcome of the review. The authors also do not have any incentives to represent findings in a biased manner and strive to avoid any potential conflict.

Corresponding author

Professor Lorraine Mazerolle

The University of Queensland

Institute for Social Science Research &

ARC Centre of Excellence in Policing and Security (CEPS) GPN3, Room 409

Brisbane, Queensland 4072, Australia

Mobile: [phone]

Landline: 61 7 3346 7877

E-mail: [email protected]

Campbell Systematic Reviews

Editors-in-Chief Mark W. Lipsey, Vanderbilt University, USA Arild Bjrndal, The Centre for Child and Adolescent Mental Health, Eastern and Southern Norway & University of Oslo, Norway

Editors

Crime and Justice David B. Wilson, George Mason University, USA

Education Sandra Wilson, Vanderbilt University, USA

Social Welfare William Turner, University of Bristol, UK Geraldine Macdonald, Queens University, UK & Cochrane Developmental, Psychosocial and Learning Problems Group

International

Development

Peter Tugwell, University of Ottawa, Canada Howard White, 3ie, India

Methods Therese Pigott, Loyola University, USA

Ian Shemilt, University of Cambridge, UK

The Campbell Collaboration (C2) was founded on the principle that systematic reviews on the effects of interventions will inform and help improve policy and services. C2 offers editorial and methodological support to review authors throughout the process of producing a systematic review. A number of C2's editors, librarians, methodologists and external peer-reviewers contribute.

The Campbell Collaboration P.O. Box 7004 St. Olavs plass0130 Oslo, Norway http://www.campbellcollaboration.org/

Web End =www.campbellcollaboration.org

Birte Snilstveit, 3ie, UK Hugh Waddington, 3ie, UK

Managing Editor Karianne Thune Hammerstrm, The Campbell Collaboration

Editorial Board

Crime and Justice David B. Wilson, George Mason University, USA Martin Killias, University of Zurich, Switzerland

Education Paul Connolly, Queen's University, UK Gary W. Ritter, University of Arkansas, USA

Social Welfare Aron Shlonsky, University of Toronto, Canada Jane Barlow, University of Warwick, UK

International

Development

Table of contents

TABLE OF CONTENTS ......................................................................... III

FIGURES .............................................................................................. VI

TABLES ............................................................................................... VII

EXECUTIVE SUMMARY/ABSTRACT ..................................................... 8

Background ................................................................................................................................

8

Objectives ...................................................................................................................................

8

Search Strategy ..........................................................................................................................

8

Selection Criteria ....................................................................................................................... 9Data collection and Analysis ...................................................................................................... 9Results ...................................................................................................................................... 10Authors Conclusions ............................................................................................................... 10

PLAIN LANGUAGE SUMMARY .............................................................. 11

4 BACKGROUND FOR THE REVIEW ................................................. 12

4.1 Logic Model................................................................................................................... 14 4.2 Procedural Justice ........................................................................................................ 15 4.3 Police Performance and Legitimacy ............................................................................. 17 4.4 Distributive Justice ....................................................................................................... 19 4.5 Legal Legitimacy ...........................................................................................................20 4.6 Traditional, historical and cultural legitimacy stemming from tradition .................... 21 4.7 Types of police interventions to increase legitimacy .................................................... 21 4.8 Purpose and contribution ............................................................................................. 22

5 OBJECTIVES OF THE REVIEW ....................................................... 24

6 METHODS ....................................................................................... 27

6.1 Criteria for inclusion and exclusion of studies in the review........................................ 27 6.1.1 Period of time to be covered by systematic review ................................................................... 27

6.1.2 Types of study designs ............................................................................................................... 28

3.1.3 Types of statistical data ............................................................................................................. 29

3.1.4 Types of participants/units of analysis ..................................................................................... 29

iii The Campbell Collaboration | www.campbellcollaboration.org

3.1.5 Types of police-led legitimacy interventions ............................................................................ 29

3.1.6 Types of outcome measures ...................................................................................................... 30

3.1.7 Language and geographic origin ............................................................................................... 33

3.1.8 Publication status ....................................................................................................................... 33

3.2 Search strategy for identification of relevant studies ................................................... 33 3.2.1 Keyword formulation ................................................................................................................. 34

3.2.2 Search Field ............................................................................................................................ 36

3.2.3 Database selection ................................................................................................................. 36

3.2.4 Additional Searches ............................................................................................................... 38

3.3 Extracting and coding research for the review ............................................................. 38 3.4 Recruitment, training and inter-rater reliability ......................................................... 40 3.4.1 Recruitment ............................................................................................................................... 40

3.4.1 Training, coding and test of inter-rater reliability ................................................................... 40

3.5 Statistical procedures and conventions ....................................................................... 40

4 RESULTS ......................................................................................... 43

4.1 Systematic Review Results............................................................................................ 43 4.1.1 Characteristics of included studies ........................................................................................... 44

4.2 Meta-analysis ................................................................................................................ 49 4.2.1 Moderator analyses .................................................................................................................... 50

4.2.2 Sensitivity analyses ................................................................................................................ 50

4.3 Direct Outcomes ........................................................................................................... 54 4.3.1 Perceived Legitimacy ................................................................................................................. 56

4.3.2 Perceived Procedural Justice ................................................................................................ 59

4.3.3 Compliance and Cooperation ................................................................................................ 62

4.3.4 Satisfaction and Confidence .................................................................................................. 65

4.4 Indirect Outcomes ........................................................................................................ 69 4.4.1 Reoffending ................................................................................................................................ 69

4.5 Results Summary .......................................................................................................... 73

5 DISCUSSION AND CONCLUSION .................................................... 75

6 PLANS FOR UPDATING THE REVIEW ............................................ 79

7 ACKNOWLEDGMENTS .................................................................. 80

8 STATEMENT CONCERNING CONFLICT OF INTEREST ................... 81

9 REFERENCES ................................................................................. 82

APPENDIX 1: INVENTORY OF POLICE-LED LEGITIMACY INTERVENTIONS ................................................................................ 88

APPENDIX 2: TECHNICAL APPENDIX ................................................. 97

iv The Campbell Collaboration | www.campbellcollaboration.org

APPENDIX 3: REFERENCES OF ELIGIBLE STUDIES.......................... 109

APPENDIX 4: CODING SHEET FOR LEGITIMACY IN POLICING REVIEW .............................................................................................. 121

v The Campbell Collaboration | www.campbellcollaboration.org

Figures

Figure 1. Theoretical model of police legitimacy process .............................................................. 15

Figure 2. Funnel plot of standard error by log odds ratio: All included direct outcomemeasures (i.e., >1 outcome measure per study) .......................................................... 53

Figure 3. Funnel plot of standard error by log odds ratio: All included indirect outcome measures (i.e., >1 outcome measure per study) .......................................................... 54

Figure 4. Forest plot of effect sizes, legitimacy ............................................................................. 58

Figure 5. Forest plot of effect sizes, procedural justice ................................................................. 61

Figure 6. Forest plot of effect sizes, compliance ........................................................................... 64

Figure 7. Forest plot of effect sizes, satisfaction and confidence .................................................. 68

Figure 8. Forest plot of effect sizes, reoffending............................................................................

72

vi The Campbell Collaboration | www.campbellcollaboration.org

Tables

Table 1. Keywords for systematic literature search ...................................................................... 35

Table 2. Attrition of publications .................................................................................................. 44

Table 3. Individual study characteristics .......................................................................................47

Table 4. List of published and unpublished sources ..................................................................... 51

Table 5. Relationship of study characteristics to odds ratio effect sizes for the outcomeof legitimacy (k = 7) ...................................................................................................... 57

Table 6. Effects of study quality indicators on the outcome of perceived legitimacy

(k=7) .............................................................................................................................. 57

Table 7. Relationship of study characteristics to odds ratio effect sizes for the outcomeof perceived procedural justice (k = 14) ....................................................................... 59

Table 8. Effects of study quality indicators on the outcome of procedural justice (k=14) ........... 60

Table 9. Relationship of study characteristics to odds ratio effect sizes for the outcomeof compliance (k = 8) ................................................................................................... 62

Table 10. Effects of study quality indicators on the outcome of compliance (k=8) ..................... 63

Table 11. Relationship of study characteristics to odds ratio effect sizes for the outcomeof satisfaction (k = 29) ................................................................................................. 65

Table 12. Effects of study quality indicators on the outcome of satisfaction/confidence

(k=29) ...........................................................................................................................67

Table 13. Relationship of study characteristics to Hedges' g effect sizes for the outcomeof reoffending (k = 8) ................................................................................................... 70

Table 14. Effects of study quality indicators on the outcome of reoffending (k=8) ...................... 71

Table 15. Summary of results for direct outcomes ........................................................................74

Table 16. Summary of results for indirect outcomes .....................................................................74

vii The Campbell Collaboration | www.campbellcollaboration.org

Executive Summary/Abstract

BACKGROUND

Police require voluntary cooperation from the general public to be effective in controlling crime and maintaining order. Research shows that citizens are more likely to comply and cooperate with police and obey the law when they view the police as legitimate. The most common pathway that the police use to increase citizen perceptions of legitimacy is through the use of procedural justice. Procedural justice, as described in the literature, comprises four essential components. These components are citizen participation in the proceedings prior to an authority reaching a decision (or voice), perceived neutrality of the authority in making the decision, whether or not the authority showed dignity and respect toward citizens throughout the interaction, and whether or not the authority conveyed trustworthy motives.

Police departments throughout the world are implicitly and explicitly weaving the dialogue of these four principles of procedural justice (treating people with dignity and respect, giving citizens voice during encounters, being neutral in decision making, and conveying trustworthy motives) into their operational policing programs and interventions.

OBJECTIVES

This review synthesizes published and unpublished empirical evidence on the impact of interventions led by the public police to enhance citizen perceptions of police legitimacy. Our objective is to provide a systematic review of the direct and indirect benefits of policing approaches that foster legitimacy in policing that either report an explicit statement that the intervention sought to increase legitimacy or report that there was an application of at least one of the principles of procedural justice: participation, neutrality, dignity/respect, and trustworthy motives.

SEARCH STRATEGY

Studies were identified using six electronic databases (CSA, Informit, Ingenta Connect, Ovid, Proquest and Web of Knowledge) and two library catalogues (National Police Library and the Cambridge University Library and dependent libraries). We also searched the reference list

8 The Campbell Collaboration | www.campbellcollaboration.org

of each eligible study, and reviewed the biographies and publication lists of influential authors in the field of procedural justice and police legitimacy, to determine if there were any relevant studies not retrieved in the original search.

SELECTION CRITERIA

Studies were included if they described any type of public police intervention (e.g. routine patrols, traffic stops, community policing, reassurance policing, problem-oriented policing, conferencing) that either explicitly stated that the intervention was aimed at improving police legitimacy (through either a directive, training or organizational innovation) or explicitly used at least one of the principles of procedural justice. Studies had to include at least one direct outcome, such as citizen compliance, cooperation, or satisfaction with police, aimed at improving legitimacy, and could also include indirect outcomes, such as reduction in reoffending, or crime and social disorder. We included only studies that evaluated interventions if they were led by public police from any level of government (i.e., local, state and federal law enforcement officers). To be included in the systematic review, studies must have used one of the following research designs: an experimental (randomized) design involving at least two conditions, with one condition being the intervention and the other a control condition; a quasi-experimental (non-randomized) design involving at least two conditions, with one condition being the intervention and the other a comparison condition; a quasi-experimental interrupted time-series design that involved measurement of an aggregate outcome, such as crime rate, in equally spaced time intervals prior to and following the initiation of the police-led intervention.

DATA COLLECTION AND ANALYSIS

The systematic search identified 963 unique studies on police legitimacy and/or procedural justice and policing, of which 933 were obtained. Of those, 163 studies reported on police-led interventions. A final set of 30 studies, containing 41 independent evaluations, was eligible for meta-analysis. Data analysis was conducted using Comprehensive Meta-Analysis 2.0 (CMA), a statistical meta-analysis software package. We conducted separate meta-analyses using random effects models for each outcome of policing interventions that had been measured by at least two evaluations. The outcomes analyzed were: Direct legitimacy, procedural justice, cooperation/compliance, and satisfaction/confidence; Indirect revictimization or reoffending. We obtained or calculated a single effect size per study per outcome, either a standardized mean difference (g) for a continuous outcome, or an odds ratios for outcomes reported as dichotomous.

We also explored possible moderators of policing legitimacy including intervention type, research design, respondent type, crime type, year of publication, and country of publication, using analogs to the ANOVA implemented via subgroup analyses in CMA. In addition, we conducted a series of sensitivity analyses to test the robustness of the results to the following:

9 The Campbell Collaboration | www.campbellcollaboration.org

inclusion of studies where data was imputed, inclusion of poor quality studies (e.g. lack of treatment integrity), and we inspected possible sources of bias in the data, including publication bias and small-study effects.

RESULTS

There were 41 independent evaluations available for meta-analysis: 7 assessed legitimacy as an outcome, 14 assessed procedural justice, 8 assessed compliance/cooperation, 29 assessed satisfaction/confidence, and 26 assessed reoffending. The direct outcome satisfaction/confidence showed the highest overall effect that was statistically significant (OR 1.75, 95% confidence limits 1.54, 1.99), followed by compliance/cooperation (OR 1.62, 95% confidence limits 1.13, 2.32), and procedural justice (OR 1.47, 95% confidence limits 1.16, 1.86). The estimated effect size for the direct outcome legitimacy (OR 1.58, 95% confidence limits 0.85, 2.95), while quite large, has a wide confidence interval, indicating a high degree of uncertainty around the estimate. Interventions showed a marginal effect on reoffending as an indirect outcome measure (g = 0.07, 95% confidence limits 0.14, 0.00). When reoffending was broken down by measurement method, studies that measured reoffending using official police data and self-reported reoffending showed no effect (g = 0.03, 95% confidence limits 0.05, 0.11); however, studies that measured self-reported victimization showed a large decrease in revictimization as a result of the interventions (g = 0.13, 95% confidence limits 0.23, 0.05).

AUTHORS CONCLUSIONS

The main finding of this review is that the effects of legitimacy policing interventions on each direct outcome measure are in a positive direction. For all but the legitimacy outcome, the results were statistically significant. We note that there is a clear lack of randomized experiments in the international research literature that specifically seek to isolate and test the component parts of a legitimacy policing intervention. Notwithstanding the variability in the mode in which legitimacy policing is delivered (i.e., the study intervention) and the complexities around measurement of legitimacy outcomes, our review shows that the dialogue component of front-line police-led interventions is important for promoting citizen satisfaction, confidence, compliance and cooperation with the police, and for enhancing perceptions of procedural justice. In practical terms, this means that police can achieve positive changes in citizen attitudes to police through adopting procedurally justice dialogue as a component part of any type of police intervention. We conclude that the type of police intervention (the vehicle for delivering a procedurally just encounter) is secondary to the procedurally just dialogue that underpins the intervention.

10 The Campbell Collaboration | www.campbellcollaboration.org

Plain Language Summary

Police require voluntary cooperation from the general public to be effective in controlling crime and maintaining order. Research shows that citizens are more likely to cooperate with the police and obey the law when they view the polices authority as legitimate. One way that the police can increase their legitimacy and gain cooperation and respect from citizens is by using procedurally just dialogue that adopts language that treats citizens with dignity and respect, conveys trustworthy motives, allows citizens to speak up and express their views during encounters, and by not profiling people based on race, gender or any other characteristic. The objective of our review was to systematically assess the direct and indirect benefits of interventions led by the public police that contained elements of this type of procedurally just dialogue. The systematic search found 163 studies that reported on police-led interventions, and a final set of 30 studies contained data suitable for meta-analysis. The direct outcomes analyzed were legitimacy, procedural justice, and citizen cooperation/compliance and satisfaction/confidence in the police. In addition, an indirect outcome, reoffending, was also analyzed. The main finding of this review is that police interventions that comprised dialogue with a procedural justice component (or stated specifically that the intervention sought to increase legitimacy) did indeed enhance citizens views on the legitimacy of the police, with all direct outcomes apart from legitimacy itself being statistically significant. Our review shows that by police adopting procedurally just dialogue, they can use a variety of interventions to enhance legitimacy, reduce reoffending, and promote citizen satisfaction, confidence, compliance and cooperation with the police.

11 The Campbell Collaboration | www.campbellcollaboration.org

1 Background for the Review

Police require voluntary cooperation from the general public to be effective in controlling crime. They need citizens to comply with their directives and they need people to demonstrate a tacit willingness to obey the law in general. This understanding of the relationship between perceptions of legitimacy and how these perceptions are shaped by police working productively (or not) with citizens has a long history. Indeed, clashes between police and citizens during the 1960san era of civil demonstrations and unrestled many scholars to observe that people obey the law and cooperate with legal authorities primarily if and when they view those legal authorities as legitimate (Bayley & Mendelsohn, 1968; Bellman, 1935; Decker, 1981; Parratt, 1938; Reiss, 1971; Walker, Richardson, Williams, Denyer, & McGaughey, 1972; Winfree & Griffiths, 1971; see also Tyler, 2006). Contemporary research continues to show that citizens are more likely to comply with police directives when they view the police as legitimate (Tyler, 1990, 1997; see also Tyler, 1988, 1994, 1998, 2001, 2003, 2004; Tyler & Fagan, 2008; Tyler & Huo, 2002).

Legitimacy is defined by Tyler (2006, p. 375) as a psychological property of an authority, institution, or social arrangement that leads those connected to it to believe that it is appropriate, proper, and just. The key defining feature of a legitimate authority is that people feel obliged to voluntarily comply with that authoritys directives. This voluntary compliance is distinct from compliance out of fear of punishment or expectations of reward (Tyler, 2006). In policing, legitimacy reflects a social value orientation toward authority and institutions (Hinds & Murphy, 2007, p. 27) and is central to our understanding of policing, civil order and the derivation of compliance, cooperation and obedience (Tyler, 2004). Evidence shows that it is a persons belief in the legitimacy of the authority or institution issuing a command that leads people to feel that the authority or institution is entitled to be deferred to and obeyed (Sunshine & Tyler, 2003, p. 514).

A number of empirical studies find that perceptions of police legitimacy and procedural justice are related to compliance with police during policecitizen encounters and cooperation with police more broadly. For example Mastrofski, Snipes, and Supina (1996, p.269) examined policecitizen encounters and found that compliance with police was related to the legitimacy of the police intervention. McCluskey, Mastrofski and Parks (1999) found similar results in their study of policecitizen encounters, highlighting the importance of legitimacy to compliance. Sunshine and Tyler (2003) and Tyler and Fagan (2008) also demonstrate that legitimacy is not only related to compliance but to the

12 The Campbell Collaboration | www.campbellcollaboration.org

willingness of citizens to report crime to police and to work with other community members to control and prevent crime. In this way, perceptions of legitimacy also influence the likelihood that citizens will engage in informal and formal crime prevention activities such as working with others in the community to address problems and reporting crime or suspicious activities to the police (Sunshine & Tyler, 2003, p.541; see also Murphy, Hinds, & Fleming, 2008; Tyler, 2004).

We also know that citizen perceptions of police legitimacy encourage law-abiding behavior not just during an actual or potential policecitizen encounter, but also outside of encounters, during everyday life (Sunshine & Tyler, 2003; Tyler & Huo, 2002). As Tyler (2004, p.85) suggests, unless the police are widely obeyed by the public, the capacity of police to maintain order is compromised (see also Tyler, 1990). Research shows that when people perceive the police as legitimate they are more likely to report higher levels of satisfaction and confidence in the police (both for individual officers and the institution), perceive the police as effective in their crime control efforts, be more willing to assist police, as well as be more likely to accept the manifest outcomes of an interaction with police (Tyler, 2004). Police legitimacy thus engenders compliance, fosters cooperation, improves citizen satisfaction with police and thus facilitates the capacity of police to maintain order and control crime.

Community attitudes and perceptions of legitimacy are also affected through the way police engage with third parties (Mazerolle & Ransley, 2005), such as local business owners, school principals and parents. Berrien and Winship (2002), for example, argue that the involvement of third partiesin this case a group of church ministers that were part of the Ten Point Coalition Collaborationfostered the legitimization of police activities within an inner-city Boston community. By providing an umbrella of legitimacy for police efforts to prevent and control crime, Berrien and Winship (2002, p.203) assert that the involvement of the Ten Point Coalition in Boston contributed to a reduction in youth violence. Thus, by improving or increasing perceptions of police legitimacy, police may increase citizen compliance with the law (i.e., in the absence of police directives), in addition to encouraging cooperation and compliance with police.

Police departments throughout the world are increasingly interested in implementing operational programs that seek to both implicitly and explicitly increase police legitimacy. Our review of the extant literature (see below) suggests that there are five different pathways that the police use to increase citizen perceptions of legitimacy. These include police using procedurally just approaches during encounters with citizens, seeking to improve their performance and communicating these improvements to the public, engaging in distributive justice, relying on the law itself, and/or by drawing on the strength of their traditional reputation. Our logic model below (see Figure 1) identifies these five pathways or processes that the literature suggests are important for fostering perceptions of police legitimacy.

13 The Campbell Collaboration | www.campbellcollaboration.org

1.1 LOGIC MODEL

A summary of the theories relating to the causes and consequences of police legitimacy is depicted in Figure 1, which shows that there are five pathways to police legitimacy. We included all five pathways in our systematic search of the extant literature (see Bennett, Denning, Mazerolle, & Stocks, 2009). However, we limited our systematic review and meta-analysis to the procedural justice causal pathway represented by the solid arrows in Figure 1 because interventions deriving from the other four casual pathways did not have comparable direct outcomes.

To ensure that there was some intent of the intervention to enhance citizen perceptions of legitimacy, we focused our review on police interventions that either explicitly stated that the intervention sought to increase legitimacy OR that the dialogue in the intervention used at least one of the principles of procedural justice. The importance of procedurally just dialogue during frontline policecitizen encounters is highlighted most recently by Bottoms and Tankebe (2012) in their argument that it is the dialogic character in policing that cultivates perceptions of legitimacy. For Bottoms and Tankebe (2012), Max Webers original discussion of legitimacy provides foundation for arguing that legitimacy is fundamentally dialogic. They argue that the consequences of ongoing claims to legitimacy from the power holders (i.e., front line police) and iterative responses from citizens means that legitimacy needs to be perceived as always dialogic and relational in character (Bottoms & Tankebe, 2012, p. 129).

We included both associated direct and indirect outcomes of legitimacy in our review. We did not, however, include perceptions of police effectiveness as a measured outcome. The literature shows that police performance is a predictor of police legitimacy and cooperation with police; however police effectiveness (or performance) is also the logical outcome of improved cooperation and compliance with police (Tyler & Fagan, 2008). As such, we sought to provide a review that included a set of like interventions generating like outcomes that are well understood in the literature.

14 The Campbell Collaboration | www.campbellcollaboration.org

Figure 1. Theoretical model of police legitimacy process

1.2 PROCEDURAL JUSTICE

The research literature is replete with studies that argue that the primary pathway to promoting legitimacy is through the use of procedural justice (Tyler, 2001). It is this procedural approach (referred to as procedural justice) that scholars identify as the most important pathway to police legitimacy (Hinds & Murphy, 2007; Tyler, 2001, 2004). Tyler (2004, p. 91) argues that the legitimacy of authorities and institutions is rooted in public views about the appropriateness of the manner in which the police exercise their authority. The procedural justice model describes the way in which the police can exercise this authority in a fair and just way through both the quality of treatment and the quality of the decision making process (Reisig, Bratton, & Gertz, 2007, p. 1006). In recent research procedural justice is thus operationalized as the way in which police treat citizens and the fairness of the decisions made (Reisig et al., 2007; Sunshine & Tyler, 2003).

Thibaut and Walker (1975) first used the term procedural justice to refer to ones perception of treatment during decision-making processes. In the field of policing, renewed academic interest in procedural justice emerged in the late 1980s and early 1990s when police agencies throughout the world were implementing community policing initiatives while incidents of police corruption and police misconduct (e.g., racial profiling, excessive force) pervaded the public conscience (Kelling & Moore, 1988; Reiner, 1985, 2000). The procedural justice perspective also came at a time when policy makers expressed concerns about police inadequacies in dealing with upsurges in crime (Maher & Dixon, 1999;

15 The Campbell Collaboration | www.campbellcollaboration.org

Weisburd & Braga, 2006), leading to a general loss of confidence in traditional police responses to crime (Weisburd & Eck, 2004). These concerns created fertile ground for the study of police legitimacy, and the concomitant study of policecitizen encounters based on fair and respectful processes and procedures.

Since this time, scholars across many different contexts and disciplines (e.g., taxation compliance and organizational behavior) have studied the impact of treatment and decision making during interactions with authorities. Procedural justice, as described in the literature, typically comprises four essential components: citizen participation in the proceedings prior to an authority reaching a decision (or citizen voice), perceived neutrality of the authority in his/her decision, whether or not the authority showed dignity and respect throughout the interaction, and whether or not the authority conveyed trustworthy motives (Goodman-Delahunty, 2010; Tyler, 2008; Tyler & Huo, 2002; Tyler & Murphy, 2011). These four core factors shape police encounters with citizens and subsequently influence perceptions of police legitimacy.

Research finds policecitizen encounters involving the use of these principles of procedural justice enhance the quality of policecitizen interactions, leading citizens to be more satisfied with the interaction and outcome (Mastrofski et al., 1996; McCluskey, 2003; Reiss, 1971; Tyler & Fagan, 2008; Wells, 2007). People who feel they have been dealt with in a procedurally fair way are less likely to believe that they have been personally singled out (e.g., racially profiled) and are more likely to accept the decisions (e.g., fine or sentence) made by authorities (Tyler & Wakslak, 2004).

These results demonstrate a direct link between procedurally just encounters and citizen perceptions of the police specific to the encounter. Yet whether or not positive encounters with police can influence more generalized beliefs about procedural justice and legitimacy of the police is not as well understood in the extant literature. In a general sense, we do know that if the police are evaluated as exercising their authority fairly, they are viewed as more legitimate (see Elliott, Thomas, & Ogloff, 2011; Fischer et al., 2008; Murphy, Hinds, & Fleming, 2008; Reisig, Bratton, & Gertz, 2007). When authorities are not viewed as procedurally just, their legitimacy is undermined, leading to disobedience and resistance (Fischer et al., 2008). Sunshine and Tyler (2003) explore the influence of more general evaluations of the procedural justice of the police upon peoples judgments about the legitimacy of the police, finding that global views of procedural justice are a key antecedent of legitimacy. These judgments were not linked to specific policecitizen encounters, but considered more general perceptions of police.

Research finds that policecitizen contacts involving the use of procedural justice, by enhancing the quality of policecitizen interactions, have a direct effect, leading citizens to be more satisfied with the interaction (Mastrofski, Snipes, & Supina, 1996; McCluskey, 2003; Reiss, 1971; Tyler & Fagan, 2008; Wells, 2007). Tyler and Wakslak (2004) also identify a number of other positive outcomes from procedurally just encounters; for example, people

16 The Campbell Collaboration | www.campbellcollaboration.org

who had recent, contact with courts or police were less likely to believe they were racially profiled, and more likely to accept the decision made by authorities, if they were treated in a procedurally just manner (Tyler & Wakslak, 2004). Similarly, research studies in Australia (Hinds & Murphy, 2007) and the United Kingdom (Jackson & Sunshine, 2007) show that individual perceptions of procedural justice are associated with perceptions of police legitimacy and satisfaction with police. Research also identifies the potential wider community benefits of procedurally just encounters. Murphy and colleagues (2008), for example, examined the impact of a community policing intervention on perceptions of police legitimacy and cooperation with police, finding that procedurally just encounters influenced changes in general perceptions of police legitimacy over time, and that perceptions of police legitimacy subsequently influenced cooperation with police.

Describing procedural justice as a holistic antecedent of legitimacy, however, oversimplifies what is, in fact, a complex process. Hawdon (2008), for example, offers an emerging perspective that highlights this complexity by hypothesizing that the relationship between procedural justice and legitimacy is not linear, but rather circular and reciprocal. That is, according to Hawdon (2008), perceptions of institutional legitimacy lead to a belief about procedural justice, which then in turn shapes legitimacy, and so on. While this supposition has not been tested, other authors have also found evidence for this self-perpetuating cycle (e.g., Brandl, Frank, Worden, & Bynum, 1999; Reisig & Chandek, 2001; Tyler, 2004). Despite its theoretical complexity, a number of studies explore and show the importance of the relationship between procedural justice and legitimacy (Hinds & Murphy, 2007; Jackson & Sunshine, 2007; Paternoster, Brame, Bachman, & Sherman, 1997; Sunshine & Tyler, 2003; Tyler, 1990).

Overall, the power of improving the legitimacy of the police lies in what perceptions of legitimacy can offer the police (both individually and institutionally) and the community. Of benefit to the police is increased compliance: police are more able to encourage the willingness to comply with both police directives and the law when they are perceived to be a legitimate authority (Sunshine & Tyler, 2003). Noncompliance, on the other hand, poses a danger for both the police and citizens, as it may precede violence towards police officers, increasing the risk of harm to both the police and citizens at the encounter (Reiss, 1971). As Tyler and Huo (2002) and Sunshine and Tyler (2003, p. 519) suggest: the public are more likely to allow intrusive police tactics when the police are perceived to be legitimate, thus allowing police more operational flexibility in their efforts to control crime.

1.3 POLICE PERFORMANCE AND LEGITIMACY

Citizen perceptions of police performance is another pathway that scholars often cite as being important for influencing citizen perceptions of legitimacy. When the public either sees evidence of the police performing well or believes that they perform their job well they have been found to view the polices authority as more legitimate (Hinds & Murphy, 2007; Tyler, 2006). There is some evidence that this relationship between police performance and

17 The Campbell Collaboration | www.campbellcollaboration.org

police legitimacy may vary across cultural contexts. For example, Hinds and Murphy (2007) examined perceptions of procedural justice, police effectiveness, police legitimacy and satisfaction with police among 2,611 Australians. Hinds and Murphy (2007) found that while procedural justice was a stronger predictor of legitimacy than police performance in the Australian context, the procedural justice effect was weaker than that found by Tyler and his colleagues in the US.

Tankebe (2009) examined police legitimacy research in the context of a developing nation: Ghana. In a survey of 450 households in Ghana, Tankebe (2009) found that police effectiveness was the only policing variable associated with the willingness to cooperate with police (i.e., procedural justice and trust in police were not related to cooperation in his final statistical models). Tankebe (2009, p. 1281) suggests that in countries like Ghana that experience high rates of crime and police misconduct, issues of police effectiveness and public security may be crucial. Moreover, he concludes that in contexts where consent and cooperation are often elicited by force, procedural justice concerns may be less important for police legitimacy and cooperation (Tankebe, 2009). Tankebes (2009) research demonstrates that procedural justice might not be the fundamental modus operandi for all police in all cultural contextspolice performance or effectiveness may also be important (see also Murphy & Cherney, 2012).

Jonathon-Zamir and Weisburd (2009) examined the relationship between police effectiveness and legitimacy and level of perceived threat in Israel. They found that while performance of the police in their ability to combat crime does play a significant role in police legitimacyand increasingly so in times of threatprocedural justice remains the prime antecedent. However, they conclude that there does not seem to be a zero-sum game at play between performance and procedural justice. In situations of security threats, there appears to be a growing desire for forceful action and end results, but not at the expense of high standards of procedural fairness (Jonathon-Zamir & Weisburd, 2009, p. 27).

Similarly, research by Tyler and his colleagues also suggests procedural justice is more important to gaining public trust in police and cooperation with police than police performance or effectiveness. Tyler (2004, p. 86) argues that the way citizens view and respond to the police is only loosely linked to police effectiveness in fighting crime (see also Sunshine & Tyler, 2003). Moreover, Tyler and his colleagues generally find police legitimacy is the key antecedent of cooperation and compliance (e.g., Sunshine & Tyler, 2003). For example, in their study of 483 New Yorkers, Sunshine and Tyler (2003) found that police legitimacy, rather than police performance, was associated with compliance and that while police performance was linked with cooperation, police legitimacy was a stronger predictor. Similarly, they found that while performance and procedural justice predicted legitimacy, procedural justice was more strongly associated with police legitimacy (Sunshine & Tyler, 2003; see also Tyler & Fagan, 2008).

18 The Campbell Collaboration | www.campbellcollaboration.org

According to Tyler and Fagan (2008), legitimacy, cooperation and citizens obligation to obey the police will improve police effectiveness. In comparing the effects of procedural justice and police effectiveness or performance on police legitimacy, cooperation and compliance, the logical outcome is often cited as a reduction in the crime rate (i.e., an improvement in police effectiveness). For example Tyler and Fagan (2008, p. 223) state:

To be effective in lowering crime and creating secure communities, the police must be able to elicit cooperation from community residents such cooperation potentially involves, on the part of the public, both obeying the law and working with the police or others in the community to help combat crime in the community.

Overall, we acknowledge that there may be some reciprocal component to the police effectivenesslegitimacy relationship and that police performance is undoubtedly important to police legitimacy. Nonetheless, given that research generally finds that procedural justice is a better predictor of police legitimacy than police performance (e.g., Sunshine & Tyler, 2003; Tyler & Fagan, 2008), our review was better placed to focus on the procedural justice to legitimacy pathway rather than trying to capture the high performancelegitimacy pathway, unless of course the intervention was explicitly intending to enhance legitimacy (see inclusion and exclusion criteria below).

1.4 DISTRIBUTIVE JUSTICE

Distributive justice is another antecedent of police legitimacy and refers to perceptions regarding the fairness of the distribution of police services and activities between different communities, groups and individuals (Jonathon-Zamir & Weisburd, 2009, p. 7). An extensive body of research demonstrates that ethnicity, age, and economic status (amongst other variables) are key factors in determining individual perceptions of police fairness and legitimacy. Racial discrimination in policing, for example, is explored extensively in the literature. Researchers in the US (see Sunshine & Tyler, 2003; Tyler & Huo, 2002), in the UK (see Bowling & Philips, 2002; Bradford & Jackson, 2008) and in Australia (see Bird, 1992; Pickering, McCulloch, & Wright-Neville, 2008; Sivasubramaniam & Goodman-Delahunty, 2008) find that perceptions of distributive justice can and do influence perceptions of legitimacy and that treating people or groups on a discriminatory basis undermines police legitimacy.

Research attention also focuses on the distributive justice of policing young people. Given that young people often experience high contact levels with the police and that youthpolice relations are frequently strained, research finds that the extra policing attention afforded to youth (such as moving on groups of youth who have not done anything wrong) leads young people to take a view that the police are exercising their authority unfairly (Fagan & Tyler, 2005; Hurst & Frank, 2000; Leiber, Nalla, & Farnworth, 1998; Murphy & Gaylor, 2010; Piquero, Fagan, Mulvey, Steinberg, & Odgers, 2005).

19 The Campbell Collaboration | www.campbellcollaboration.org

In terms of economic status, Jackson and Bradford (2009) note that opinions about distributive justice are very different between those who have long histories of difficult relations with the police and those who do not. They argue that just as the police represent for many [people] order, stability and cohesion, to people from [low socio-economic] groups they represent the unfair priorities of the dominant social order, an interfering state, or even oppression (Jackson & Bradford, 2009, pp. 67).

Many studies find that different cultural and ethnic groups have the same perceptions of both the antecedents and consequences of procedural justice (Bradford & Jackson, 2010; MacCoun, 2005; Murphy & Cherney, 2011; Tankebe, 2009; Tyler & Huo, 2002). This makes it somewhat difficult to tease out distributive justice from procedural justice (e.g., see Tyler, 1994). Moreover, recent research argues that distributive justice is a less important antecedent than procedural justice. Murphy and Gaylor (2010), for example, found that while instrumental factors such as police performance, distributive justice and youth/police relations were important the overwhelming factor that predicted views about police legitimacy was procedural justice (p. 16).

1.5 LEGAL LEGITIMACY

Despite the seemingly obvious relationship between the legitimacy of the police and the legitimacy of the laws they enforce, researchers have paid much less attention to the issue of legal legitimacy as an antecedent of police legitimacy than they have to the issue of procedural justice. In fact, Murphy, Tyler and Curtis (2009) and Murphy and Cherney (2012) point out that while a significant body of past research into police legitimacy focuses on the perceived legitimacy of those tasked with enforcing the law, a key gap exists into the role played by the perceptions of the legitimacy of the legal system and the laws and rules being enforced. Nevertheless researchers find that in order to garner legitimacy, not only do the police need to be fair and procedurally just in the performance of their duties, they also need to be seen to be enforcing or supporting rules, policies and laws that are perceived to be legitimate. In other words, legal legitimacy is an important antecedent to cooperation and compliance with the police (e.g., Jackson, Bradford, Hough, & Murray, 2011). At the same time, some scholars suggest that how police conduct their duties can affect perceptions of the rules and laws being enforced, with procedural justice in a personal encounter with the police found to influence not only views of police legitimacy, but legal legitimacy as well (Barnes, 1999; Tyler, 1990; Tyler & Huo, 2002).

Significantly, researchers point out, public perceptions of legal legitimacy moderate the effects of procedural justice. In fact, Murphy and Cherney (2012) find that those individuals that question the legitimacy of the laws or rules being enforced have a stronger positive response to procedural justice than those who do not question legal legitimacy. However, in the case of those who do not support the law and have disengaged entirely, procedural justice is likely to compound noncompliance (Murphy & Cherney, 2012). Further, while an authority (e.g., the police) may be seen to be legitimate, the policies, rules and laws that the

20 The Campbell Collaboration | www.campbellcollaboration.org

authority enforces may be seen to be illegitimate (Murphy, Tyler, & Curtis, 2009). Indeed, a study funded by the National Institute of Justice in the US found that whereas only 27% of Americans expressed confidence in the criminal justice system, the policeas a component of the criminal justice systemwere rated more than twice as highly at 59% (Tyler, 2004). Overall, research suggests that the legitimacy of police performance, procedure and the perceived legitimacy of the system within which the police operate and the laws they enforce need to be understood as separate elements of police legitimacy.

1.6 TRADITIONAL, HISTORICAL AND CULTURAL LEGITIMACY STEMMING FROM TRADITION

The last antecedent to legitimacy articulated in the literature is loosely and variously referred to as the role police play as traditional and symbolic representatives of social order and cohesion. Tradition is widely understood to be anything that is transmitted or handed down from the past to the present. It carries with it historical elements and cultural beliefs and practices. Traditional responses and held beliefs relating to police legitimacy may, for example, characterize community held beliefs in the legitimacy or illegitimacy of a police service over many years, or many generations. These beliefs (and their associated actions and behaviors) are not necessarily based in fact, but are handed down from one generation to the next as a set of expectations and responses. Moreover, people are a part of communities and are influenced by the attitudes of those with whom they interact on a daily basis. Over time people may develop and pass on shared views of the legitimacy of laws or those that administer them. They may also belong to historically socially excluded groups who have been subject to police discrimination over a period of many years or even generations, and come to share a tradition of noncompliance or compliance with authority. These belief systems are embedded at a community level and are highly influential in evaluations as to how well the police perform their duties on the basis of legal, distributive and procedural justice.

Reiner (2000) explains that perceptions of police legitimacy derive from traditionally held views because the sources of order lie outside the ambit of the police, in the political economy and culture of a society Subtle, informal social controls, and policing processes embedded in other institutions, regulate most potential deviance (Reiner, 2000, p. xi). Overall, Reiner (2000) argues that when informal control processes are successful, the police will appear highly effective in crime prevention and deal effectively and legitimately with the crime and disorder that does occur.

1.7 TYPES OF POLICE INTERVENTIONS TO INCREASE LEGITIMACY

The dialogic nature of legitimacy-enhancing policing (see Bottoms & Tankebe, 2012) suggests that the mechanism or vehicle in which the engagement between police and citizens

21 The Campbell Collaboration | www.campbellcollaboration.org

occurs is less important than the manner in which the intervention is delivered. As such, we understood that a broad set of police interventions could potentially increase citizen perceptions of police legitimacy so long as the interventions had common dialogue. The four, universally clear components of procedural justice (neutrality, dignity and respect, trustworthy motives and participation in decision making) provided a foundation to include interventions that had like elements. As such, we sought to locate and review police interventions that either explicitly articulated that the intervention sought to increase citizen perceptions of legitimacy OR included at least one element of procedural justice. The range of interventions that we expected to encompass in our review included community policing initiatives such as Neighborhood Watch, beat policing, reassurance policing, and contact patrols, all of which provide a range of opportunities for police and citizens to engage in positive ways. We also expected that many problem-oriented policing strategies, crime prevention through environmental design programs, and risk-focused policing initiatives could contain elements of procedural justice and provide opportunities for police to enhance citizen perceptions of police. The legitimacy literature also focuses on restorative justice conferencing interventions as a key vehicle for enhancing police legitimacy. Likewise, inter-agency initiatives that include collaboration between police and social service agencies to respond to domestic violence, collaborations between police and schools to reduce truancy, and other, broader multi-agency strategies are often cited as police efforts to enhance police legitimacy.

Special police training programs such as life skills training, diversity training, crisis intervention training, victim-focused training, and community policing training often include explicit training in procedural justice as a means to enhance legitimacy. Likewise, organizational innovations, such as the creation of smaller geographically-based command units, within which officers reported to their command unit representative, are sometimes argued as an explicit means for frontline officers to enhance citizen perceptions of police legitimacy. We also expected that some of the school-based interventions, including police officers located within schools to foster ties to students, would create opportunities for students to interact with police in an informal setting and thus be likely to increase perceptions of legitimacy.

Overall, whilst our review has strict inclusion criteria around the legitimacy-enhancing nature of the police intervention (see below), our search cast a broad net on a wide range of different interventions that might be expected to achieve this result. Thus, for our review, the mode of service delivery was very much secondary to our primary concern of gathering studies that contained legitimacy enhancing dialogue (see Bottoms & Tankebe, 2012)

1.8 PURPOSE AND CONTRIBUTION

The proliferation and growing importance of research in the area of legitimacy, procedural justice, and order maintenance over the last 20 years demands careful review of empirical evidence to help police and policy makers understand the types of police interventions that

22 The Campbell Collaboration | www.campbellcollaboration.org

might directly or indirectly enhance police legitimacy. At present, the literature is devoid of quantitative reviews or meta-analyses summarizing the effects of police legitimacy interventions. We therefore proposed to review and provide a quantitative synthesis focusing on legitimacy interventions that are police-led and specifically focus on improving police legitimacy.

Our systematic review investigates what is known about police interventions designed to facilitate legitimacy in policing at the micro level (i.e., during policeindividual encounters such as with offenders, victims, witnesses and other citizens) and at the macro level (i.e., during police encounters with groups and communities such as community events, at schools, and in business communities). Results from this systematic review provide evidence for policy makers and policing agencies to:

1) implement modes of police delivery that advance citizen perceptions of legitimacy;

2) provide police with evidence-based models to assist them in performing their duties;

3) improve citizen compliance and enhance the publics perceptions with respect to the police.

23 The Campbell Collaboration | www.campbellcollaboration.org

2 Objectives of the Review

This review synthesizes the existing published and unpublished empirical evidence on the impact of police efforts that seek to enhance citizen perceptions of police legitimacy. Studies that evaluate legitimacy interventions are included if they were led by public police from any level of government (i.e., local, state and federal law enforcement officers). We have carefully chosen the word led by public police to eliminate those interventions that are led by the courts, other criminal justice agencies or regulatory entities within non-police agencies (e.g., taxation departments) and which involve police, but are not organized, led or administered by publicly-funded police officers.

We provide a systematic review of the direct and indirect benefits of approaches that foster legitimacy in policing through either an explicit statement that the intervention sought to increase legitimacy, or the application of at least one of the principles of procedural justice: participation, neutrality, dignity/respect, trustworthy motives.

Our review does not include police interventions that stem from one of the other four pathways that lead to legitimacy as defined in Figure 1, unless the intervention explicitly stated that it sought to increase citizen perceptions of police legitimacy. That is, we do not include evaluations of generic police interventions that report effectiveness and efficiency outcomes unless the intervention explicitly included a statement that the intervention sought to increase legitimacy or included an element of procedural justice. Similarly, we do not include studies that refer to the legal or traditional role of police, unless the intervention explicitly included a statement that the intervention sought to increase legitimacy or included an element of procedural justice. The key to our review was that the intervention needed to include an element of procedural justice or a clear statement that the intervention sought to increase legitimacy. Generic policing interventions, therefore, could not be included unless the study articulated that the intervention did something specific to enhance legitimacy. Hotspots policing is a case in point: Bragas (2007) systematic review of hotspots policing shows that it is an effective approach to controlling crime and disorder problems. Yet, during most hotspots policing interventions the police rarely get out of their cars. Without a clear articulation that an intervention like hotspots policing was doing something specific to engage with citizens to enhance legitimacy, we could not be sure that the intervention itself could directly influence citizen perceptions of legitimacy. This is especially important given the dialogic importance of legitimacy (see Bottoms & Tankebe, 2012). Therefore, after an extensive search of the literature (see below, Section 3.2.1 and Table 1),

24 The Campbell Collaboration | www.campbellcollaboration.org

we opted to focus our review on police interventions that included an explicit statement saying that the intervention sought to increase legitimacy, or described at least one element of procedural justice. As such, to be eligible for inclusion in our review, our criteria dictated that we needed to see at least a statement that the intervention sought to increase legitimacy or at least one of the principles of procedural justice articulated as a component part of the intervention.

Our review thus includes any type of public police intervention (e.g., routine patrol, traffic stops, hotspots policing, problem-oriented policing, conferencing) that involved contact with citizens where there was a clear statement that the intervention involved some training, directive or organizational innovation that sought to enhance legitimacy or used at least one core ingredient of procedural justice: police encouraging citizen participation, remaining neutral in their decision making, demonstrating trustworthy motives, or showing dignity and respect throughout interactions.

The studies included in our review also had to include at least one direct outcome consistent with improving legitimacy. Based on the academic research literature, we sought to better understand the direct benefits of fostering legitimacy in policing. These direct benefits include increased perceptions of compliance, cooperation and citizen satisfaction with police.

We also sought to explore the indirect benefits of legitimacy policing, which include measures of reoffending (or revictimization), and crime and disorder. We sought to examine micro-level interventions between police and individual citizens that aim to foster legitimacy as well as those macro-level interventions between police and communities that also aim to foster legitimacy in policing more generally.

Specifically, this review provides:

A summary of empirical evidence of police interventions aimed at encouraging legitimacy in policing;

An inventory of interventions identified in the literature that have, to date, been used for the purpose of promoting legitimacy in policing, either explicitly by stating that the intervention aimed to improve police legitimacy, or implicitly by using interventions with the key ingredients of procedural justice;

A summary of mean effect sizes (i.e., Hedges g, odds ratio) for the interventions;

A summary of the direct (e.g., perceived compliance, cooperation, satisfaction) outcomes of interventions that seek to enhance legitimacy in policing, as well as indirect (e.g., reoffending, crime and disorder) outcomes where applicable;

Moderating variables that may influence the effectiveness of interventions designed to improve police legitimacy either explicitly or implicitly through the employment of at least one procedural justice ingredient and a direct legitimacy outcome. These moderating variables included the particular form of the

25 The Campbell Collaboration | www.campbellcollaboration.org

legitimacy intervention (for example, community policing or restorative justice conferencing); the population under study (specifically, offenders or the general public), the type of crime targeted (property crime or violent crime), and the evaluation design (experimental or quasi-experimental). We also looked at the effect of date of publication, to see whether intervention effects had changed over time, and author group, to see whether particular influential researchers achieved different results to smaller, less influential researchers.

26 The Campbell Collaboration | www.campbellcollaboration.org

3 Methods

This review synthesizes existing published and unpublished empirical evidence to assess the effects of police-led interventions designed to improve legitimacy in policing (or included an intervention that either explicitly or implicitly used at least one procedural justice ingredient) on a direct legitimacy outcome. The stages of this review and the criteria used to select eligible studies are described below.

3.1 CRITERIA FOR INCLUSION AND EXCLUSION OF STUDIES IN THE REVIEW

While there is a considerable amount of research into legitimacy in policing and procedurally just approaches in policing, the overwhelming majority of studies do not use experimental designs. This was highlighted recently by Weisburd, Mastrofski, and Telep (2009, p. 1), who stated:

[e]xisting studies provide important insights into our understanding of legitimacy and procedural justice in policing, but there has not been an experimental field study of testing key propositions set forth by proponents of legitimacy policing.

The legitimacy literature is typically general in nature and is not restricted to particular participants (e.g., young people, prolific offenders, or minority groups) or discrete crimes (e.g., drug offences) or with community problems in mind (e.g., communities with a history of policecitizen hostility). As such, police-led legitimacy interventions were included that focused on specific types of individuals (e.g., young people or drug dealers or community members) or on a collection of different groups of people (or both).

Sections 3.1.1 to 3.1.6 provide detail on the period of time covered by the review, as well as the types of studies, types of statistical data, participants, interventions and outcomes. A flow diagram reflecting the inclusion/exclusion criteria is provided and incorporated in Appendix 4: Coding Sheet for Legitimacy in Policing Review.

3.1.1 Period of time to be covered by systematic review

A preliminary exploration of published and unpublished literature focusing on authors who have given significant consideration to the design of procedural justice and legitimacy (e.g.,

27 The Campbell Collaboration | www.campbellcollaboration.org

Tyler, Murphy, Hinds, Skogan, and Mastrofski) was conducted to determine the period of time that should be covered in this review. Procedural justice and legitimacy as criminal justice concepts have developed significantly from 1990 when Tylers influential book Why People Obey the Law was first published; however, our preliminary examination revealed some seminal pieces published in the 1980s that would be relevant to the review (e.g., Tyler & Linds (1986) Procedural processes and legal institutions, Roehls (1988) Measuring perceptions of procedural justice). Consequently, the research team decided to include literature from 1980 in order to be inclusive of salient material on procedural justice and legitimacy in policing. As described below, we drew from a comprehensive database created from a systematic search of legitimacy policing conducted on behalf of the UK National Policing Improvement Agency (NPIA) (Bennet et al., 2009).

3.1.2 Types of study designs

To be included in the systematic review, studies must have used one of the following research designs:

a. an experimental (randomized) design involving at least two conditions, with one condition representing a police-led legitimacy intervention designed to increase police legitimacy (either explicitly or implicitly through the employment of at least one procedural justice ingredient and a direct legitimacy outcome)see types of intervention defined in section 3.1.5and another representing a control condition. Eligible comparison/control conditions could be a police-led intervention or any other type of criminal justice intervention, but where a directive, training or organizational innovation designed to increase police legitimacy (either explicitly or implicitly through the employment of at least one procedural justice ingredient) is absent;

b. a quasi-experimental (non-randomized) design involving at least two conditions, with one condition representing an intervention designed to increase police legitimacy (either explicitly or implicitly through the employment of at least one procedural justice ingredient and a direct legitimacy outcome) and another representing a comparison condition (eligible comparison conditions were the same as above; these designs may have a pre-test but this is not an essential feature for inclusion);

c. a quasi-experimental interrupted time-series design involving measurement of an aggregate outcome, such as crime rate, in equally spaced time intervals prior to and following the initiation of the police-led intervention. The police-led intervention had to involve a directive, training or organizational innovation designed to increase police legitimacy. The unit of analysis for eligible designs was individuals. Note that the quasi-experimental designs could be surveys that included a variable allowing for the categorization of observations into either an intervention or comparison group.

28 The Campbell Collaboration | www.campbellcollaboration.org

3.1.3 Types of statistical data

To be eligible for inclusion in the analysis, studies must have been reported in such a manner that effect sizes could be identified and/or calculated (see section 3.5 for detail on statistical procedures). Studies for which the effect size could not be calculated were included in the systematic review so that a comprehensive inventory of police-led interventions designed to encourage legitimacy in policing (either explicitly or implicitly through the employment of at least one procedural justice ingredient and a direct legitimacy outcome) could be provided.

3.1.4 Types of participants/units of analysis

This review is interested in the significance of the interaction between police and the public and the impact that such interactions have at either a micro level (i.e., individuals) or macro level (i.e., group, community and/or third parties). Consequently, the review includes studies which focus on the way in which the police interact with:

The individual (citizen, victim, offender etc.), and/or

The group (community, group gathering etc.), and/or

Third parties (religious leaders, community advisors etc.).

As such, we searched for studies collecting data on individuals, groups and/or third parties as the units of analysis eligible for inclusion. In this review and meta-analysis, however, we only include studies that used individuals as the unit of analysis.

3.1.5 Types of police-led legitimacy interventions

Interventions had to involve police interventions that either (1) explicitly aimed at improving police legitimacy (through either a directive, training or organizational innovation) or (2) explicitly used at least one element of procedural justice. Eligible interventions were limited to those that specified, in the intervention description, that there was some type of training, directive or organizational innovation provided to or by the police to encourage, foster or facilitate legitimacy in policing. We also included studies that had a direct outcome listed in section 3.1.6 and specified, in the intervention description, that there was some type of training, directive or organizational innovation provided to or by the police that used at least one of the following procedural justice-based core ingredients of legitimate policing:

Citizen participation

Perceived neutrality of the authority

Dignity and respect

Trustworthy motives

The intervention could involve the police in activities (both pre-arrest and post-arrest) that depicted routine policing, traffic stops, investigations, warrant execution, problem-oriented

29 The Campbell Collaboration | www.campbellcollaboration.org

policing, conferences, school-based programs, crackdowns and other types of police approach where there was a clear training program, directive or organizational innovation to approach contact with the public for the purpose of fostering legitimacy, or where at least one of the elements consistent with procedural justice policing had been used. The context of the intervention was coded (see Appendix 4). We also included police interventions that were aimed at involving police officers in community events (Murphy et al., 2008) and/or with third parties (Berrien & Winship, 2002).

Studies that focused on how other criminal justice or regulatory agencies (e.g., taxation departments, local governments, child safety departments) interact with individuals, groups and third parties were not included. In addition, we did not include those interventions that related to within-police agency management, as these types of studies aimed to increase legitimacy within organizations in order to improve/encourage, for example, job satisfaction for police officers or reduce corruption amongst police officers.

The comparison conditions were those encounters undertaken by police or any other criminal justice institution that did not entail a directive, training and/or organizational innovation that was aimed at encouraging legitimacy in policing and/or did not utilize at least one ingredient of procedural-justice-based legitimacy policing (i.e., citizen participation, perceived neutrality, demonstrating dignity and respect, establishing trustworthy intentions). Studies had to use business as usual comparison conditions (that is, not an alternative intervention) to be eligible for inclusion in the meta-analysis.

3.1.6 Types of outcome measures

Studies were eligible if they measured the effects of interventions aimed at encouraging legitimacy in policing (or used at least one ingredient of procedurally just policing) and reported at least one of the following direct or indirect outcomes:

Direct outcomes1

Included measures of at least one of the following outcomes (the named outcome and/or at least one of the sub-constructs and/or at least one of the items listed under that outcome):

Perceived Legitimacy

1 Constructs and items identified in Tyler (2006) and Tyler, Sherman, Strang, Barnes, & Wood (2007).

30 The Campbell Collaboration | www.campbellcollaboration.org

Obligation to obey police

o Moral obligation to obey police

o Obey the police with good will Police legitimacy

o Respect for the police

o Confidence in police Obligation to obey the law

o Moral obligation to obey the law

o Obeying the law is the right thing to do Procedural Fairness (or perceived procedural fairness)2

Fairness

o Police try to be fair when making decisions

o Police give citizens the opportunity to express views before decisions are made

o Police listen to people before making decisions Neutrality

o Police make decisions based on fact, not personal biases or opinions

o Police treat people as if they can be trusted

o Police treat people as if they will do the right thing even when not forced to Respect

o Police treat people with dignity and respect

o Politeness of police Willingness to cooperate with police (or perceived willingness to cooperate)

Cooperation with police

2 The fourth element of procedural justice identified by Tyler (2004) and Mastrofski (2009): trustworthy motives, was captured by the outcome of trust/confidence in police.

31 The Campbell Collaboration | www.campbellcollaboration.org

o Would call police to report a crime

o Provide information to police

o Report dangerous/suspicious activities

o Willingly assist police if asked Trust/Confidence in police (or perceived trust/confidence in police)

o Trust police

o Confidence in police

o Satisfied with the way police do their job Social Ties

Belief that reoffending will create problems in social relationships

o How much of a problem would it be if you were arrested again

o How much of a problem would it create for your life if your family and friends found out you were arrested

o How much of a problem would it create for your life if the public knew you were arrested (e.g., name and offence printed in the newspaper)

o If caught again, how tough would your punishment be Compliance

o Intention to comply with police in future

o Behavioral compliance Satisfaction

Police effectiveness

o How good a job are they doing

o How satisfied are you with the way they solve problems Fairness of outcomes

o How satisfied are you with the fairness of the outcomes people receive

32 The Campbell Collaboration | www.campbellcollaboration.org

Fairness of procedures

o How satisfied are you with the fairness of the way that people are treated Indirect Outcomes3

Included measures of one or more of the following:

Reduction in reoffending

Reduction in crime

Reduction in social disorder

3.1.7 Language and geographic origin

There were no exclusions on the geographic location of the studies. To be included in this review, however, the study must have been written in English. Whilst we did locate several French and German written studies that could be deemed eligible to be included in this review, we did not have the resources for translation at the time this review was completed. We note, however, that future updates of the review will include translations from other than English written studies.

3.1.8 Publication status

Both published and unpublished studies were eligible for this systematic review.

3.2 SEARCH STRATEGY FOR IDENTIFICATION OF RELEVANT STUDIES

The research team utilized a database of documents identified during the NPIA systematic search of the police legitimacy literature (Bennett et al., 2009). The search strategy included published and unpublished literature that was available from January 1, 1980 to April 1, 2009. Of the 20,600 hits reviewed, 2,526 records were identified and coded as relevant to procedural justice and/or police legitimacy. Researchers recorded search information (date of search, database and search terms used), research information (design, method(s), agency, outcome, population) and reference information in a Microsoft Excel spreadsheet so

3 Constructs and items identified in Sherman et al. (1998) and Shapland et al. (2008).

33 The Campbell Collaboration | www.campbellcollaboration.org

that further examination of the data could be conducted at a future date. The sections below detail the search strategy used for the systematic search conducted on behalf of the NPIA (see Bennett et al., 2009).

3.2.1 Keyword formulation

The NPIA search was conducted on all of the five antecedents of legitimacy: (1) procedural justice, (2) distributive justice, (3) police performance, (4) police, law and legitimacy, and (5) police, legitimacy and tradition (see Bennett et al., 2009). An initial list of terms was formulated and organized into broad concepts or Tiers as follows:

Tier 1: Criminal Justice Agencies to retrieve literature relating to criminal justice organizations (e.g., the police) as opposed to other organizations (e.g., tax office, armed forces).

Tier 2: Justice Approaches in Policing and Associated Terms. Terms in Tier 2 included concepts related to broader legitimacy policing such as distributive justice, procedural fairness and procedural justice. Synonyms were identified for the phrase procedural justice from literature by authors considered foundational to the development of procedural justice and legitimacy policing as concepts in the criminal justice setting.

Tier 3: Outcomes Relevant to Legitimacy Policing. Research suggests that there are measurable outcomes to procedural justice approaches and/or legitimacy policing (e.g., compliance). As with Tier 2 terms, the research team reviewed literature by foundational authors to draw out additional keywords that would assist with retrieving relevant literature.

Tier 4: Evidence Focused Filters. A central objective was to develop a search strategy that would identify quality publications relevant to the research questions. Consequently, research-related terms were included.

The research team conducted a series of pilots on single and combined terms before deciding on a final list of keywords, which are presented in Table 1.

34 The Campbell Collaboration | www.campbellcollaboration.org

Table 1. Keywords for systematic literature search

Tier 1

Criminal Justice Agencies

Study

Studies

Research Empirical Evaluation

Theor* (used in conjunction with legitimacy in Tier 3 keywords only)

Compound terms (e.g., procedural justice, criminal justice) were considered as a single term and entered into searches in quotes (e.g., procedural justice). This strategy ensured that the database searched for the entire term rather than procedural AND justice, which would clearly produce very different results. In addition, search terms with multiple iterations from a base word stem (e.g., fair, fairness, fairly) were typed in as word* (e.g., fair*). This approach enabled the researcher to capture relevant literature with fewer searches, thereby saving time.

The research team determined that the Tiers searched independently generated a vast number of hits. For example, searching on the term police in isolation resulted in 59,869 records using Cambridge Scientific Abstracts (CSA), whilst procedural justice on its own produced 849 hits, compliance produced 10,005 and study produced 309,253.

Results from a series of pilots suggested that the most effective searches (material retrieved relative to material found) combined Tier 1 and 2 search terms, all of which were focused on criminal justice agents, legitimacy, procedural justice plus associated terms. Additionally, combining criminal justice agencies (Tier 1) with outcomes (Tier 3) drew out literature on methods/factors that would have an impact on effects such as compliance, cooperation and confidence but might not necessarily have been derived from procedurally just procedures (e.g., risk of being caught). These results produced less favorable results but did provide some important material that was not captured using other search term combinations. Evidentiary terms (Tier 4) were added to Tier 1 and Tier 3 terms to help increase the inclusion rate (the number of relevant documents retrieved relative to the number of relevant documents found).

In summary, there were two search iterations conducted, resulting in 104 searches per database/data source:

1. Tier 1 + Tier 2 (8 x 7 keywords = 56 searches) and

2. Tier 1 + Tier 3 + Tier 4 (8 x 6 x 1 keywords = 48 searches)

35 The Campbell Collaboration | www.campbellcollaboration.org

Tier 2

Justice Approaches to Policing & Associated Terms

Tier 3

Outcomes Relevant to Legitimacy Policing

Tier 4

Evidence Focused Filters (using Boolean functions)

Police

PolicingCriminal Justice Law Enforcement Court

Prison Correction* Authorities

Procedural Justice Procedural Fairness Fair Procedure Fair Process Effective Policing Police Effectiveness Distributive Justice

Compliance

Comply

Confidence

Cooperat* (Cooperate, cooperation)

Fair* (fair, fairness, fairly) Legitima* (legitimacy, legitimate)

A database with each search string was developed so that researchers could cut and paste the keywords into select databases to reduce errors (incorrect spelling, missing keywords or mixing keyword combinations).

3.2.2 Search Field

Where this functionality exists in a database, the search field option allows researchers to limit the keyword search to title, abstract, reference list, whole document or a combination of fields. Results from a series of pilots indicated that the search anywhere in the document option produced more hits with a lower inclusion percentage than searches conducted on the abstract only, or title, abstract and descriptors. For example, police and procedural justice generated 136 records when the search field anywhere was used in CSA, with a 60% inclusion. When the abstract only field was used with the same terms in CSA, 61 records were returned and the inclusion rate jumped to 90%. Consequently, the research team decided to search on the abstract when this option was available.

3.2.3 Database selection

A fundamental objective was to develop a search strategy that could be replicated by other researchers in the future. Consequently, we utilized electronic databases/resources that could be generally accessed (e.g., not restricted material through an organizations intranet). Additionally, it was considered important to locate grey literature or material that is not formally published, such as working papers, unpublished dissertations, and reports (e.g., government, nongovernment, technical reports). After a review of subscription content to examine areas of content overlap between databases, and database functionality (e.g., capacity to search on multiple terms, restrict searches to abstract or similar, and download citations to reference manager such as EndNote), the research team decided on eight data sources, comprising six electronic databases/resources (CSA, Informit, Ingenta Connect, Ovid, Proquest and Web of Knowledge) and two library catalogues (National Police Library and the Cambridge University Library and dependent libraries). The databases are listed below with their corresponding weblink and the sub-databases used in the search strategy. Many of the databases used in our search subscribe to an extensive array of government sites and journal providersthese secondary databases are not all listed here. However, we highlight below specific key sites (e.g., NIJ and NCJRS) that are searched within the eight electronic data sources.

1. CSA

http://www.csa.com

a. Criminal Justice Abstracts

b. Sociological Abstracts

i. Education Resources Information Centre (ERIC)

ii. CSA Social Services Abstracts

36 The Campbell Collaboration | www.campbellcollaboration.org

c. SAGE Criminology

d. SAGE Sociology

e. SAGE Political Science

2. Informit

http://www.informit.com.au/databases

a. Australian Federal Police Digest

b. Australian Criminological Database (CINCH) Criminology

3. Ingenta Connect

http://www.ingentaconnect.com/

i. Informaworld (Taylor and Francis journals)

ii. Academic Press

iii. Elsevier

iv. Wiley Interscience (Blackwell Publishing)

4. Proquest

http://www.proquest.com

a. ProQuest Dissertations and Theses

b. ProQuest Psychological Journals

c. ProQuest Social Science Journals

d. ProQuest Legal Module

5. Ovid

http://gateway.ovid.com/autologin.html

a. PsycEXTRA

i. National Criminal Justice Reference Service (NCJRS)

ii. National Institute of Justice (NIJ)

iii. Home Office Publications

b. PsycINFO

6. Web of Knowledge

www.isiknowledge.com/

a. Web of Science Arts and Humanities Citation List

i. Science Citation Index

b. Web of Science Social Sciences Citation List

37 The Campbell Collaboration | www.campbellcollaboration.org

7. National Police Library via the National Policing Improvement Agency www.npia.police.uk/en/8495.htm

8. Cambridge University Library and Dependent Libraries Catalogue

http://ul-newton.lib.cam.ac.uk/

There was an emphasis on electronic data sources for retrieving information. However, in addition to the databases listed above, the research team also reviewed biographies and/or references from authors who have written influentially on the topic of procedural justice and police legitimacy. An author was considered influential when repeatedly cited within the compass of searched materials. Specifically, publication lists and biographies of the following authors were reviewed: Tom Tyler, Kristina Murphy, Lyn Hinds, Stephen Mastrofski, James Hawdon, Justice Tankebe and Michael Reisig. This method of searching publication lists was used primarily to add additional references that were not retrieved in the general search strategy.

3.2.4 Additional Searches

The research team checked the references of each eligible study included in the review to determine if there were other studies of interest that were not retrieved in the original search. Any new literature of interest was obtained and assessed for eligibility.

3.3 EXTRACTING AND CODING RESEARCH FOR THE REVIEW

Four trained research assistants (RAs) were responsible for interrogating the results of the systematic search in order to locate literature relevant and eligible for the systematic review. Preliminary eligibility characteristics are presented in section 3.1.

The following procedure for extracting and coding data for the review was applied.

1. Identification of potentially relevant material for meta-analysis. There were 963 records identified from the systematic literature search that provided the starting point for this systematic review. RAs used filters incorporated in the Excel spreadsheet to produce an initial list of potentially relevant material. Many documents, for example, were discussion pieces of legitimacy in policing or procedural justice: such material was helpful for the background literature review, but was not appropriate for the meta-analysis.

2. Coding of literature. The RAs utilized a detailed coding sheet (see Appendix 4) to assess whether literature was eligible for inclusion into the review. Eligible studies were then coded independently by two RAs, who entered results into the Microsoft Word coding document. Coding differences were resolved in a conference involving the lead authors. The coding sheet requests the following preliminary information:

I. Eligibility Checklist

38 The Campbell Collaboration | www.campbellcollaboration.org

II. Criteria for Eligibility (e.g., study includes an intervention and a comparison group)

III. Search information

IV. Reference information (e.g., authors, publication type )

Where the document met the criteria for eligibility, RAs then recorded the following information:

V. Intervention Information

VI. Unit of Analysis

VII. Research Design & Quality Assessment of Methodology

VIII. Outcomes Reported

IX. Outcome Variables

X. Effect Size/Reports of Statistical Significance

XI. Data

XII. Conclusions by Authors

3. Additional searches. The research team checked the references of eligible studies to determine if there were other studies of interest that were not retrieved in the original search. Any literature of interest was obtained and assessed for eligibility.

4. Data entry. Once the coding of literature had been completed, RAs entered data into Comprehensive Meta-Analysis 2.0.

5. Quality assessment. We recognize that the assessment of study quality can incorporate both objective and subjective elements. Therefore, we assessed the quality of studies in terms of their respective research design, sample bias, equivalency between groups, attrition bias, integrity of intervention delivery, integrity of maintaining differences between the treatment and control conditions, level of monitoring of the treatment delivery, research standards adhered to in terms of gathering outcome data, whether or not the analysis was conducted on intention to treat or actual evidence of treatment, whether or not mistakes in randomization occurred and how the mistakes were corrected (if at all), consistency of intervention periods and follow up/post intervention time frames both within and between experimental and comparison groups. We created an interval-level summed score of study quality from answers to the questions in Section VII of the coding sheet, with the lower the score, the higher the quality of design. We also assessed the impact of the individual components on effect size.

6. Treatment of qualitative research. Qualitative studies were not included in the current study. However, we did acknowledge seminal pieces of research in our literature background and discussion.

39 The Campbell Collaboration | www.campbellcollaboration.org

7. Independent assessment of retrieved studies. A reference list of research eligible for the meta-analysis was circulated to the following experts/authors who have given significant consideration to the concept legitimacy in policing and were in the top 5% of scholars in our search (ordered alphabetically): Stephen Mastrofski, Tina Murphy, Lawrence Sherman, Wesley Skogan, Heather Strang, Justice Tankebe, Tom Tyler and David Weisburd. Comments returned from these experts were very helpful and confirmed that we had not missed any publications of significance.

8. Coding quality assessment. Eligible studies were double coded and cross referenced to ensure consistency and quality of data entry.

3.4 RECRUITMENT, TRAINING AND INTER-RATER RELIABILITY

3.4.1 Recruitment

Four research assistants (RAs) were recruited to review, assess and code literature. RAs were either working on or had completed their undergraduate degree and had experience in data entry, statistics and database management. Each RA completed an inter-rater reliability test to ensure consistency in evaluating, assessing and coding literature.

3.4.1 Training, coding and test of inter-rater reliability

Training, double coding, and testing inter-rater reliability (IRR) were critical to ensuring consistent assessment and coding of research material.

The training process was facilitated by an initial meeting in which the lead authors provided an overview of the project, research objectives, systematic search, search strategy for the review, coding sheet and analysis.

RAs were initially assigned a training task in which they were required to determine the eligibility of 10 documents drawn from the Excel search by completing the coding sheet for each document. The lead authors reviewed each RAs assessment of the 10 documents and provided feedback on any discrepancies in determination of eligibility and/or coding. Following this training exercise, an IRR test was performed on a further 10 documents drawn from the Excel search results.

All eligible documents were double coded and coding differences were resolved in a conference involving the lead authors.

3.5 STATISTICAL PROCEDURES AND CONVENTIONS

Data synthesis was conducted using Comprehensive Meta-Analysis 2.0 (CMA), a statistical meta-analysis software package. We conducted separate meta-analyses for each outcome

40 The Campbell Collaboration | www.campbellcollaboration.org

measure, including direct (legitimacy, procedural justice, cooperation, compliance, satisfaction) and indirect (reoffending, crime, disorder) outcomes of policing interventions. Listed below is a short summary of the statistical procedures and conventions used in the review:

Independence of Effects. We computed effect sizes (log odds ratios and standardized mean differences) from a range of data available in the primary studies, using the methods implemented in CMA. For each of the five outcomes of interest, an effect size was computed, along with its variance. Studies were allowed to contribute an effect to each of the five outcome areas of interest to this review. Some studies reported data on multiple outcomes within one of our five outcomes of interest. In these cases, we calculated a single effect size per outcome for each study by using the appropriate statistical procedure (e.g., conversion from percentage success rate or regression coefficient to log odds ratio). These study specific procedures are detailed in the Technical Appendix (Appendix 2).

Effect Size. We obtained or calculated a single effect size per study per outcome. For outcomes reported as continuous in the primary papers, we calculated a standardized mean difference measure (Cohens d) and adjusted it for small-study effects (converted to Hedges g). For outcomes reported as dichotomous in the primary papers, we calculateda log odds ratio effect size and standard error. In reporting the meta-analysis, we used g for continuous outcomes and converted log odds ratios to odds ratios for dichotomous outcomes. Odds ratios were considered to be easier to interpret than log odds ratios for reporting purposes. We decided to preserve the authors conventions by presenting outcomes that were dichotomous in the original studies as odds ratios, which present the odds of a positive outcome in one group compared to the odds of the outcome in another group; and presenting outcomes that were continuous in the original studies as standardized mean differences, which represent differences in mean scores. We used the inverse-variance weight method to combine study effects and fit random effects models.

Heterogeneity. We assessed heterogeneity in the outcome measures using the Q-statistic (Hedges & Olkin, 1985) for each analysis. We used an I2 statistic (Higgins & Thompson, 2002) to estimate the proportion of the total variance in our dataset that could be attributed to between-study variance. I2 is measured from 0 to 100%, where a

41 The Campbell Collaboration | www.campbellcollaboration.org

large I2 indicates that the difference in results may be affected by factors other than the intervention.

Moderators. We explored possible moderators of policing legitimacy including intervention type, population under study, study design, year of publication and author using analogs to the ANOVA implemented via subgroup analyses in CMA.

Missing data. Where it was clear that effect sizes could be drawn but missing content made this impossible, the study was included if missing data were provided by corresponding with the original authors by May 15, 2010.4

Sensitivity. We conducted a series of sensitivity analyses to test the robustness of the results to the following: inclusion of studies where data was imputed, inclusion of poor quality studies (e.g., lack of treatment integrity).

Publication and small-sample bias. Additionally, as proposed by Sutton, Duval, Tweedie, Abrams and Jones (2000), we assessed the vulnerability of studies to publication and small-sample bias.

4 There were two studies where calculation of effect size required us to make some assumptions in the absence of clarification from study authors (see Appendix 2).

42 The Campbell Collaboration | www.campbellcollaboration.org

4 Results

First, we describe the attrition of publications starting from the search results and ending with the publications that were eligible for the meta-analysis. Second, we present results for publications with data suitable for meta-analysis of direct (legitimacy, procedural justice, compliance and cooperation, and satisfaction and confidence) and indirect (reoffending) outcomes. In Appendix 1, we provide a narrative of the studies that included an evaluation of a police-led legitimacy intervention designed to increase (either explicitly or implicitly through the employment of at least one procedural justice ingredient and a direct procedural justice outcome) police legitimacy with or without a design or data that enabled further statistical synthesis.

4.1 SYSTEMATIC REVIEW RESULTS

The systematic search identified 963 unique sources (e.g., published or unpublished documents) on police legitimacy and/or procedural justice and policing. We were not able to obtain 30 sources to review for eligibility despite a number of different attempts (through our own efforts as well as through the employment of an information specialist). These sources tended to be university dissertations where the university and/or supervisors could not locate the author, and/or organizational reports. Several of these missing sources had ambiguous citations and may have been unlocatable because they were incorrectly documented in the online database. Of the 933 sources we obtained, 163 studies reported on 176 police-led interventions aimed at improving legitimacy (either explicitly or implicitly), while 770 of the sources did not report on an intervention; instead, they were literature reviews, theoretical articles, or correlational studies. Of the 163 studies reporting on interventions, 69 were further excluded from the review because they were process evaluations only, or contained no comparison group, or compared two levels of treatment with no control group. The remaining 94 studies that contained comparative information were further screened for suitability for meta-analysis. Of these, 64 studies had no comparative data, did not report on an outcome of interest, or did not collect data at the individual level. These were excluded. The final set of 30 studies eligible for the meta-analysis contained 41 independent evaluations. Table 2 displays the attrition of publications.

43 The Campbell Collaboration | www.campbellcollaboration.org

Table 2. Attrition of publications

Stage of Review K

Unique sources 963

Not obtained sources 30

Obtained sources 933

Inventory of interventions

Not reporting a police-led intervention 770

Reporting a police-led intervention 163

Narrative review

Not an evaluation 69

Evaluations 94

Meta-analysis

Not eligible for meta-analysis 64

Studies eligible for meta-analysis 30

Evaluations eligible for meta-analysis 41

Three studies were excluded as non-legitimacy interventions according to our predefined inclusion criteria; that is, they did not identify a police intervention that either (1) explicitly sought to increase police legitimacy or (2) used at least one procedural justice element as an aim or component of the intervention. These studies were Hovell, Seid, & Liles, 2006; Friday, Lord, Exum, & Hartman, 2006; Urban, 2005. Only one study was included in our review that explicitly aimed at increasing police legitimacy but included no elements of procedural justice in the intervention (Weisburd, Morris, & Ready, 2008).

4.1.1 Characteristics of included studies

The 30 studies differed according to their intervention strategies, components of procedural justice, and a number of other factors. The following section describes these differences, which are summarized in Table 3.

Intervention strategies. The specific strategy used to influence citizen perceptions of police legitimacy differed between studies. The most common type of intervention strategy was community policing type interventions, where a closer partnership between the police and the community was established through community-oriented police training, the creation of

44 The Campbell Collaboration | www.campbellcollaboration.org

special community-oriented task forces or foot patrol officers, the provision of grants for community policing activities (e.g., Weed and Seed), or a combination of these. Nineteen studies evaluated some type of community policing strategy. Within these 19, two defined the intervention as reassurance policing, which differs from community policing in its specific targeting of fear of crime (Singer, 2004; Tuffin, Morris, & Poole, 2006); nine studies evaluated a specific set of community policing grants known as Weed and Seed (Dunworth & Mills, 1999a, 1999b, 1999c, 1999d, 1999e, 1999f, 1999g, 1999h; Zevitz, Palazzari, Frinzi, & Mallinger, 1997); and one study identified its intervention explicitly as beat policing (Bond & Gow, 1997). The other seven community policing studies evaluated a range of activities defined as community policing within the studies (Dai, 2007; Eckert, 2009; Murphy, Hinds, & Fleming, 2008; Panetta, 2000; Ren, Cao, Lovrich, & Gaffney, 2005; Robinson & Chandek, 2000; Skogan & Steiner, 2004). Three studies evaluated alternatives to traditional police complaints procedures, with one using an informal resolution process (Holland, 1996); one using an explicitly restorative justice-based procedure (Young, Hoyle, Cooper, & Hill, 2005); and one using an explicitly procedural justice-based procedure (Kerstetter & Rasinski, 1994). Two studies used police-led restorative justice conferencing, an alternative to court proceedings in which victims and offenders attended a police facilitated meeting to discuss the offence and possible reparations (Shapland et al., 2007; Sherman et al., 1998). Three studies used problem-oriented policing strategies (Hartstone & Richetelli, 2005; McGarrell & Chermak, 2004; Weisburd, Morris, & Ready, 2008). Although many of the interventions we screened used neighborhood watch strategies, only one study of this type (Hall, 1987) was eligible for the meta-analysis. Finally, one study used informal contact between police officers and school-age children (Hinds, 2009).

Outcomes. In the process of coding the outcome measures, we encountered substantial heterogeneity among conceptual and operational definitions of key outcomes. Difficulty in defining and measuring latent variables, such as satisfaction, was reflected in a wide variety of measures for each construct. For example, Skogan (2004) measured perceptions of police with 10 items measuring dimensions of police demeanor, responsiveness, and performance, while Hall (1987) measured perceptions of police using a single item: The Santa Ana Police Department is effective. Since some authors reported statistics for individual items (e.g., Sherman et al., 1998) while other authors only reported statistics for an aggregate scale (e.g., Ren, 2005), we could not perform meta-analysis on selected items that were the same across studies. Therefore, we decided to simply accept the authors definitions of the outcomes reported in their studies, even if these differed from other authors definitions. This meant that some authors operational definitions conflicted with others. For example, Ren (2005) identified confidence as a key outcome and measured it with seven items asking whether officers were fair, courteous, honest, not intimidating, worked with citizens, treated citizens equally, and showed concern; while Murphy (2008) used four items measuring confidence in police, police professionalism, whether police do their job well, and respect for police and called that legitimacy. Some studies varied in their terminology even within the study, such as the article by Weisburd and colleagues (2008) that referred to one of their constructs as procedural justice and legitimacy interchangeably. It was also not common for authors to

45 The Campbell Collaboration | www.campbellcollaboration.org

report validity or reliability statistics for their measurements, making it difficult for us to assess how differences in measurement may have affected studies estimates of intervention effectiveness. We acknowledge that this heterogeneity in measurement may have affected the results of the review. We also recognize that the ability of meta-analysis to address this issue is constrained by the quality of the body of primary research.

Elements of procedural justice. Only one study stated that the intervention explicitly aimed to increase legitimacy but included no elements of procedural justice in the intervention (Weisburd et al., 2008). The other studies all included at least one element of procedural justice. The restorative justice conferencing interventions tended to explicitly include more than one element of procedural justice (Shapland et al., 2007; Sherman et al., 1998). Other studies that explicitly included more than one element of procedural justice in the intervention were the reassurance policing interventions (Singer, 2004; Tuffin et al., 2006), some community policing interventions (Dai, 2007; Murphy et al., 2008; Ren et al., 2005; Skogan & Steiner, 2004; Zevitz et al., 1997), the alternative complaints procedures (Holland, 1996; Kerstetter & Rasinski, 1994; Young et al., 2005), and the informal contact intervention (Hinds, 2009). Citizen participation alone was a component of the Neighborhood Watch program (Hall, 1987). It was also a component in two of the problem-oriented policing interventions (Hartstone & Richetelli, 2005; McGarrell & Chermak, 2004) and several of the community policing interventions (Dunworth & Mills, 1999a, 1999b, 1999c, 1999d, 1999e, 1999f, 1999g, 1999h; Eckert, 2009; Robinson & Chandek, 2000). Trustworthy motives were the key procedural justice element of the beat policing intervention (Bond & Gow, 1997). None of the interventions explicitly included either neutrality or dignity and respect as key elements of the intervention; these were generally included with other elements in the interventions that used multiple elements of procedural justice.

Research design and data collection methods. Four studies were randomized field experiments (Shapland et al., 2007, 2008; Sherman et al., 1998; Weisburd et al., 2008), including one problem-oriented policing study and the two restorative justice conferencing studies. Fifteen studies used prepost only designs (Bond & Gow, 1997; Dunworth & Mills, 1999a, 1999b, 1999c, 1999d, 1999e, 1999f, 1999g, 1999h; Eckert, 2009; Hartstone & Richetelli, 2005; Hinds, 2009; Kerstetter & Rasinski, 1994; Murphy et al., 2008; Singer, 2004), and eleven studies used other nonrandomized designs (Dai, 2007; Hall, 1987; Holland, 1996; McGarrell & Chermak, 2004; Panetta, 2000; Ren et al., 2005; Robinson & Chandek, 2000; Skogan & Steiner, 2004; Tuffin et al., 2006; Young et al., 2005; Zevitz et al., 1997). Studies were required to use business as usual, or standard model policing as the comparison. The absence of randomized allocation to intervention and control conditions may have introduced bias into the results of some primary studies. Where possible, we have tried to identify any effects of primary study methodology through moderator analysis. Data reporting styles for each study are included in Table 3.

Targeted population. The studies differed with respect to their target populations, often according to the intervention strategy. Thus, the conferencing interventions targeted

46 The Campbell Collaboration | www.campbellcollaboration.org

offenders and victims (Shapland et al., 2007, 2008; Sherman et al., 1998), the community policing and reassurance policing interventions targeted community members generally (Dai, 2007; Dunworth & Mills, 1999a, 1999b, 1999c, 1999d, 1999e, 1999f, 1999g, 1999h; Eckert, 2009; Murphy et al., 2008; Panetta, 2000; Ren et al., 2005; Singer, 2004; Skogan & Steiner, 2004; Tuffin et al., 2006; Zevitz et al., 1997), the alternative complaints procedures targeted citizens with a complaint (Holland, 1996; Kerstetter & Rasinski, 1994; Young et al., 2005) and the informal interactions intervention targeted school-age children (Hinds, 2009). The problem-oriented policing strategies varied in their orientation; one targeted offenders (McGarrell & Chermak, 2004), and two targeted community members (Hartstone & Richetelli, 2005; Weisburd et al., 2008). One community policing intervention specifically targeted victims of domestic violence (Robinson & Chandek, 2000).

As demonstrated in the above descriptions, the situation arose during this review that many of the moderators we wished to investigate were in fact confounded within this group of studies. That is, studies that evaluated a particular intervention strategy tended also to use a particular evaluation methodology, look at particular outcomes, and target a particular population, in a different way to studies that evaluated other interventions. This confounding made it difficult for us to separate the effects due to particular study characteristics in the moderator analysis, and the reader is advised to keep this in mind when interpreting the results of the meta-analysis.

Table 3. Individual study characteristics

Study Outcomes Intervention Research design

Respondents N

Bond 1997 Cooperation Revictimisation

Satisfaction

Beat policing Prepost only Community members 905

Dai 2007 Compliance Community policing

Quasi-experimental

Community members 818

Prepost only Community members

457 136 546 473 483 391 633 407

Dunworth 1999

Akron Hartford

Las Vegas Manatee Pittsburgh Salt Lake City Seattle Shreveport

Satisfaction Revictimisation

Community policing (Weed and Seed)

Eckert 2009 Legitimacy Community policing

Prepost only Community members 636

Hall 1987 Effectiveness Neighborhood

Watch

Quasi-experimental

Community members 118

47 The Campbell Collaboration | www.campbellcollaboration.org

Study Outcomes Intervention Research design

Respondents N

Hartstone 2003 Revictimisation Problem oriented policing

Prepost only Community members 831

Hinds 2009 Legitimacy Informal contact Prepost only School children 414

Holland 1996 Satisfaction Alternative complaints process

Quasi-experimental

Complainants 384

Kerstetter 1994 Confidence Alternative complaints process

Prepost only Complainants 199

McGarrell 2004 Effectiveness Reoffending

Problem oriented policing

Quasi-experimental

Offenders 365

Murphy 2008 Legitimacy Community policing

Prepost only Community members 102

Procedural Justice

Satisfaction

Compliance

Panetta 2000 Procedural Justice Community policing

Quasi-experimental

Community members 190

Ren 2005 Confidence Community policing

Quasi-experimental

Volunteers 838

Robinson 2000 Cooperation Community policing

Quasi-experimental

Victims 336

Conferencing Experimental Offenders and victims

Shapland 2008 & Shapland 2007

Reoffending Satisfaction Procedural Justice

London Robbery (LOR)

158

London Burglary (LOB)

186

Northumbria Property (NCP)

105

Northumbria Assault (NCA)

165

Sherman 1998 Legitimacy

Procedural Justice Satisfaction

Conferencing Experimental Offenders and victims

Drink Driving (DD)

900

48 The Campbell Collaboration | www.campbellcollaboration.org

Study Outcomes Intervention Research design

Respondents N

Juvenile Property Shoplifting (JPS)

Compliance Reoffending

80

93

Juvenile Personal Property (JPP)

Youth Violence (YV)

80

Singer 2004 Satisfaction Reassurance policing

Prepost only Community members 1205

Skogan 2004 Satisfaction Community policing

Quasi-experimental

Community members ~540

Tuffin 2006

Manchester Lancashire Leicestershire Metropolitan Police (MPS) Surrey Thames Valley

Confidence Procedural Justice Reoffending

Reassurance policing

Quasi-experimental

Community members

365 386 354 390

404 389

Weisburd 2008 Procedural Justice

Reoffending

Problem oriented policing

Experimental Community members 800

Young 2005 Satisfaction Alternative complaints process

Quasi-experimental

Complainants 36

Zevitz 1997

Metcalfe Park (MP) Avenues West (AW)

Satisfaction Community policing (Weed and Seed)

Quasi-experimental

Community members

772 530

Total: 41 Total:

17,600

4.2 META-ANALYSIS

We conducted five separate meta-analyses for five a priori defined outcome measures, including direct and indirect outcomes. All of the outcomes were measured at the micro level with data collected on individuals. These were all outcomes that had been measured by at least two evaluations; other outcomes we searched for were either not measured in any eligible studies or were only measured in one study, rendering meta-analysis impossible. Specifically, the following outcomes were analyzed:

49 The Campbell Collaboration | www.campbellcollaboration.org

Direct Outcomes: Legitimacy, Procedural Justice, Compliance, Cooperation, Satisfaction, Confidence

Indirect Outcomes: Reoffending

4.2.1 Moderator analyses

We conducted a series of analogs to the ANOVA to determine the effect of different study-level moderator variables on the treatment effect size for each outcome. These were implemented via subgroup analyses in CMA, using a maximum likelihood estimation method. The variables of interest were: intervention type, research design, respondent type, crime type, year of publication, and country of publication. Most of these outcomes were categorical with multiple levels.

Since each outcome included a different group of studies, not all moderators were tested for all outcomes. For some outcomes, it was not possible to test the effects of a moderator variable because there was no variation among the included studies in terms of that moderator. For example, all studies that recorded reoffending as an outcome used an experimental design, so we could not compare the intervention effect on reoffending between experimental and quasi-experimental designs.

4.2.2 Sensitivity analyses

To examine the impact of our analysis decisions on the results, we conducted a series of sensitivity analyses. We used a funnel plot to inspect possible sources of bias in the data, including publication bias and small-study effects. We used a trim and fill method (Duval & Tweedie, 2004) to test for publication bias. We tested for small-study effects using an approach proposed by Egger (1997).

We ran two analyses to assess the sensitivity of each outcomes results to study quality. First, we ran the analysis using only peer-reviewed studies and reports, and excluding dissertations, and assessed the sensitivity of the direction and significance of the results with and without the inclusion of these grey studies for each outcome. We also awarded studies a pass or fail mark for the following quality elements: research design, sample bias, equivalency between treatment and control groups, attrition bias, integrity of intervention delivery, integrity of treatment and control group separation, level of monitoring of the treatment delivery, research standards, intention to treat analysis, mistakes in randomized allocation, and consistency of intervention periods and follow-up time frames between experimental and control groups. We also calculated a summed interval-scale quality score for each study. We ran a series of meta-regressions in CMA to assess the impact of these elements on study effect size.

50 The Campbell Collaboration | www.campbellcollaboration.org

Publication and small-study bias

We attempted to minimize publication bias by including grey literature in our search and document retrieval efforts. Several of the included studies are unpublished dissertations, as seen in Table 4.

Table 4. List of published and unpublished sources

Study Type of document Country of intervention

Bond 1997 Book chapter Australia

Dai 2007 Thesis United States

Dunworth 1999 Report United States

Eckert 2009 Thesis United States

Hall 1987 Thesis United States

Hartstone 2003 Report United States

Hinds 2009 Journal article Australia

Holland 1996 Journal article Australia

Kerstetter 1994 Journal article United States

McGarrell 2004 Report United States

Murphy 2008 Journal article Australia

Panetta 2000 Thesis United States

Ren 2005 Journal article United States

Robinson 2000 Journal article United States

Shapland 2007/2008 Report England

Sherman 1998 Report Australia

Singer 2004 Report England

Skogan 2004 Report United States

51 The Campbell Collaboration | www.campbellcollaboration.org

Study Type of document Country of intervention

Tuffin 2006 Report England

Weisburd 2008 Journal article United States

Young 2005 Journal article England

Zevitz 1997 Report United States

Detecting publication bias

We ran a series of diagnostic tests for publication bias in CMA, based on the effects for multiple outcomes from each study (i.e., more than one outcome per study). The tests were run separately for direct and indirect outcomes. In each test we included all outcomes for each study to increase the power of the bias detection tests, and to test for selective reporting within studies as well as across studies.

Direct outcomes

Figure 2 presents the funnel plot for all 58 direct outcomes from the 28 studies eligible for meta-analysis. Each point on the figure represents one outcome (e.g., procedural justice) for one evaluation. Studies with a larger sample size have a smaller standard error, and such points are situated closer to the top of the funnel. Smaller studies are located toward the bottom of the funnel. We expect that large studies with positive and negative effects will be published, so there should be symmetry across both sides of the funnel at the top of the graph. However, if there is publication or small-study bias present, we expect that the points appearing at the bottom of the funnel will be clustered on the positive effect side of the graph.

52 The Campbell Collaboration | www.campbellcollaboration.org

Figure 2. Funnel plot of standard error by log odds ratio: All included direct outcome measures (i.e., >1 outcome measure per study)

Figure 2 shows a fairly symmetrical distribution with a few extreme positive results on the right matched by extreme negative results on the left. This is reinforced with a diagnostic test (Eggers test run in CMA as demonstrated in Rothstein, Sutton, & Borenstein, 2005) that showed no evidence of publication or small-study bias, t(57) = .87, p = .386 (2-tailed).

Adjusting for publication bias

We attempted to adjust the log odds ratio for publication bias using Duval and Tweedies (2004) trim and fill method. This method uses an iterative procedure to determine where missing studies are likely to fall on the funnel plot, and then adds them to the analysis to determine the overall effect with the imputed studies included. This method suggested that there were no missing studies in the analysis, and therefore the imputed point estimate was the same as that obtained using only observed studies.

These statistical tests suggest that it is unlikely that our calculated effects were due to publication bias or small-study bias. We also feel confident that our systematic search captured a large number of unpublished and small studies, since a substantial proportion of our included studies are dissertations and unpublished reports.

Indirect outcomes

Figure 3 presents the funnel plot for all 26 indirect outcomes from the 15 studies eligible for meta-analysis. Each point on the figure represents one outcome (e.g., reoffending or revictimization) for one evaluation. The funnel plot is somewhat asymmetric, with the points appearing at the bottom of the funnel clustered on the negative effect side of the graph,

53 The Campbell Collaboration | www.campbellcollaboration.org

indicating that small-sample studies are more likely to be published if they show a reduction in reoffending.

This is reinforced with a diagnostic test (Eggers test run in CMA), which showed evidence of publication or small-study bias, t(24) = 4.10, p <0.001 (2-tailed).

Figure 3. Funnel plot of standard error by log odds ratio: All included indirect outcome measures (i.e., >1 outcome measure per study)

Adjusting for publication bias

The results of the Classic Fail Safe N (run in CMA) indicated that there would need to be an additional 26 studies to nullify the effect of the meta-analysis. We attempted to adjust Hedges g for publication bias using Duval and Tweedies (2004) trim and fill method. The results suggested that there were 10 missing studies in the analysis, and that their inclusion would lead to the random effects estimate becoming non-significant (g=0.015, CI -0.062, 0.09).

These statistical tests suggest that our calculated effects for indirect outcomes may be due to publication bias or small-study bias, and that the results of this meta-analysis should therefore be treated with caution.

4.3 DIRECT OUTCOMES

Odds Ratios

The articles included in the meta-analysis that reported on the outcomes of perceived legitimacy, procedural justice, cooperation, compliance, satisfaction, and confidence primarily reported these outcomes as dichotomous, usually a percentage or number of the group experiencing a positive outcome. For example, Sherman and colleagues (1998) reported on the percentage of respondents in the treatment and control groups who agreed with the question the police are legitimate. We converted these binary outcome measures for each study into an odds ratio (OR). The odds ratio is the odds of an event for the people who experienced the intervention divided by the odds of an event for the people who

54 The Campbell Collaboration | www.campbellcollaboration.org

experienced the comparison. In this case, the OR represented the ratio of the odds of a positive response to the question or questions used by the authors to measure perceptions of legitimacy, procedural justice, cooperation, compliance, satisfaction, and confidence for the two conditions.

Where studies reported on a continuous measure of these constructs rather than a dichotomous one, we computed a standardized mean difference (d) effect size and converted it into an odds ratio using the methods discussed in Lipsey and Wilson (2001). For details on study-specific calculation methods used, please refer to Appendix 2.

The true variance between studies is represented by the parameter 2. We used a maximum likelihood method to estimate 2 (as outlined in Borenstein, Hedges, Higgins, and Rothstein, 2009) and added the resulting estimate of between-study variance to each study weight. We then combined the weighted effect size estimates for an overall estimate of effect size for each outcome. These steps were implemented in CMA.

Combined outcomes

We decided to consider several of the originally proposed outcomes as a single outcome. We combined studies that measured citizen satisfaction with police and citizen confidence in police together in a single meta-analysis. This was done because of the low number of eligible studies that could have been included in each of these outcomes if they were kept separate. The operational definitions of satisfaction with and confidence in police often overlap in literature on policing. For example, the British Crime Survey uses the items, Police in the local area are doing a good or excellent job and Police are dealing with the things that matter to people in the community to measure public confidence in police (Home Office, 2011), while long-term community policing evaluations in Chicago use items such as How good a job are the police in your neighborhood doing in keeping order in the streets and sidewalks and How good a job are the police doing in dealing with the problems that really concern people in your neighborhood to measure public satisfaction with police (Skogan, 2004). We also included measures such as The police are effective in this category of outcome measure as these items were also referred to in the primary texts variously as confidence in police, satisfaction with police, and perceptions of police.

We also combined outcomes labeled citizen compliance and cooperation with police by study authors. Cooperation and compliance may be measured in two ways: through self-report as participants intent to cooperate or comply in future, or by direct observation of behavioral compliance and cooperation. Only one study in our sample (Dai, 2007) measured compliance using behavioral observation; all others used self-reported intent to comply or cooperate in future. Only two studies measured cooperation. Combining compliance and cooperation allowed us to retain all of these studies in the meta-analysis to ensure broad coverage and meaningful results.

55 The Campbell Collaboration | www.campbellcollaboration.org

4.3.1 Perceived Legitimacy

Only four studies (comprising seven evaluations) actually measured legitimacy as an outcome of the intervention and provided an effect size for legitimacy. Figure 4 summarizes the seven evaluations included in the meta-analysis on perceived legitimacy. Six evaluations had an OR greater than one, indicating that for these studies the policing intervention was associated with an increase in perceptions of police legitimacy. However, only one of the evaluations with an OR greater than one was statistically significant: the drink driving experiment from the Canberra RISE evaluations (Sherman et al., 1998).

The weighted mean OR for the seven evaluations combined was 1.58 using a random effects model. However, the 95% confidence interval for the OR was very wide and included 1 (lower limit = 0.85, upper limit = 2.95). This result indicates that when between-study heterogeneity was considered in the model, there was no discernible effect of policing interventions on perceptions of police legitimacy. Although the point estimate is highly positive, the variation between studies was too large to allow us to attribute the effect to the intervention, rather than the study-level differences. A possible explanation for this result can be found in an examination of the primary studies. The definition and measurement of legitimacy varied widely between primary studies, making the studies so heterogeneous that it is impossible to separate the within-study effects of the intervention from the effects of the between-study variation. Supporting this observation, the I2 statistic indicated that 93% of the variance in the OR could be attributed to study-level factors (I2 = 93.08, 2 = .589, s.e. = .48), and the seven evaluations were significantly heterogeneous according to the Q statistic (Q(6) = 86.73, p < .001).

Moderator analyses conducted in CMA showed a significant variation in odds ratio effect sizes between studies according to a number of factors (see tables 5 and 6). The effects of study-level moderators are difficult to interpret in this case because four of the included interventions came from the Canberra RISE study (Sherman et al., 1998), which differed from the other included interventions on almost all of our coded moderator variables. The results of the subgroup analyses collectively indicate that the RISE studies found a greater increase in legitimacy as a result of the intervention than all of the other studies, but given their distinct characteristics we cannot interpret this difference. We need more evaluation studies to measure legitimacy as an outcome, and we need that measurement to be standard across studies in order to make any further judgments about the effect of legitimacy interventions on legitimacy.

The overall effect direction and significance for perceived legitimacy was not affected by the inclusion of studies using imputed data (Q(1) = 0.85, p = 0.356), or the inclusion of unpublished studies (Q(1) = 0.95, p = 0.332). The effect size decreased slightly when low-quality studies were excluded, but the significance and direction of the effect were robust to the inclusion or exclusion of these studies (see Table 6).

56 The Campbell Collaboration | www.campbellcollaboration.org

Table 5. Relationship of study characteristics to odds ratio effect sizes for the outcome of legitimacy (k = 7)

Study Characteristic k OR 95% CI

Low

95% CIHigh z p Q (df)* P

Intervention 5.98 (2) 0.050

Community policing 2 0.99 0.53 1.86 0.04 0.968

Conferencing 4 2.67 1.50 4.74 3.34 0.001

Informal Contact 1 1.08 0.43 2.68 0.16 0.875

Research Design 10.38 (1) 0.001

Prepost only 3 1.02 0.68 1.53 0.08 0.935

Randomized trial 4 2.87 1.77 4.66 4.28 <.001

Respondent Type 5.98 (2) 0.050

Community 2 0.99 0.53 1.86 0.04 0.968

Offenders 4 2.67 1.50 4.74 3.34 0.001

School children 1 1.08 0.43 2.68 0.16 0.875

*Note: This is the random-effects Q-between testing the difference across the means of the subgroups

Table 6. Effects of study quality indicators on the outcome of perceived legitimacy (k=7)

Quality Indicator b s.e. z p 2

Sample bias 0.66 0.63 1.05 0.295 0.29

Equivalency of control group 0.46 0.65 0.70 0.483 0.32

Consistency of intervention and follow-up periods 0.60 0.47 1.26 0.207 0.26

Research standards adhered to 0.46 0.65 0.70 0.483 0.32

Quality composite score 0.16 0.21 0.76 0.449 0.31

Year of publication 0.10 0.03 3.58 <.001 0.08

57 The Campbell Collaboration | www.campbellcollaboration.org

Figure 4. Forest plot of effect sizes, legitimacy

58 The Campbell Collaboration | www.campbellcollaboration.org

4.3.2 Perceived Procedural Justice

Six studies provided outcome data on procedural justice, giving fourteen independent effect sizes overall. Figure 5 summarizes the 14 evaluation studies included in the meta-analysis on procedural justice. Thirteen of the fourteen evaluations had an OR greater than one, indicating that for most included studies the policing intervention was associated with an increase in perceived procedural justice. Of the 13 evaluations with an OR greater than one, 5 were statistically significant: the drink driving experiment from the Canberra conferencing study (Sherman et al.,1998), and the Leicestershire, Manchester, Metropolitan and Thames Valley arms of the English reassurance policing evaluation (Tuffin et al., 2006).

Overall, the interventions were associated with a large, significant increase in perceptions of procedural justice. The weighted mean OR for the 14 evaluations combined was 1.47 using the random effects model, and the 95% confidence interval did not include 1 (lower limit = 1.16, upper limit = 1.86).

The 14 evaluations were significantly heterogeneous according to the Q statistic (Q(13) = 45.37, p < .001). The I2 statistic indicated that 71% of the variation in the OR could be attributed to study-level factors (I2 = 71.35, 2 = .13, s.e. = .08). Results of the moderator analysis are shown in Table 7. Of all the intervention strategies included in this outcome, reassurance policing interventions tended to show higher effect sizes than other intervention strategies. In addition, interventions targeting community members tended to show larger effects on procedural justice than interventions targeting offenders alone.

The results were sensitive to the publication status of the studies. The evaluations included in this outcome were primarily unpublished reports and dissertations. Only two were peer-reviewed journal articles. These peer-reviewed studies recorded an overall OR of 1.11, with a large confidence interval that included 1 (95%CI lower = 0.64, 95%CI upper = 1.91, p = 0.715). By contrast, the studies that were not published and peer reviewed recorded a large positive overall OR of 1.56, with a confidence interval that did not include 1 (95%CI lower = 1.21, 95%CI upper = 2.01, p = 0.001). The results were not sensitive to methods chosen by the reviewers; the effect sizes for procedural justice were not significantly affected by any of the quality indicators we recorded (see Table 8), or any assumptions or imputed data used when we calculated the studies effect sizes (Q(1) = 0.50, p = 0.481).

Table 7. Relationship of study characteristics to odds ratio effect sizes for the outcome of perceived procedural justice (k = 14)

Study Characteristic k OR 95% CI

Low

59 The Campbell Collaboration | www.campbellcollaboration.org

95% CIHigh z p Q (df)* p

Intervention 3.34 (3) 0.341

Community policing 2 1.06 0.57 1.97 0.18 0.858

Reassurance policing 6 1.85 1.29 2.64 3.38 0.001

95% CIHigh z p Q (df)* p

Conferencing 5 1.35 0.88 2.05 1.37 0.171

Problem-oriented policing 1 1.12 0.52 2.41 0.28 0.779

Research Design 1.80 (2) 0.407

Prepost test 1 1.10 0.48 2.50 0.21 0.827

Quasi-experimental 7 1.71 1.23 2.37 3.20 0.001

Randomized trial 6 1.29 0.89 1.86 1.35 0.177

Respondent Type 0.226 (1) 0.635

Community 9 1.53 1.13 2.06 2.80 0.005

Offenders 5 1.34 0.87 2.08 1.32 0.186

*Note: This is the random-effects Q-between testing the difference across the means of the subgroups

Table 8. Effects of study quality indicators on the outcome of procedural justice (k=14)

Quality Indicator b s.e. z p 2

Mistakes 0.40 0.47 0.85 0.395 0.124

Sample bias 0.41 0.25 1.66 0.098 0.095

Equivalency of control group 0.40 0.47 0.85 0.395 0.124

Delivery integrity 0.38 0.32 1.21 0.227 0.114

Treatment integrity 0.19 0.25 0.75 0.455 0.125

Monitoring of treatment delivery 0.32 0.43 0.750 0.454 0.124

Consistency of follow-up 0.27 0.31 0.860 0.390 0.120

Quality composite score 0.10 0.08 1.23 0.221 0.114

Year of publication 0.01 0.03 0.35 0.725 0.132

60 The Campbell Collaboration | www.campbellcollaboration.org

Study Characteristic k OR 95% CI

Low

Figure 5. Forest plot of effect sizes, procedural justice

61 The Campbell Collaboration | www.campbellcollaboration.org

4.3.3 Compliance and Cooperation

Eight evaluations reported compliance or cooperation as an outcome of the intervention. Figure 6 summarizes the eight evaluations included in the meta-analysis on citizen compliance and cooperation. Seven of the eight evaluations had an OR greater than one, indicating that for these studies the policing intervention was associated with an increase in compliance or cooperation. Of the seven evaluations with an OR greater than one, three were statistically significant (Sherman et al.s Drink Driving experiment, 1998; Dais 2007 community policing evaluation; and Bond and Gows 1997 beat policing evaluation).

Overall, the interventions had a large, significant, positive effect on the combined compliance and cooperation measure. The weighted mean OR for the eight evaluations combined was 1.62 using the random effects model. The 95% confidence interval for the OR did not include 1 (lower limit = 1.13, upper limit = 2.32). Studies measuring cooperation tended to show higher effect sizes than studies measuring compliance (see Table 9); however, this difference was not significant, indicating that including both sets of studies in the one meta-analysis did not substantially affect the result.

Study-level factors contributed significantly to the variation in the effect sizes recorded by the studies. The eight evaluations were significantly heterogeneous according to the Q statistic (Q(7) = 22.05, p = .002). The large I2 statistic indicated that 68% of the variance in the OR may be a result of study factors (I2 = 68.26, 2 = .17, s.e. = .15). As shown in Table 9, police-led restorative justice conferences tended to have larger effects on compliance and cooperation than any other type of intervention. Similarly, interventions targeting offenders recorded significantly larger effect sizes for compliance and cooperation than interventions targeting victims of crime or the general public. Studies attention to treatment integrity significantly affected their results, such that studies with lower treatment integrity tended to report higher effect sizes for the combined measure (see Table 10).

The sensitivity analysis indicated that the results were different when published and unpublished studies were not included together in the same meta-analysis. The two published, peer-reviewed articles reported a combined OR of 0.94, indicating a negative effect of legitimacy policing on compliance and cooperation, although the confidence interval was very wide and included 1 (lower limit = 0.71, upper limit = 1.24, p = 0.663). In contrast, the six unpublished sources had a combined OR that was very large and highly significant (OR = 2.17, lower limit = 1.67, upper limit = 2.80, p < 0.001). This analysis did not use imputed data.

Table 9. Relationship of study characteristics to odds ratio effect sizes for the outcome of compliance (k = 8)

Study Characteristic k OR 95% CI

Low

62 The Campbell Collaboration | www.campbellcollaboration.org

95% CIHigh z p Q (df)* p

Outcome 0.98 (1) 0.321

95% CIHigh z p Q (df)* p

Compliance 6 1.81 1.19 2.73 2.80 0.005

Cooperation 2 1.23 0.65 2.33 0.62 0.534

Intervention 3.26 (2) 0.196

Community policing 3 1.18 0.76 1.82 0.74 0.459

Beat policing 1 2.08 0.92 4.67 1.77 0.077

Conferencing 4 2.00 1.28 3.12 3.03 0.002

Research Design 1.06 (2) 0.590

Prepost test 2 1.47 0.77 2.80 1.17 0.242

Quasi-experimental 2 1.29 0.66 2.52 0.74 0.461

Randomized trial 4 1.97 1.16 3.32 2.53 0.011

Respondent Type 6.34 (2) 0.042

Community 3 1.64 1.09 2.48 2.36 0.018

Offenders 4 2.02 1.34 3.04 3.38 0.001

Victims 1 0.80 0.44 1.46 -0.72 0.473

*Note: This is the random-effects Q-between testing the difference across the means of the subgroups

Table 10. Effects of study quality indicators on the outcome of compliance (k=8)

Quality Indicator b s.e. z p 2

Sample bias 0.42 0.28 1.48 0.138 0.067

Attrition bias 0.57 0.52 1.09 0.274 0.101

Equivalency of control group 0.57 0.52 1.09 0.274 0.101

Delivery integrity 0.05 0.37 0.12 0.901 0.120

Treatment integrity 0.81 0.30 2.71 0.007 0.032

Consistency of intervention and follow-up periods 0.35 0.33 1.05 0.292 0.086

Intention to treat 0.57 0.52 1.09 0.274 0.101

Quality composite score 0.05 0.10 0.51 0.609 0.119

Year of publication 0.02 0.04 0.62 0.533 0.108

63 The Campbell Collaboration | www.campbellcollaboration.org

Study Characteristic k OR 95% CI

Low

Figure 6. Forest plot of effect sizes, compliance

64 The Campbell Collaboration | www.campbellcollaboration.org

4.3.4 Satisfaction and Confidence

The most commonly used outcome measure in our population of studies was some measure of satisfaction, confidence, or perception of police effectiveness. Since measurement of this outcome was not standardized between studies, and it would not be defensible to assume these measures were independent of one another, we included all three attitudinal measures in the one meta-analysis. Figure 7 summarizes the twenty-nine evaluations included in the meta-analysis on satisfaction with and confidence in the police. Of the 29 included evaluations, 27 had an OR greater than one, indicating that for these studies the policing intervention was associated with an increase in positive attitudes towards the police. Of the 27 evaluations with an OR greater than one, 16 were statistically significant (Bond & Gow, 1997; Dunworth & Mills, 1999a, 1999d, 1999e; Hall, 1987; Holland, 1996; Kerstetter & Rasinski, 1994; McGarrell & Chermak, 2004; Murphy et al., 2008; Ren et al., 2005; Shapland et al., 2007; Skogan & Steiner, 2004; Tuffin et al., 2006; Zevitz et al., 1997).

Overall, legitimacy interventions resulted in a large, significant increase in positive perceptions of police. The weighted mean OR for the 29 evaluations combined was 1.75 using the random effects model. The 95% confidence interval did not include 1 (lower limit = 1.54, upper limit = 1.99). Each of the outcome measurements (satisfaction, confidence, and effectiveness) also independently recorded an overall significant positive effect size, indicating that the choice to combine them did not affect the overall result.

The differences between studies contributed significantly to the variation in effect sizes. The 29 evaluations were significantly heterogeneous according to the Q statistic (Q(28) = 66.68, p < .001). The large I2 statistic indicated that 58% of the variance in the OR may be a result of study factors (I2 = 58.01, 2 = .07, s.e. = .03). Interventions targeting victims alone tended to record smaller effect sizes than interventions targeting either community members in general or offenders alone (see Table 11). The integrity of intervention delivery affected the results, such that studies with lower delivery integrity tended to record higher effect sizes for the combined outcome (see Table 12).

The findings for the combined satisfaction outcome were not sensitive to the inclusion of unpublished and published studies in the same meta-analysis (Q(1) = 3.22, p = 0.073), although the five published, peer reviewed studies recorded higher effect sizes overall (OR = 2.27, lower limit = 1.67, upper limit = 3.08) than the unpublished studies (OR = 1.67, lower limit 1.45, upper limit = 1.92). None of the effect sizes included in this outcome required imputed data for their calculation.

Table 11. Relationship of study characteristics to odds ratio effect sizes for the outcome of satisfaction (k = 29)

Study Characteristic k OR 95% CI

Low

65 The Campbell Collaboration | www.campbellcollaboration.org

95% CIHigh z p Q (df)* p

Outcome 2.43 (2) 0.296

Study Characteristic k OR 95% CI

Low

95% CIHigh z p Q (df)* p

Confidence 9 1.65 1.30 2.10 4.09 <0.001

Satisfaction 18 1.75 1.49 2.05 6.77 <0.001

Effectiveness 2 2.58 1.55 4.30 3.64 <0.001

Intervention 10.86 (7) 0.145

Community policing 3 1.79 1.31 2.45 3.62 <0.001

Beat policing 1 2.77 1.29 5.97 2.60 0.009

Reassurance policing 7 1.48 1.13 1.94 2.87 0.004

Weed and Seed 10 1.61 1.33 1.95 4.84 <0.001

Conferencing 3 1.67 1.00 2.79 1.96 0.050

Neighborhood Watch 1 3.05 1.40 6.64 2.81 0.005

Problem-oriented policing 1 2.31 1.23 4.31 2.62 0.009 Alternative complaintsprocess 3 2.78 1.81 4.25 4.70 <0.001

Research Design 0.80 (2) 0.672

Prepost test 15 1.69 1.43 2.01 6.03 <0.001

Quasi-experimental 11 1.92 1.53 2.41 5.58 <0.001

Randomized trial 3 1.67 0.97 2.85 1.86 0.064

Respondent Type 0.386 (2) 0.824

Community 25 1.75 1.52 2.01 7.90 <0.001

Offenders 2 2.03 1.24 3.32 2.80 0.005

Victims 2 1.59 0.72 3.48 1.15 0.251

*Note: This is the random-effects Q-between testing the difference across the means of the subgroups

66 The Campbell Collaboration | www.campbellcollaboration.org

Table 12. Effects of study quality indicators on the outcome of satisfaction/confidence (k=29)

Quality Indicator b s.e. z p 2

Mistakes 0.56 0.41 1.37 0.171 0.062

Sample bias 0.15 0.13 1.08 0.281 0.066

Attrition bias 0.56 0.41 1.37 0.171 0.062

Equivalency of control group 0.08 0.17 0.44 0.658 0.066

Delivery integrity 0.28 0.14 1.96 0.050 0.057

Treatment integrity 0.21 0.17 1.22 0.222 0.064

Monitoring 0.12 0.016 0.79 0.427 0.068 Consistency of intervention and follow-upperiods 0.17 0.14 1.18 0.238 0.065

Intention to treat 0.08 0.25 0.34 0.731 0.069

Quality composite 0.05 0.04 1.21 0.226 0.067

Year of publication 0.02 0.01 1.56 0.119 0.061

67 The Campbell Collaboration | www.campbellcollaboration.org

Figure 7. Forest plot of effect sizes, satisfaction and confidence

68 The Campbell Collaboration | www.campbellcollaboration.org

4.4 INDIRECT OUTCOMES

Whilst our review sought to examine the direct outcomes of police efforts to improve legitimacy, we also coded what the academic research literature defines as the indirect benefits of legitimacy policing. These indirect outcomes are articulated as reductions in crime and disorder as well as reductions in self-reported reoffending (or revictimization). In our review, 15 studies comprising of 26 evaluations reported on the indirect outcome of reoffending. These studies varied in their measurement of reoffending; some used self-reported or official police records of reoffending (McGarrell & Chermak, 2004; Shapland et al., 2008; Sherman et al., 1998; Weisburd et al., 2008), and some used self-reported victimization (Bond & Gow, 1997; Dunworth & Mills, 1999a, 1999b, 1999c, 1999d, 1999e, 1999f, 1999g, 1999h; Hartstone & Richetelli, 2003; Tuffin et al., 2006).

Because these outcomes were generally measured on a continuous scale, rather than dichotomously, we used the standardized mean difference (g) as the effect size. Following Hedges and Olkins (1985) procedure for meta-analysis, we converted the reoffending outcome measure for each study into a standardized mean difference score (d). The d score describes the difference between the experimental and control groups on the outcome measure, and is not affected by different studies measurement scales. We then adjusted the d score for each study to account for bias resulting from small study sample size (Hedges & Olkin, 1985), resulting in a g value for each study. Formulae for this procedure were drawn from Deeks and Higgins (2005). The following calculations were conducted using CMA.

4.4.1 Reoffending

Fifteen studies measured reoffending as an outcome of the intervention, contributing a total of twenty-six effect sizes to the meta-analysis. Figure 8 summarizes the 26 evaluations included in the meta-analysis on reoffending. Of the 26 effect sizes, 20 had a negative g value, indicating that for these studies the policing intervention was associated with a decrease in reoffending. Of the 20 evaluations with a negative g value, only two were statistically significant (Dunworth & Mills, 1999d,e).

Despite most individual studies showing a null effect for reoffending, the meta-analysis showed that the interventions overall resulted in a decrease in reoffending that was marginally significant at the .05 level. The weighted mean g for the 26 evaluations combined was 0.07 using the random effects model (see Table 16). The 95% confidence interval for g included zero at the very upper limit (lower limit = 0.14, upper limit = 0.00). A substantial difference existed between the effect sizes of the studies that measured reoffending through official statistics or offender self-report, and those that measured reoffending using victim self-reports of revictimization (see Table 13). The moderator analysis demonstrated that studies measuring reoffending using official data recorded a null intervention effect overall (g = 0.03, 95%CI lower = 0.05, 95%CI upper = 0.11, p = 0.473). However, studies measuring reoffending using victimization self-reports found a large and significant decrease

69 The Campbell Collaboration | www.campbellcollaboration.org

in revictimization as a result of the interventions overall (g = 0.13, 95%CI lower = 0.23, 95%CI upper = 0.05, p = 0.001).

Study-level variation did not contribute significantly to the differences in effect sizes between the studies, suggesting that the majority of the variation in effect size was due to the effectsof the intervention. The 26 evaluations were not significantly heterogeneous according to the Q statistic (Q(25) = 36.30, p = .067). The I2 statistic indicated that 31% of the variance in g may be a result of study factors (I2 = 31.11, 2 = .01, s.e. = .01). Studies using a prepost only evaluation design tended to record significantly larger effect sizes than evaluations using other quasi-experimental designs, or randomized trials (see Table 13). The large effect sizes of the Weed and Seed group of community policing interventions, all of which used pre post only evaluation designs, likely contributed to this effect. Since prepost only designs are susceptible to a range of biases (Shadish, Cook, & Campbell, 2002), this is reason to interpret the results of this meta-analysis with some caution. In addition, studies integrity of intervention delivery significantly affected their results, such that studies with lower delivery integrity tended to report larger decreases in reoffending as a result of the intervention (see Table 14). Only one peer reviewed study was included in this outcome (Weisburd et al., 2008). When this study was not included in the analysis, the results indicated a significant effect of legitimacy interventions on reoffending; the unpublished studies alone had an overall g of 0.08 (lower limit = 0.52, upper limit = 0.01, p = 0.035; Q(1) = 0.68, p = 0.411). The Weisburd et al. study was also the only study for which we imputed any data to calculate the effect size (see Appendix 2).

Table 13. Relationship of study characteristics to Hedges' g effect sizes for the outcome of reoffending (k = 8)

Study Characteristic k g s.e. 95% CI

Low

70 The Campbell Collaboration | www.campbellcollaboration.org

95% CIHigh z p Q (df)* p

Outcome 7.418 (1) 0.006

Reoffending 10 0.03 0.04 0.05 0.11 0.72 0.473

Revictimisation 16 0.13 0.04 0.23 0.05 3.11 0.001

Intervention 7.01 (4) 0.135

Reassurance policing 6 0.06 0.08 0.21 0.09 0.77 0.440

Beat policing 1 0.10 0.14 0.37 0.17 0.71 0.479

Weed and Seed 8 0.22 0.07 0.37 0.08 3.01 0.003

Problem oriented policing 3 0.03 0.07 0.11 0.17 0.43 0.666

Conferencing 8 0.02 0.06 0.14 0.10 0.36 0.723

Research design 7.65 (2) 0.022

Prepost only 10 0.18 0.06 0.29 0.07 3.25 0.001

Quasi-experimental 7 0.01 0.06 0.11 0.13 0.18 0.855

95% CIHigh z p Q (df)* p

Randomized trial 9 0.01 0.05 0.09 0.09 0.01 0.990

Respondent type 3.406 (1) 0.065

Community members 17 0.11 0.04 0.19 0.03 2.59 0.009

Offenders 9 0.02 0.06 0.09 0.12 0.33 0.741

*Note: This is the random-effects Q-between testing the difference across the means of the subgroups

Table 14. Effects of study quality indicators on the outcome of reoffending (k=8)

Quality Indicator b s.e. z p 2

Sample bias 0.12 0.07 1.82 0.068 0.006

Delivery integrity 0.28 0.13 2.10 0.036 0.006

Treatment integrity 0.08 0.08 1.04 0.299 0.008

Monitoring 0.03 0.15 0.19 0.847 0.009

Consistency of intervention and follow-up periods 0.01 0.08 0.11 0.914 0.009

Quality composite score 0.00 0.04 0.07 0.947 0.009

Year of publication 0.01 0.01 0.65 0.518 0.010

Study Characteristic k g s.e. 95% CI

Low

71 The Campbell Collaboration | www.campbellcollaboration.org

Figure 8. Forest plot of effect sizes, reoffending

72 The Campbell Collaboration | www.campbellcollaboration.org

4.5 RESULTS SUMMARY

Overall, the results show promising outcomes for the effects of police-led interventions that sought to enhance legitimacy. As seen in Tables 15 and 16, the combined outcome of satisfaction and confidence showed the highest overall effect for the intervention studies included in our review. The confidence interval for this effect was reasonably small, indicating that the effect of legitimacy interventions on satisfaction and confidence was not only large but reliable in the population of studies. The second highest effect size was found for the combined compliance and cooperation outcome. When this outcome was broken down by study population, we found that interventions targeting offenders tended to report higher effect sizes for compliance and cooperation than interventions targeting victims or the general population.

The outcome of procedural justice reported a smaller, but positive and very stable increase as a result of police legitimacy interventions. Neither procedural justice nor satisfaction/confidence were significantly affected by the moderators in our analysis, which suggests that legitimacy interventions have robust effects on these outcomes regardless of the context in which they are implemented.

Interestingly, studies that generally stated that the interventions were designed to enhance legitimacy did not, according to our analysis, actually affect the construct of legitimacy in the way it was measured by the authors of the primary studies. The estimated effect size for the outcome legitimacy is quite large; however, the confidence interval is very large and indicates a high amount of uncertainty in the estimate. An investigation of the studies contributing effect sizes to the legitimacy outcome reveals that this is partly due to the four RISE groups (Sherman et al., 1998) reporting higher legitimacy scores than the other studies included in this outcome. Since the RISE studies differed from the other studies in terms of population under study, research methods, and legitimacy measures, it is difficult to determine why exactly their effect sizes were different to the others. These sources of heterogeneity are worthy of further investigation. We simply need more intervention studies that consistently measure legitimacy as an outcome.

Finally, police-led legitimacy interventions showed a marginal effect on reoffending as an indirect outcome measure. When reoffending was broken down by measurement method, studies that measured reoffending using official police data and self-reported reoffending showed no effect of legitimacy interventions; however, studies that measured self-reported victimization showed a large decrease in revictimization as a result of the interventions. This difference could be indicative of many things, but a possible explanation is that official data and self-reported offending are influenced in two ways by interventions: either the actual offending may decrease, but reporting increase, giving an overall null effect. In contrast, in the victimization measures there is less incentive to underreport. Another possible explanation is that legitimacy interventions actually affect reoffending behavior and revictimization differently.

73 The Campbell Collaboration | www.campbellcollaboration.org

The results of the meta-analysis indicate that legitimacy interventions have a reliable impact on some outcomes and a widely variable impact on others. The sensitivity analyses indicated that these results were generally not due to methodological decisions made by the reviewers. The moderator analyses did demonstrate that study-level variables, such as evaluation design, may have influenced the results for some outcomes. However, the fact that only a small number of studies were found that could be included in the meta-analysis limits the robustness of the moderator analysis somewhat. Because there were so few studies that fulfilled all of our inclusion criteria and reported sufficient data to perform the meta-analysis, some of the levels of the moderator variables were represented by only one study (i.e., had only one observation). For example, for the outcome of legitimacy, only one study reported an effect size for violent crime, and only one study reported on drink driving. This means that the inclusion of additional effect sizes from new studies could substantively change the results of the moderator analysis for some outcomes.

Table 15. Summary of results for direct outcomes

Random Effects Model Results

Legitimacy Procedural Justice

95% CI Lower 0.85 1.16 1.13 1.54 Odds Ratio 1.58 1.47 1.62 1.75 95% CI Upper 2.95 1.86 2.32 1.99 P .148 .001 .009 <.001

Q 86.73 45.37 22.05 66.68 I2 93% 71% 68% 58% K 7 14 8 29

Table 16. Summary of results for indirect outcomes

Random Effects Model Results

Reoffending

95% CI Lower 0.14 Standardized Mean Difference 0.07 95% CI Upper 0.00

P 0.053

Q 36.30 I2 31% K 26

74 The Campbell Collaboration | www.campbellcollaboration.org

Compliance Cooperation

Satisfaction Confidence

5 Discussion and Conclusion

Our systematic review explored the direct and indirect outcomes of a range of police-led interventions that sought to enhance citizen perceptions of police legitimacy. We included studies that evaluated police approaches to crime prevention or crime control where the intervention explicitly sought to enhance legitimacy or comprised at least one of the four principles of procedural justice: citizen participation, neutrality during the policecitizen encounter, efforts by police to communicate dignity and/or respect for citizens, and trustworthy motives. We included any type of public police intervention (e.g., routine patrols, traffic stops, community policing, reassurance policing, problem-oriented policing, conferencing) where there was a clear statement (articulated in the source material) that the intervention involved some type of training, directive or organizational innovation that sought to increase legitimacy. We also included studies where the stated intervention (articulated in the source material) involved some type of training, directive or organizational innovation that used at least one of the four core ingredients of procedural justice: police encouraging citizen participation, remaining neutral in their decision making, demonstrating dignity and respect throughout interactions, and conveying a sense of trustworthiness in their motives. Bottoms and Tankebes (2012) central thesis that legitimacy is dialogic in nature is consistent with our efforts to cast a wide net across the extant evaluation literature and gather as many different types of interventions that captured the essence of legitimacy policing. For our review, we were more interested in the manner in which interventions were delivered than the mechanism or vehicle in which the engagement between police and citizens occurred. As such, we understood that a broad set of police interventions could potentially increase citizen perceptions of police legitimacy so long as the interventions had common, legitimacy enhancing dialogue.

The studies included in our review also had to report at least one direct outcome measure that fell within the broader construct of legitimacy. These direct outcomes included measures of perceived legitimacy, perceived procedural justice and measures of citizen compliance, cooperation, confidence and satisfaction with the police. We also included studies in our systematic review that reported a range of indirect outcomes of police efforts to foster legitimacy. From the outset, we expected these indirect outcomes to include changes, post intervention, in levels of reoffending, crime and/or disorder. Overall, our search of the literature found a relatively small and diverse group of studies that met our review criteria. Moreover, very few studies used quasi-experimental or experimental methods to explore the direct and indirect impacts of legitimacy policing.

75 The Campbell Collaboration | www.campbellcollaboration.org

Our review finds that police can use a variety of police-led interventions (including conferencing, community policing, problem-oriented policing, reassurance policing, informal police contact, and neighborhood watch) as vehicles for promoting and enhancing citizen satisfaction with and confidence in police, compliance and cooperation, and perceptions of procedural justice. We conclude, therefore, that it is the procedurally just features of the training, directive or organizational innovation that foster legitimacy rather than any specific type of strategy that leads to enhanced citizen perceptions of legitimacy. It is conceivable, therefore, that with some training or a clear directive, any type of police intervention could be used to facilitate legitimacy. From traffic stops to field contacts, we suggest that if police apply the principles of procedural justice during any of their encounters with citizens, they create opportunities to enhance perceptions of legitimacy.

We also find that police can enhance citizen perceptions and attitudes toward compliance, cooperation, satisfaction and confidence with police when there is a directive, training or organizational innovation involving at least one of the following ingredients of procedural justice: explicit efforts by the police to actively involve citizen participation during the encounter, clear efforts on behalf of the police to be neutral in their decision making during the encounter, police demonstrating dignity and respect toward the citizen during exchanges, or police working hard to communicate their trustworthy intentions. Even if just one of these components of procedural justice was a part of the intervention, our results suggest that the intervention is likely to increase citizen levels of compliance, cooperation and satisfaction. That is, a little bit of being nice during policecitizen interactions goes a long way.

Our analysis, therefore, suggests that the actual vehicle (or intervention mode) for police to engage with citizens is less important for fostering positive outcomes than the substantive content of the interaction itself (see also Bottoms & Tankebe, 2012). That is, the police have many and varied opportunities to positively influence citizen perceptions and there appears to be no downside for the police actively using the principles of procedural justice during any type of police intervention. Thus, building an understanding and capacity to engage with citizens in a procedurally just manner is clearly important for police across all types of engagement: from responding to calls for service, to taking calls over the phone, to how police engage with all sectors of society during problem solving and community policing activities.

Our review reveals that the outcome measures of satisfaction and confidence are particularly affected positively by police legitimacy enhancing activities. Satisfaction and confidence are well-established constructs in the research literature and arguably the most tangible of all of the direct outcomes in the legitimacy policing literature. We do, however, note that there are still variations in how scholars conceptualize and measure satisfaction and confidence. It is possible that the relative ease of measuring citizen satisfaction and/or confidence compared to measuring the more complex constructs around citizen perceptions of compliance, legitimacy, and procedural justice influenced our compelling finding that legitimacy policing interventions positively influence citizen satisfaction and confidence.

76 The Campbell Collaboration | www.campbellcollaboration.org

We also point out that of all the outcome measures tested in our meta-analysis, the only measure that did not reveal a statistically significant effect was the amorphous and often confounding measure of police legitimacy. This outcome was in the positive direction, but with the small number of studies in our review that used police legitimacy as an outcome measure, we had insufficient statistical power to detect a statistically significant outcome. Given that four of the seven evaluations that included a specific measure of police legitimacy emerged from the RISE conferencing experiment, we suggest that if it is to be included as a stand-alone outcome, and help advance our understanding of this outcome, further research is needed that includes more robust and consistent measurement of legitimacy,

Our systematic review also sought to examine the indirect outcomes of legitimacy policing. In our review, we searched for studies that measured the impact of legitimacy policing on crime, or revictimization/re-offending. We identified 4 eligible studies that captured revictimization and 11 eligible studies that measured reoffending. Conferencing, problem-oriented policing, community policing, reassurance policing and risk focused policing all featured in our meta-analysis examining reoffending and revicitimization. Our analysis shows a marginally significant impact in a favorable direction of these legitimacy policing interventions. Self-reported revictimization was significantly reduced as an outcome of legitimacy policing.

Overall, the main finding of our review is that for every single one of our outcome measures the effect of legitimacy policing is in a positive direction, and, for all but the legitimacy outcome, statistically significant. Notwithstanding the variability in the study interventions, the complexities around measurement and the differences in evaluation design, the story is overwhelmingly one that supports the police undertaking training, directives or organizational innovations across a wide range of police interventions to facilitate the adoption of practices that advance citizen perceptions of legitimacy. In practical terms, this means police adopting at least one of the principles of procedural justice as a component part of any type of police intervention, whether as part of routine police activity or as part of a defined program. Clearly training programs that facilitate police using, on a daily basis, the principles of procedural justice are likely to not only improve the willingness of citizens to cooperate and comply with directives, but are also likely to reduce reoffending when used in direct encounters with offenders, and reduce revictimization.

We note that there is a clear lack of randomized experiments in the international research literature that specifically seek to isolate and test the component parts of a legitimacy policing intervention, and then assess the impact of the intervention on both the direct and indirect outcomes we identified in this review. We suggest that future studies of legitimacy policing employ randomized controlled trials that isolate specific interventions and test different modes of delivery (such as hotspots policing, reassurance policing, directed patrols, conferencing) under different field conditions (such as during roadside encounters, in response to calls for service, during investigative interviews), and capture a range of different direct and indirect outcome measures. We further suggest that future evaluations of

77 The Campbell Collaboration | www.campbellcollaboration.org

legitimacy policing explore the manner in which qualitative components of legitimacy-enhancing interventions influence the indirect and direct outcomes as described in our review. We particularly believe it will be important to isolate the presence (or absence) of police using the mechanics of legitimacy-enhancing police approaches (i.e., complying with at least one procedural justice ingredient) from those interventions where the police adopt the normative values of legitimacy policing. That is, it is unclear, from our review, whether or not it is the quality of the delivery of the policecitizen encounter or the mere presence of procedural justice elements during the policecitizen encounter that leads citizens to perceive the police as legitimate. Despite these limitations, our review provides evidence that legitimacy policing is an important precursor for improving the capacity of policing to prevent and control crime.

78 The Campbell Collaboration | www.campbellcollaboration.org

6 Plans for Updating the Review

Professor Lorraine Mazerolle, Dr Sarah Bennett, Jacqueline Davis and Elise Sargeant will be responsible for updating this review, contingent on new funding being made available. We note that the peer review process of this report has identified a potential study (written in French) that may be eligible for inclusion in an updated review. This report is titled Evaluation de la victimisation et des perceptions subjectives de la criminalit dans diffrentes rgions vaudoises by Philippe Lamon, Sandrine Haymoz, and Martin Killias. With input from Professor Killias and our French national research colleague, Ms Brigitte Bouhours, we have made an initial assessment that the study may be eligible for inclusion. Final assessment of eligibility cannot be made until the study is fully translated5.

5 This study now has been fully translated and reviewed. It has been assessed as ineligible because the study did not meet the criteria for a legitimacy intervention.

79 The Campbell Collaboration | www.campbellcollaboration.org

7 Acknowledgments

We would like to acknowledge the financial support from the National Policing Improvement Agency (NPIA), George Mason University and the Campbell Collaboration which made this systematic search and review possible. We are particularly grateful to David Wilson and Kerrie Mengersen for providing valuable insights on content and analysis techniques. We are also appreciative of the comments received from the anonymous reviewers. Our thanks to Phyllis Schultz, Information Specialist at Rutgers University for her assistance in locating difficult to find publications. Gentry White provided statistical support. The authors would also like to thank Sarah-Ann Burger, Jegar Pitchforth and Jen Owens for their assistance in reviewing and coding literature.

80 The Campbell Collaboration | www.campbellcollaboration.org

8 Statement Concerning Conflict of Interest

Professor Lorraine Mazerolle, Dr Sarah Bennett, Jacqueline Davis, Elise Sargeant and Dr Matthew Manning have no financial interest in any of the existing research being conducted in this area of research (legitimacy in policing). The authors have no vested interest in the treatments that are the subject of this review or in the outcome of the review. The authors also do not have any incentives to represent findings in a biased manner and strive to avoid any potential conflict.

81 The Campbell Collaboration | www.campbellcollaboration.org

9 References

Barnes, G. C. (1999) Procedural justice in two contexts: Testing the fairness of diversionary conferences for intoxicated drivers. Unpublished PhD Dissertation: University of Maryland.

Bayley, D., & Mendelsohn, H. (1968). Minorities and the police: Confrontation in America. New York: Free Press.

Bellman, A. (1935). A police service rating scale. Journal of Criminal Law and Criminology, 26, 74114.

Bennett, S., Denning, R., Mazerolle, L., & Stocks, B. (2009). Procedural justice: A systematic literature search and technical report to the National Policing Improvement Agency. Brisbane, Australia: ARC Centre of Excellence in Policing and Security.

Berrien, J., & Winship, C. (2002). An umbrella of legitimacy: Boston's police department Ten Point Coalition Collaboration. In G. S. Katzmann (Ed.), Securing our children's future (pp. 200228). Washington, DC: Brookings Institution Press.

Bird, G. (1992). The times they are a changing: Policing multicultural Australia. In P. Moir & H. Eijkman (Eds), Policing Australia: Old Issues, New Perspective (p. 352-83). Melbourne: MacMillan

Borenstein, M., Hedges, L. V., Higgins, J., & Rothstein, H. R. (2009). Introduction to meta-analysis. West Sussex, UK: Wiley.

Bottoms, A., & Tankebe, J. (2012). Beyond procedural justice: A dialogic approach to legitimacy in criminal justice. The Journal of Criminal Law and Criminology, 102(1), 119 170.

Bowling, B., & Philips, C. (2002). Racism, crime and justice. Harlow: Pearson Education.

Bradford, B., & Jackson, J. (2008). Contact and confidence: Findings from the 2005/06 Metropolitan Police Public Attitude Survey. London School of Economics: Working Paper.

Bradford, B., & Jackson, J. (2010). Different things to different people? The meaning and measurement of trust and confidence in policing across diverse social groups in London. Available at:http://ssrn.com/abstract=1628546

Web End = http://ssrn.com/abstract=1628546 .

82 The Campbell Collaboration | www.campbellcollaboration.org

Braga, A. A. (2007). The effects of hot spots policing on crime. The Campbell Collaboration Library of Systematic Reviews. doi: 10.4073/csr.2007.1

Brandl, S. G., Frank, J., Worden, R. E., & Bynum, T. S. (1999). Global and specific attitudes toward the police: Disentangling the relationship. Justice Quarterly, 11, 119134.

Decker, S. H. (1981). Citizen attitudes toward the police: A review of past findings and suggestions for future policy. Journal of Police Science and Administration, 9, 8087.

Deeks J., & Higgins J. (2005). Standard statistical algorithms in Cochrane reviews. Version 5. Technical report. The Cochrane Collaboration.

Duval, S., & Tweedie, R. (2004). Trim and fill: A simple funnel plot-based method of testing and adjusting for publication bias in meta-analysis. Biometrics, 56(2), 455463.

Egger, M., Smith, G. D., Schneider, M., & Minder, C. (1997). Bias in meta-analysis detected by a simple, graphical test. British Medical Journal, 315, 629634.

Elliott, I., Thomas, S. D. M., & Ogloff, J. R. P. (2011). Procedural justice in contacts with the police: the perspective of victims of crime. Police Practice and Research, 1-13.

Fagan, J., & Tyler, T. R. (2005). Legal socialisation of children and adolescents. Social Justice Research, 18, 217242.

Goodman-Delahunty, J. (2010). Four ingredients: New recipes for procedural justice in Australian policing. Policing, 4(4), 403410.

Hawdon, J. (2008). Legitimacy, trust, social capital, and policing styles: A theoretical statement. Police Quarterly, 11(2), 182201.

Hawdon, J. E., Ryan, J., & Griffin, S. P. (2003). Policing tactics and perceptions of police legitimacy. Police Quarterly, 6(4), 469491.

Hedges, L., & Olkin, I. (1985). Statistical Methods for Meta-analysis. Toronto: Academic Press.

Higgins, J. P. T., & Thompson, S. G. (2002). Quantifying heterogeneity in a meta-analysis. Statistics in Medicine, 21(11), 15391558.

Hinds, L., & Murphy, K. (2007). Public satisfaction with police: Using procedural justice to improve police legitimacy. Australian and New Zealand Journal of Criminology, 40(1), 2742.

Home Office (2011). Crime in England and Wales: Quarterly update to September 2010. London: Author.

83 The Campbell Collaboration | www.campbellcollaboration.org

Hurst, Y. G., & Frank, J. (2000). How kids view cops: the nature of juvenile attitudes towards the police. Journal of Criminal Justice, 28, 189202.

Jackson, J., & Bradford, B. (2009). Crime, policing and social order: on the expressive nature of public confidence in policing. British Journal of Sociology, 60 (3), 493521.

Jackson, J., Bradford, B., Hough, M., & Murray, K. H. (2011). Policing by consent and public compliance with the law: Notes on legal legitimacy and cynicism. In A. Crawford & A. Hucklesby, (Eds). Legitimacy and Compliance in Criminal Justice, London: Routledge.

Jackson, J., & Sunshine, J. (2007). Public confidence in policing: A neo-Durkheimian perspective. British Journal of Criminology, 47(2), 214233.

Jonathon-Zamir, T., & Weisburd, D. (2009). Does police performance increase in importance for the public during times of security threats, and do evaluations of procedural justice decline in importance? Findings from a quasi-experimental study of antecedents of police legitimacy in Israel. Israel: Hebrew University.

Kelling, G. L., & Moore, M. H. (1988). The evolving strategy of policing (Perspectives on Policing Series 4). Washington,DC: National Institute of Justice.

Leiber, M. J., Nalla, M. K., & Farnworth, M. (1998). Explaining juvenile attitudes toward the police. Justice Quarterly, 15, 15174.

Lipsey, M. W., & Wilson, D. B. (2001). Practical meta-analysis (Vol. 49). Thousand Oaks, CA: SAGE Publications.

MacCoun, R. J. (2005). Voice, control and belonging: The double-edged sword of procedural fairness. Annual Review of Law and Social Science, 1, 171201.

Maher, L., & Dixon, D. (1999). Policing and public health: Law enforcement and harm minimization in a street-level drug market. British Journal of Criminology, 39(4), 488512.

Mastrofski, S. D. (2009). Using systematic social observation to advance experimental criminology. Presentation. Retrieved from http://gemini.gmu.edu/cebcp/JerryLeePresentations/Mastrofski.pdf

Mastrofski, S. D., Snipes, J. B., & Supina, A. E. (1996). Compliance on demand: The public's response to specific police requests. Journal Of Research In Crime And Delinquency, 33(3), 269305.

Mazerolle, L., & Ransley, J. (2005). Third Party Policing. Cambridge: Cambridge University Press.

McCluskey, J. D. (2003). Police requests for compliance: Coercive and procedurally just tactics. New York: LFB Scholarly Publishing LLC.

84 The Campbell Collaboration | www.campbellcollaboration.org

McCluskey, J. D., Mastrofski, S. D., & Parks, R. B. (1999). To acquiesce or rebel: Predicting citizen compliance with police requests. Police Quarterly, 2(4), 389416.

Murphy, K., & Cherney, A. (2011). Fostering cooperation with the police: How do ethnic minorities in Australia respond to procedural justice-based policing? Australian & New Zealand Journal of Criminology, 44(2), 235257

Murphy, K., & Cherney, A. (2012). Understanding cooperation with police in a diverse society. British Journal of Criminology, 52, 181201

Murphy, K., Hinds, L., & Fleming, J. (2008). Encouraging public cooperation and support for police. Policing and Society, 18(2), 136155.

Murphy, K., Tyler, T. R., & Curtis, A. (2009). Nurturing regulatory compliance: Is procedural justice effective when people question the legitimacy of the law? Regulation & Governance, 3(1), 126.

Murphy, K, & Gaylor, A. (2010). Policing youth: Can procedural justice nurture youth cooperation with police? Alfred Deakin Research Institute: Deakin University

Parratt, S. D. (1938). A scale to measure effectiveness of police functioning. Journal of the American Institute of Criminal Law and Criminology, 27, 895905.

Paternoster, R., Brame, R., Bachman, R., & Sherman, L. W. (1997). Do fair procedures matter? Law and Society Review, 31, 163204.

Pickering, S., McCulloch, J., & Wright-Neville, D. (2008). Counter-terrorism policing: Community, cohesion and security. New York: Springer.

Piquero, A. R., Fagan, J., Mulvey, E. P., Steinburg, L., & Odgers, C. (2005). Criminology: Developmental trajectories of legal socialisation among serious adolescent offenders. The Journal of Criminal Law and Criminology, 96, 267298.

Reiner, R. (1985). Retrospect on the riots. New Society, 74(1191), 148150.

Reiner, R. (2000). The politics of the police (3rd ed.). Oxford: Oxford University Press.

Reisig, M. D., Bratton, J., & Gertz, M. G. (2007). The construct validity and refinement of process-based policing measures. Criminal Justice and Behavior, 34(8), 10051028.

Reisig, M. D., & Chandek, M. S. (2001). The effects of expectancy disconfirmation on outcome satisfaction in police-citizen encounters. Policing: An International Journal of Police Strategies and Management, 24, 8799.

Reiss, A. J. (1971). The police and the public. New Haven, CT: Yale University Press.

85 The Campbell Collaboration | www.campbellcollaboration.org

Roehl, J. A. (1988). Measuring perceptions of procedural justice. Washington, DC: George Washington University.

Rothstein, H. R., Sutton, A. J., & Borenstein, M. (2005). Publication bias in meta-analysis: Prevention, assessment, and adjustments. West Sussex, United Kingdom: Wiley.

Sivasubramaniam, D., and Goodman-Delahunty, J. (2008), Ethnicity and trust: Perceptions of police bias. International Journal of Police Science and Management, 10, 38840.

Sunshine, J., & Tyler, T. R. (2003). The role of procedural justice for legitimacy in shaping public support for policing. Law and Society Review, 37(3), 513548.

Sutton, A. J., Duval, S. J., Tweedie, R. L., Abrams, K. R., & Jones, D. R. (2000). Empirical assessment of effect of publication bias on meta-analyses. British Medical Journal, 320, 15741577.

Tankebe, J. (2009). Public cooperation with the police in Ghana: Does procedural fairness matter? Criminology, 47, 126593.

Thibaut, J., & Walker, L. (1975). Procedural justice: A psychological analysis. Hillsdale, NJ: Erlnaum.

Tyler, T. R. (1988). What is procedural justice? Criteria used by citizens to assess the fairness of legal procedures. Law and Society Review, 22, 103135.

Tyler, T. R. (1990). Why people obey the law. New Haven, CT: Yale University Press.

Tyler, T. R. (1994). Governing amid diversity: Can fair decision-making procedures bridge competing public interests and values? Law and Society Review, 28, 701722.

Tyler, T. R. (1997). The psychology of legitimacy: A relational perspective on voluntary deference to authorities. Personality and Social Psychology Review, 1(4), 323345.

Tyler, T. R. (1998). Public mistrust of the law: A political perspective. University of Cincinnati Law Review, 66, 847876.

Tyler, T. R. (2001). Public trust and confidence in legal authorities: What do majority and minority group members want from legal authorities? Behavioral Sciences and the Law, 19, 215235.

Tyler, T. R. (2003). Procedural justice, legitimacy, and the effective rule of law. In M. Tonry (ed.), Crime and Justice: A Review of Research (Vol. 30, pp. 431505). Chicago: University of Chicago Press.

Tyler, T. R. (2004). Enhancing police legitimacy. The ANNALS of the American Academy of Political and Social Science, 593(1), 8499.

86 The Campbell Collaboration | www.campbellcollaboration.org

Tyler, T. R. (2006). Psychological perspectives on legitimacy and legitimation. Annual Review of Psychology, 57, 375400.

Tyler, T. R. (2008). Psychology and institutional design. Review of Law & Economics, 4(3), 801887.

Tyler, T. R., & Fagan, J. (2008). Legitimacy and cooperation: Why do people help the police fight crime in their communities? Ohio State Journal of Criminal Law, 6, 231275.

Tyler, T. R., & Huo, Y.J. (2002). Trust in the law. New York: Russell Sage.

Tyler, T. R., & Lind, E. A. (1986). Procedural processes and legal institutions. Paper presented at the International Conference on Social Justice in Human Relations,Leiden.

Tyler, T. R., Sherman, L. W., Strang, H., Barnes, G. C., & Woods, D. J. (2007). Reintegrative shaming, procedural justice, and recidivism: The engagement of offenders' psychological mechanisms in the Canberra RISE drinking-and-driving experiment. Law and Society Review, 41(3), 553585.

Tyler, T. R., & Wakslak, C.J. (2004). Profiling and police legitimacy: Procedural justice, attributions of motive, and acceptance of police authority. Criminology, 42(2), 253281.

Walker, D., Richarson, R. J., Denyer, T., Williams, O., & McGaughey, S. (1972). Contact and support: An empirical assessment of public attitudes toward the police and the courts. North Carolina Law Review, 51, 4379.

Weisburd, D., & Braga, A. A. (Eds.) (2006). Policing innovation: Contrasting perspectives. Cambridge: Cambridge University Press.

Weisburd, D., & Eck, J. E. (2004). What can police do to reduce crime, disorder, and fear? The ANNALS of the American Academy of Political and Social Science, 593, 4265.

Weisburd, D., Mastrofski, S. D., & Telep, C. W. (2009). The Alexandria, Virginia Legitimacy Policing Experiment. Unpublished manuscript. Administration of Justice Department, George Mason University.

Wells, L. E. (2007).Type of contact and evaluations of police officers: The effects of procedural justice across three types of policecitizen contacts. Journal of Criminal Justice, 35(6), 612621.

Winfree, T., & Griffiths, C. (1971). Adolescent attitudes toward the police. In T. Ferdinand (Ed.), Juvenile Delinquency: Little Brother Grows Up. Beverly Hills, CA: Sage.

87 The Campbell Collaboration | www.campbellcollaboration.org

Appendix 1: Inventory of police-led legitimacy interventions

Overall, 163 studies described or evaluated some kind of policing intervention (see Table A1). Sixty-four of these studies described community policing strategies. Community policing covers a wide range of strategies, but generally includes an emphasis on the community and police sharing responsibility for crime and disorder, and working together to prevent and reduce crime and disorder. The strategies reported in our sample of studies included Neighborhood Watch programs, beat policing, reassurance policing, and contact patrols, as well as community policing broadly defined.

Twenty-nine of the studies described problem-oriented policing strategies that used various place-based and people-based strategies to address a particular crime or disorder problem. Problem-oriented policing strategies included community alarm systems, crime prevention through environmental design, risk-focused policing, special problem-oriented task forces, and grants awarded to police departments to address particular problems such as gun crimes or violence against women.

Fourteen studies described the formation of specialized enforcement teams; that is, police units with a specific focus or directive. Specialized teams included organized crime units, drug teams, domestic violence units, victim assistance units, and crisis intervention teams that worked with paramedics to coordinate responses to emergencies.

Nine studies described restorative justice conferencing interventions, usually implemented as an alternative to court processing and detention for the processing of minor crimes or juveniles. These conferences were usually administered by a police officer and involved the victim and offender discussing the crime and agreeing on reparations.

Eight studies described interventions that used inter-agency cooperation as a key part of the intervention. These interventions included collaboration between police and social service agencies to respond to domestic violence, collaborations between police and schools to reduce truancy, and other, broader multi-agency strategies.

Eight studies described special police training programs such as life skills training, diversity training, crisis intervention training, victim-focused training, community policing training, and basic training on how to recognize and deal with people with mental health problems. Six studies described targeted enforcement strategies including enhanced forensic science

88 The Campbell Collaboration | www.campbellcollaboration.org

techniques, zero tolerance policing, undercover checks for compliance with alcohol sales laws, and broad enforcement strategies.

Six studies described organizational innovations involving changing the structure of the police force. One example is the creation of smaller geographically-based command units, within which officers reported to their command unit representative. Other organizational innovations included the implementation of crime recording standards or police oversight procedures.

Five studies described alternative procedures for the resolution of complaints against police, including informal complaints resolution processes, restorative justice-based approaches, and procedural justice-based approaches. Five studies described specific directives to police officers. Most of these directives were mandatory arrest for domestic violence offences.

Three studies described school-based interventions including school resource officers: police officers based at schools to foster ties to students and enhance enforcement capabilities within the school. This category also included an intervention that created opportunities for students to interact with police in an informal setting.

Three studies described citizen education initiatives. One was a citizen police academy, in which citizens attended a short course informing them of the duties of police officers and giving them some basic training in day-to-day aspects of police work. Two studies described a program that trained young people to resist joining gangs.

89 The Campbell Collaboration | www.campbellcollaboration.org

Table A1. Inventory of intervention strategies

Study (First author/publication year) Intervention name Intervention category

Buren 2003 Citizen oversight of police complaints Alternative complaint resolution

Holland 1996 Informal police complaints resolution Alternative complaint resolution

Kertstetter 1994 Procedural justice complaints resolution Alternative complaint resolution

Kreisel 1998 Alternative complaint resolution

Young 2005 Restorative justice complaints resolution Alternative complaint resolution

Brewster 2005 Citizen police academies Citizen education

Sellers 1998 Gang Resistance Education Training Citizen education

Winfree 1999 Gang Resistance Education Training Citizen education

Beedle 1984 Citizen contact patrols Community policing

Bennett 1990 Citizen contact patrols Community policing

Bond 1997 Beat policing Community policing

Brensilber 1999 Weed & Seed Woburn Community policing

Brimley 2005 Comprehensive Indian Resources for Community and Law Enforcement

Community policing

Brownlee 1998 Dedicated Police Teams Community policing

Bynum 2000 Weed & Seed Grand Rapids Community policing

Child & Family Policy Center 1999 Weed & Seed Des Moines Community policing

Challinger 2004 Crime Stoppers Community policing

Cook 1994 Community policing

Cox 1999 Weed & Seed New Britain Community policing

Crawford 1994 Community policing Community policing

Crawford 1995 Community policing

Dai 2007 Community policing Community policing

Driscoll 2003 Weed & Seed East Wheeling Community policing

Dunworth 1999a Weed & Seed Akron Community policing

Dunworth 1999b Weed & Seed Hartford Community policing

Dunworth 1999c Weed & Seed Las Vegas Community policing

90 The Campbell Collaboration | www.campbellcollaboration.org

Study (First author/publication year) Intervention name Intervention category

Dunworth 1999d Weed & Seed Manatee & Sarasota Community policing

Dunworth 1999e Weed & Seed Pittsburgh Community policing

Dunworth 1999f Weed & Seed Salt Lake City Community policing

Dunworth 1999g Weed & Seed Seattle Community policing

Dunworth 1999h Weed & Seed Shreveport Community policing

Eckert 2009 Community policing Community policing

Grossman 2004 Community policing

Hall 1987 Neighborhood Watch Community policing

Hallas 2004 Weed & Seed Youngstown Community policing

Harris 1998 Weed & Seed Wilmington Community policing

Harris 2005 Weed & Seed Wilmington Community policing

Henig 1984 Neighborhood Watch Community policing

Holmberg 2005 Proximity policing Community policing

Johnson 1997 Community policing and aggressive enforcement Community policing

Josi 2000 Weed & Seed Savannah Community policing

Kerley 2000 Community policing Community policing

Kessler 1997 Community policing Community policing

Lurigio 1993 Citizen police cooperation Community policing

McDevitt 2008 Community Oriented Policing Service Community policing

Mesko 2005 Community policing Community policing

Moon 2005 Mini police stations Community policing

Morrison 2000 Weed & Seed Trumbull & La Mesa Community policing

Murphy 2008 Community policing Community policing

Nth Cent. Florida Health Planning Council 1999

Weed & Seed Virginia Community policing

Panetta 2000 Community policing Community policing

Pate 1989 Community policing Baltimore Community policing

Pickering 2007 Community policing Community policing

Pierce 1997 Safe Haven Madison Community policing

91 The Campbell Collaboration | www.campbellcollaboration.org

Weed & Seed Ocala Community policing

Virginia Dept Criminal Justice Services 2000

Study (First author/publication year) Intervention name Intervention category

Ren 2005 Community policing Community policing

Richter Community policing

Robinson 2000 Community policing

Rohe 2001 Community policing

Rosenthal 1997 Weed & Seed Philadelphia Community policing

Roth 2000 Community Oriented Policing Service Community policing

Ryan 2007 Community policing

Sagar 2005 Street Watch Community policing

Shaler 2004 Weed & Seed Lincoln County Community policing

Sharp 2008 Civilian policing Community policing

Singer 2004 Local Managemant of Community Safety Program (Reassurance policing)

Community policing

Skogan 2004 Community policing Community policing

Stokes 2006 Commercial Safety Services Community policing

Taylor 2005 Police shopfronts Community policing

Tuffin 2006 Reassurance policing Community policing

Virta 2002 Community policing

Zevitz 1997 Weed & Seed Milwaukee Community policing

Angrist 2006 Minneapolis Domestic Violence Experiment Directive

Barbrey 2003 Team-based policing Directive

Feder 2000 Directive

Guzik 2006 Presumptive arrest for domestic violence Directive

Lawrenz 1988 Presumptive arrest for domestic violence Directive

Willis 2007 COMPSTAT Intelligence-led policing

Davis 1997 Domestic violence follow-up calls Multi-agency intervention

Hanewicz 1982 Social Justice Team project Multi-agency intervention

Matthews 1993 Multi-agency intervention

Moffett 1996 Corpus Christi Truancy Reduction Impact

Program

92 The Campbell Collaboration | www.campbellcollaboration.org

Multi-agency intervention

Study (First author/publication year) Intervention name Intervention category

Penfold 2004 Multi-agency intervention

Shepherd 1998 Multi-agency intervention

Shernock 2004 Multi-agency intervention

White 2001 Truant Recovery Program Multi-agency intervention

Berry 1992 Multiple

Gladstone 1980 Multiple

Clarke 2003 Organizational innovation

Loveday 2007 Basic Command Units Organizational innovation

Perez 1992 Police review systems Organizational innovation

Simmons 2003 National Crime Recording Standard Organizational innovation

Sun 2003 Field training officers Organizational innovation

Terpstra 2009 Managerialization of police Organizational innovation

Arthur 1980 Life skills training program Police training

Berringer 2004 Police training

Brennan 1994 Cleartalk Police training

Harrington 2002 Diversity training Police training

Kaiser 1995 CAPS training Police training

Meerbaum 1981 Crisis intervention training Police training

Pearce 1983 Crisis intervention training Police training

Rosenbaum 1987 Victim-focused training Police training

Applegate 2004 Problem oriented policing

Burt 2001 Violence Against Women grants Problem oriented policing

Carroll 2002 Problem oriented policing

Criminal Justice Commission 1998 Problem-solving policing Problem oriented

93 The Campbell Collaboration | www.campbellcollaboration.org

Study (First author/publication year) Intervention name Intervention category

policing

Dunham 1995 Metro-Dade Police & Street-Level Drug Enforcement

Problem oriented policing

Dunworth 2000 Youth Firearms Violence Initiative Problem oriented policing

Goldstein 1990 Problem oriented policing

Hakkert 2001 Concentrated traffic enforcement Problem oriented policing

Hartstone 2003 Problem-solving policing Problem oriented policing

Honess 1993 Vehicle Watch Problem oriented policing

Jones 1995 Traffic safety enforcement Problem oriented policing

Keaton 2009 Byrne Collaborative Problem oriented policing

Kent 2000 Problem-solving policing Problem oriented policing

Mazerolle 1997 Problem oriented policing

McGarrell 2004 Indianapolis Violence Reduction Partnership Problem oriented policing

Part 2006 Domestic violence intervention Problem oriented policing

Kennedy 2001 Operation Ceasefire (Boston Gun Project) Problem oriented policing

Rodabough 2002 Juvenile curfew Problem oriented policing

Roehl 1984 Urban Crime Prevention Program Problem oriented policing

Crime Prevention Unit, SA Attorney-General 2002

South Australian Office of Crime Statistics & Research 2005

94 The Campbell Collaboration | www.campbellcollaboration.org

Residential Break & Enter Project Problem oriented policing

Fake ID Project Problem oriented policing

Sharkey 1985 Selective speed enforcement Problem oriented policing

Smith 2000 Drug Task Force Problem oriented policing

Tita 2003 Boston Gun Project replication Problem oriented policing

Study (First author/publication year) Intervention name Intervention category

Urban 2005 Operation Night Watch Problem oriented policing

Walker 2001 Community alarm system Problem oriented policing

Wallis 1980 Crime prevention through environmental design Problem oriented policing

Weisburd 2008 Risk-focused policing Problem oriented policing

Weisel 1990 Problem oriented policing

Hoyle 2002 Restorative cautioning Restorative justice

Koss 2004 Restorative justice conferencing Restorative justice

McCold 1998 Restorative justice conferencing Restorative justice

McCold 2003 Restorative justice conferencing Restorative justice

Shapland 2007 Justice Research Consortium Restorative justice

Shapland 2008 Justice Research Consortium Restorative justice

Sherman 1998 Restorative justice conferencing Restorative justice

Swanson 2007 Restorative justice

Volpe 2003 Police-led mediation Restorative justice

Dogutas 2007 School resource officers School based intervention

Hinds 2009 Informal contact School based intervention

Murray 2003 School resource officers School based intervention

Battle 1999 Organized Crime Drug Control Task Force Specialized team

Bower 2001 Crisis intervention teams Specialized team

Butler 1983 Specialized team

Corcoran 2005 Victim assistance crisis teams Specialized team

Department of Justice, Victoria 2008 Enforcement Operations Specialized team

Friday 2006 Specialised domestic violence unit Specialized team

Henderson 2004 Automatic number plate recognition intercept teams

95 The Campbell Collaboration | www.campbellcollaboration.org

Specialized team

Hovell 2006 Family Violence Response Team Specialized team

Katz 1997 Police gang unit Specialized team

Study (First author/publication year) Intervention name Intervention category

Lardner 1992 Child Protection Unit Specialized team

Rubenser 2000 Nuisance Task Force Specialized team

Schmitz 1998 Homicide & Violent Crime Task Force Specialized team

Tennant 1986 Mental health unit Specialized team

Turnbull 1996 Get It While You Can Specialized team

Burrows 2005 Enhanced forensic science techniques Targeted enforcement

Chandler 2001 Undercover compliance checks Targeted enforcement

Goldkamp 2008 Operation Sunrise Targeted enforcement

Korander 2005 Zero tolerance Targeted enforcement

Novak 1999 Targeted enforcement

Rigotti 1997 Targeted enforcement

96 The Campbell Collaboration | www.campbellcollaboration.org

Appendix 2: Technical Appendix

This section describes the data reporting and operational definitions of constructs for each study included in the meta-analysis, along with the transformations undertaken to compute the relevant effect sizes.

Legitimacy

Study Operational definition Reported statistics Effect size

calculation

Eckert 2009 9 item scale 1. In the last year, have you reported a crime to the police?

2. In the last year, have you contacted the police about something suspicious?

3. In the last year, have you reported any other problem to the police?

4. The police do a good job of preventing crime

5. The police do a good job of keeping order on the streets and sidewalks

6. The police are polite when dealing with people

7. The police are helpful when dealing with people

8. The police are helpful when dealing with victims of crime

9. The police are fair when dealing with people

t (d.f.), p

Hinds 2009 How willing would you be to assist police if asked?

Standardized regression coefficient and its p value from a structural equation model

We used the reported standardized regression coefficient and p value to calculate a t value. We then used this t value to calculate d, and convert d to a log OR, using formulae provided by Lipsey and Wilson (2011). This procedure may have produced an upwardly

97 The Campbell Collaboration | www.campbellcollaboration.org

Study Operational definition Reported statistics Effect size

calculation

biased effect size, as the pooled standard deviation for the t value of a regression coefficient has been adjusted for other variables in the regression, while the pooled standard deviation for the t used to calculate a mean difference has not been adjusted in this way. Therefore, we have investigated the sensitivity of the results to the inclusion of this study. We did not correct for covariates.

Murphy 2008 1. I have confidence in the police

2. I think the police perform their job professionally

3. Police do their job well 4. I have great respect for the police

Means and standard deviations for each item

Sherman 1998 1. Had increased respect for the justice system

2. Had increased respect for the law

3. Had increased respect for the police

Percent positive responses for each item

Reconstructed a 2x2 contingency table and entered contingency table frequencies into CMA

98 The Campbell Collaboration | www.campbellcollaboration.org

Procedural Justice

Study Operational definition Reported statistics Effect size

calculation

Murphy 2008

1. Police are concerned about respecting a citizens individual rights

2. Police treat people as if they can be trusted to do the right thing

3. Police treat people as if they only do the right thing when forced to (reversed)

Means and standard deviations for each item, number of people measured both pre and post intervention

Direct via CMA

Panetta 2000

Community Policing Leadership scale

Percent positive response for entire scale

Reconstructed a 2x2 contingency table and entered contingency table frequencies into CMA

Sherman 1998

1. Understood what was going on at treatment

2. Understood what my rights were

1. Treatment was fair overall 2. Treatment respected my rights 3. Offenders with the same offence were treated the same

4. Police were fair leading up to treatment

5. Police were fair at treatment 6. Police in Canberra enforce the law fairly

1. If the police had facts wrong, able to correct

2. If police treated me unfairly, able to complain

3. Felt too intimidated at treatment to speak (reversed)

1. Felt I had some control over the outcome

2. Had an opportunity to express my views

3. Had enough control over the way things were run

4. Treatment took account of what I said

5. Felt pushed around by others with power (reversed)

6. Felt pushed into things I didnt agree with (reversed)

1. Felt treated no better or worse than others

2. All sides had a fair chance to present views

3. Felt disadvantaged by age, income, sex, etc. (reversed)

1. Felt I could trust the police during treatment

Percent positive responses for each item, number in treatment and control groups

Reconstructed a 2x2 contingency table and entered contingency table frequencies into CMA

99 The Campbell Collaboration | www.campbellcollaboration.org

Study Operational definition Reported statistics Effect size

calculation

2. I was made to confess to things I did not do (reversed) 1. Police were polite to me at treatment

2. I was treated with respect at treatment

3. Police were rude when I was apprehended (reversed)

Tuffin 2006

1. How much effort do the police in your local area put into finding out what local people think?

2. How willing are the police to listen and respond to peoples views?

3. How effective are the police in your local area at working with the community?

Percent positive response before and after intervention in treatment and control groups

Created an average score across the three measures. Reconstructed 2x2 contingency table. Created a post-intervention log OR corrected for pre-intervention log OR (corrected log OR = post intervention log OR pre intervention log OR). Calculated variance of corrected log OR as sum of pre intervention log OR and post intervention log OR (most conservative estimate).

Weisburd 2008

A seven-item summated scale that captures self-reported respect, trust, fairness, and honesty of Redlands police officers

Standardized regression coefficients and standard errors

We used the reported standardized regression coefficient and standard error to calculate a t value. We then used this t value to calculate d, and convert d to a log OR, using formulae provided by Card (2011). This procedure may have produced an upwardly biased effect size, as the pooled standard deviation for the t value of a regression coefficient has been adjusted for other variables in the regression, while the pooled standard deviation for the t used to calculate a mean difference has not been adjusted in this way. Therefore, we have investigated the sensitivity of the results to the inclusion of this study. We did not correct

100 The Campbell Collaboration | www.campbellcollaboration.org

Study Operational definition Reported statistics Effect size

calculation

for covariates.

Shapland 2008

Overall, how fair did you feel the criminal justice process was?

Item response percentages and number in each group

Collapsed victim and offender perspectives, collapsed five point scale to dichotomous outcome, calculated number in each cell from percentages, constructed a contingency table

101 The Campbell Collaboration | www.campbellcollaboration.org

Compliance/Cooperation

Study Operational

definition

Reportedstatistics Effect size calculation

Murphy 2008 (Compliance)

1. I would never call the police under any circumstances (Reversed)

2. I would call the police if I witnessed something I believed should involve them

Means and standard deviations for each item, number of people measured both pre and post intervention

Direct via CMA

Dai 2007 (Compliance)

1. Citizen gave no indication one way or the other (Reversed)

2. Citizen refused to do what the police requested (Reversed) Note: Researcher observations of actual behavior, not self-report

Standardized regression coefficient and its standard error from a logistic regression with noncompliance as the outcome

Used the standardized regression coefficient as a log odds ratio and its standard error as the standard error of the log odds ratio. Since the outcome reported was noncompliance but we were interested in compliance, we reversed the sign on the regression coefficient

Sherman 1998 (Compliance)

1. Treatment will help prevent reoffending

2. Treatment will help me obey the law

Percent positive responses for each item

Calculated numbers from percentages and constructed contingency table

Bond 1997 (Cooperation)

Post intervention percent positive response in treatment and comparison groups, number in treatment and comparison groups

Constructed a 2x2 contingency table and entered contingency table frequencies into CMA

Robinson 2000 (Cooperation)

Observed victim participation in prosecution of offender

Chi-square statistic, number in treatment and control group, and direction of effect

Entered directly into CMA

102 The Campbell Collaboration | www.campbellcollaboration.org

Satisfaction/Confidence

Study Operational

definition Reported statistics

Effect size calculation

Means and standard deviations for each item, number of people measured both pre and post intervention

Direct via CMA

Murphy 2008 (Satisfaction)

1. How good a job are the police doing in dealing with the problems that really concern people in your suburb?

2. How good a job are the police doing in your suburb in working together with residents to solve local problems?

3. How good a job do you think police are doing to prevent crime in your suburb?

In the primary publication, only standardized regression coefficients for variables including the dummy variable of volunteer status were reported. However, supplementary data from an alternative publication by the authors reported means and standard deviations for the volunteer and non-volunteer samples (Ren, Zhao, Lovrich, & Gaffney, 2006).

Direct via CMA

Ren 2005 (Confidence)

1. The police officers are usually fair

2. The police officers are usually courteous

3. The police officers are usually honest

4. The police officers are usually not intimidating

5. The police officers work with citizens together in solving problems

6. The police officers treat all citizens equally in general

7. The police officers show concern when asked questions

Singer 2004 (Satisfaction)

Ten items measuring satisfaction with overall police performance and particular aspects of performance

Percentage of respondents answering Excellent or Good for each item

Reconstructed a 2x2 contingency table and entered contingency table frequencies into CMA

Skogan 2004 (Satisfaction)

1. In general, how polite are the police when dealing with people in your neighborhood?

2. When dealing with peoples problems in your neighborhood, are the police generally about their

Percent positive answers for each item

Reconstructed a 2x2 contingency table and entered contingency table frequencies into CMA

103 The Campbell Collaboration | www.campbellcollaboration.org

Study Operational

definition Reported statistics

Effect size calculation

problems? 3. In general, how helpful are the police when dealing with people in your neighborhood? In general, how fair are the police when dealing with people in your neighborhood?

1. How responsive are the police in your neighborhood to community concerns?

2. How good a job are the police doing in dealing with the problems that really concern people in your neighborhood?

3. How good a job are the police doing in working together with residents in your neighborhood to solve local problems?

1. How good a job do you think the police in your neighborhood are doing in helping people out after they have been victims of crime?

2. How good a job do you think they are doing to prevent crime in your neighborhood?

3. How good a job are the police in your neighborhood doing in keeping order on the streets and sidewalks?

Sherman 1998 (Satisfaction)

1. Satisfied with the way case was dealt with 2. Satisfied that case was dealt with the way it was, compared to the other treatment

Percent positive responses for each item

Reconstructed a 2x2 contingency table and entered contingency table frequencies into CMA

Hall 1987 (Effectiveness)

The Santa Ana Police Department is effective

Raw frequencies Dichotomized responses into positive and neutral/negative responses. Reconstructed a 2x2 contingency table and

104 The Campbell Collaboration | www.campbellcollaboration.org

Study Operational

definition Reported statistics

Effect size calculation

entered contingency table frequencies into CMA

Tuffin 2006 (Confidence)

Taking everything into account, how good a job do you think the police in your local area are doing?

Reconstructed 2x2 contingency table. Created a post-intervention log OR corrected for pre-intervention log OR (corrected log OR = post intervention log OR pre intervention log OR). Calculated variance of corrected log OR as sum of pre intervention log OR and post intervention log OR (most conservative estimate).

Post intervention percent positive response in treatment and comparison groups, number in treatment and comparison groups

Reconstructed a 2x2 contingency table and entered contingency table frequencies into CMA

Bond 1997 (Satisfaction)

Zevitz 1997 (Satisfaction)

The police department has done a good job in making me feel safe in this area

Percentage pre and post in treatment and control groups and numbers in each group

Calculated numbers from percentages and reconstructed contingency table

Young 2005 (Satisfaction)

Complaint overall satisfaction with complaints process

Item response percentages and number in each group

Collapsed and dichotomized satisfaction measures, calculated numbers from percentages and constructed contingency table

Shapland 2008 (Satisfaction)

How satisfied are you with what the criminal justice system did about this offence

Item response percentages and number in each group

Collapsed victims and offenders measures, dichotomized satisfaction measure, calculated numbers from percentages and constructed contingency table

McGarrell 2004 (Effectiveness)

Working together is effective

Percent agree in treatment and control group and

Constructed contingency table

105 The Campbell Collaboration | www.campbellcollaboration.org

Study Operational

definition Reported statistics

Effect size calculation

number in each group

Holland 1996 (Satisfaction)

Complainants satisfaction with the way the complaint investigation or informal resolution was handled

Item response percentages and number in each group

Dichotomized measures by adding very satisfied to satisfied, and very unsatisfied to unsatisfied. Constructed contingency table and entered contingency table frequencies into CMA

Collapsed and dichotomized satisfaction measures, created contingency table. Entered into CMA as a pre-post contingency table with the pre-post correlation = 0 (not reported and zero is most conservative estimate)

Dunworth 1999 a-h (Satisfaction)

1. In general, how good a job are the police doing to keep order on the streets and sidewalks in this neighborhood these days?

2. How good a job are the police doing in controlling the street sale and use of illegal drugs in the neighborhood these days?

Kertstetter 1994 (Confidence)

Does the police review panel give you a lot more,a little more, or less confidence that your police department will thoroughly and impartially investigate citizen complaints about its officers?

Item response percentages and number of respondents at multiple time points

Collapsed and dichotomized satisfaction measures, created contingency table. Entered into CMA as a pre-post contingency table with the pre-post correlation = 0 (not reported and zero is most conservative estimate)

106 The Campbell Collaboration | www.campbellcollaboration.org

Reoffending

Study Operational definition Reported statistics Effect size calculation

Shapland 2008

Frequency of reoffending within a two-year follow-up period after the restorative justice conference or criminal justice trial

Means and standard deviations, number in treatment and control groups

Direct via CMA

Sherman 1998

Frequency of reoffending Means and standard deviations

Direct via CMA

Tuffin 2006 Self-reported victimization Percent victimized and total number in treatment and comparison groups before and after the intervention

Reconstructed 2x2 contingency table. Created a post-intervention log OR corrected for pre-intervention log OR (corrected log OR = post intervention log OR pre intervention log OR). Calculated variance of corrected log OR as sum of pre intervention log OR and post intervention log OR (most conservative estimate).

Bond 1997 Self-reported victimization in past year

Percent victimized and total number in treatment and comparison groups before and after the intervention

Reconstructed 2x2 contingency table. Created a post-intervention log OR corrected for pre-intervention log OR (corrected log OR = post intervention log OR pre intervention log OR). Calculated variance of corrected log OR as sum of pre intervention log OR and post intervention log OR (most conservative estimate).

Number in treatment and control groups, un-standardized regression coefficient and standard error

Calculated g using formulae from Card (2011), averaged g from two reoffending measures to create single measure for meta-analysis

Weisburd 2008

Self-reported reoffending (2 different measures)

McGarrell 2004

Number of arrests Mean and standard deviation in treatment and control groups

Direct via CMA

107 The Campbell Collaboration | www.campbellcollaboration.org

Study Operational definition Reported statistics Effect size calculation

Dunworth 1999 a-h

Self-reported victimization for burglary, robbery, assault, and assault with a weapon

Number that answered yes and no in pre and post tests

Added crime categories to get overall victimization measure, constructed contingency table. Entered into pre-post contingency table in CMA with pre-post correlation = 0 as no information and most conservative estimate

Hartstone 2003

Re-victimization Percent yes and no and overall number of respondents in pre and post tests

Calculated numbers from percentages, constructed contingency table. Entered into pre-post contingency table in CMA with pre-post correlation = 0 as no information and most conservative estimate

108 The Campbell Collaboration | www.campbellcollaboration.org

Appendix 3: References of eligible studies

Note: Studies marked with an asterisk (*) were included in the analyses in this report.

Angrist, J. D. (2006). Instrumental variables methods in experimental criminological research: what, why and how. Journal of Experimental Criminology, 2(1), 2344.

Applegate, R. J. (2004). Problem-oriented policing in Plymouth. Crime Prevention and

Community Safety: An International Journal, 6(2), 3550.

Arthur, G. L., Sisson, P. J., & McClung, C. E. (1980). Use of group process in teaching communication and life skills for law enforcement personnel. The Journal for Specialists in Group Work, 5(4), 196204.

Barbrey, J. W. (2003). An analysis of crime control policies in Knoxvilles public housing.

(Doctoral dissertation, University of Tennessee). Retrieved from ProQuest Dissertations & Theses (UMI Number: 3126986)

Battle, F. (1999). Measuring performance in the organized crime drug enforcement task force program. (Doctoral dissertation, University of Southern California). Retrieved from ProQuest Dissertations & Theses (UMI Number: 9987561)

Beedle, S. (1984). Citizen response to burglary information brochures: A follow-up study.

Portland, OR: Public Affairs Unit, Crime Prevention Section, Portland Police Bureau.

Bennett, T. (1990). Getting back in touch. Policing, 6(3), 510522.

Berringer, R. D. (2004). The effect of formal leadership training on the leadership styles of police field trainers. (Masters thesis, University of Prince Edward Island). Retrieved from ProQuest Dissertations & Theses (UMI Number: MQ93873)

Berry, G., & Carter, M. (1992). Assessing crime prevention initiatives: The first steps.

London: Home Office Crime Prevention Unit.

*Bond, C. E. W., & Gow, D. J. (1997). Policing the beat: The experience in Toowoomba,

Queensland. In R. Homel (Ed.), Crime Prevention Studies, Vol. 7. Policing for prevention: Reducing crime, public intoxication and injury (pp. 154173). Monsey, NY: Criminal Justice Press.

Bower, D. L., & Pettit, W. G. (2001). The Albuquerque police departments crisis intervention team. FBI Law Enforcement Bulletin, 70(2), 16.

109 The Campbell Collaboration | www.campbellcollaboration.org

Brennan, M., & Brennan, R. (1994). Cleartalk: Police responding to intellectual disability.

Wagga Wagga, Australia: Literacy Studies Network, School of Education, Charles Sturt University.

Brensilber, D., & Krasco, K. (1999). Analysis of police activity and programming in the

Woburn, Massachusetts Weed and Seed area. Statistical Analysis Center, Massachusetts Executive Office of Public Safety Programs Division. Retrieved from http://www.weedandseed.info/docs/studies_local/woburn-ma.pdf

Web End =http://www.weedandseed.info/docs/studies_local/woburn-ma.pdf

Brewster, J., Stoloff, M., & Sanders, N. (2005). Effectiveness of citizen police academies in changing the attitudes, beliefs, and behavior of citizen participants. American Journal of Criminal Justice, 30(1), 2134.

Brimley, S., Garrow, C., Jorgensen, M., & Wakeling, S. (2005). Strengthening and rebuilding tribal justice systems: Learning from history and looking towards the future. Cambridge, MA: The Harvard Project on American Indian Economic Development.

Brownlee, I., & Walker, C. (1998). The urban crime fund and total geographic policing initiatives in West Yorkshire. Policing and Society, 8(2), 125152.

Buren, B. A. (2003). An evaluation of citizen oversight in the Tempe, Arizona police department. (Doctoral dissertation, Arizona State University). Retrieved from ProQuest Dissertations & Theses (UMI Number: 3109548)

Burrows, J., Tarling, R., Mackie, A., Poole, H., & Hodgson, B. (2005). Forensic science pathfinder project: Evaluating increased forensic activity in two English police forces Home Office Online Report 46/05 Retrieved from http://webarchive.nationalarchives.gov.uk/20110220105210/rds.homeoffice.gov.uk/rds /pdfs05/rdsolr4605.pdf

Burt, M. R., Zweig, J. M., Andrews, C., Van Ness, A., Parikh, N., Uekert, B. K., & Harrell, A. V. (2001). 2001 Report: Evaluation of the STOP formula grants to combat violence against women. Washington, DC: The Urban Institute.

Butler, A. J. P., & Tharme, K. (1983). Chelmsley Wood policing experiment. Chelmsley Wood,

UK: West Midlands Police.

Bynum, T. S., McCluskey, C. P., & Patchin, J. W. (2000). Westside Operation Weed and Seed:

An Evaluation of an Early Truancy Initiative. East Lansing, MI: School of Criminal Justice, Michigan State University. Retrieved from http://www.jrsainfo.org/programs/grand-rapids.pdf

Web End =http://www.jrsainfo.org/programs/grand-rapids.pdf

Carroll, R. (2002). Partnerships: From rhetoric to reality (a case study in vehicle theft prevention). Paper presented at the Crime Prevention Conference, Sydney. Retrieved from http://www.aic.gov.au/events/aic%20upcoming%20events/2002/~/media/conferences/crimpre/carroll.pdf

Web End =http://www.aic.gov.au/events/aic%20upcoming%20events/2002/~/media/conferences http://www.aic.gov.au/events/aic%20upcoming%20events/2002/~/media/conferences/crimpre/carroll.pdf

Web End =/crimpre/carroll.pdf

Challinger, D. (2004). Crime Stoppers Victoria: An Evaluation. Technical and Background paper No. 8. Canberra: Australian Institute of Criminology.

110 The Campbell Collaboration | www.campbellcollaboration.org

Chandler, W. C. (2001). The deterrent effect of the undercover compliance check strategy to reduce the sale of alcoholic beverages to minors in North Carolina: A quasi-experimental design. (Doctoral dissertation, North Carolina State University). Retrieved from ProQuest Dissertations & Theses (UMI Number: 3030029)

Child and Family Policy Center. (1999). Des Moines Weed and Seed Evaluation: Final Report.

Retrieved fromhttp://www.weedandseed.info/docs/studies_local/des-moines-ia.pdf

Web End = http://www.weedandseed.info/docs/studies_local/des-moines-ia.pdf

Clarke, C. (2003). Hands firmly on the steering wheel: Restructuring Metro Toronto Police

Service. Policing: An International Journal of Police Strategies & Management, 26(3), 473-489.

Cook, P. M. (1994). Breaking the link on the streets. Policing, 10(2), 8087.

Corcoran, J., & Allen, S. (2005). The effects of a police/victim assistance crisis team approach to domestic violence. Journal of Family Violence, 20(1), 3945.

Cox, S. M. (1999). Final report of the evaluation of the New Britain Weed and Seed program.

New Britain, CT: Department of Criminology and Criminal Justice, Central Connecticut State University. Retrieved from http://www.weedandseed.info/docs/studies_local/new-britain.pdf

Web End =http://www.weedandseed.info/docs/studies_local/new-britain.pdf

Crawford, A. (1994). The partnership approach to community crime prevention: Corporatism at the local level? Social & Legal Studies, 3(4), 497519.

Crawford, A. (1995). Appeals to community and crime prevention. Crime, Law and Social

Change, 22(2), 97126.

Crime Prevention Unit, South Australian Attorney-General's Department. (2002). The South

Australian Residential Break and Enter Pilot Project evaluation report: Summary volume. Canberra: Commonwealth Attorney-General's Department.

Criminal Justice Commission. (1998). Beenleigh Calls for Service project: Evaluation report.

Brisbane, Australia: Author.

*Dai, M. (2007). Procedural justice during police-citizen encounters. (Doctoral dissertation, University of Cincinatti). Retrieved from ProQuest Dissertations & Theses (UMI Number: 3280116)

Davis, R. C., & Taylor, B. G. (1997). A proactive response to family violence: The results of a randomized experiment. Criminology, 35(2), 307333.

Department of Justice, Victoria. (2008). Enforcement Operations Koori Strategy, 2008

2010. Melbourne, Australia: Author.

Dogutas, C. (2007). Reactive vs. proactive strategies: The effectivenss of school resource officers to prevent violence in schools. (Doctoral dissertation, Kent State University). Retrieved from ProQuest Dissertations & Theses (UMI Number: 3286550)

Driscoll, L. N., Bora, D. J., & DeTardo-Bora, K. A. (2003). Program evaluation of the O.N.E.

Wheeling Weed and Seed. Wheeling, WV: Wheeling Jesuit University. Retrieved from http://www.weedandseed.info/docs/studies_local/wheeling.pdf

Web End =http://www.weedandseed.info/docs/studies_local/wheeling.pdf

111 The Campbell Collaboration | www.campbellcollaboration.org

Dunham, R. G., & Alpert, G. P. (1995). Controlling the use of force: An evaluation of street-level narcotics interdiction in Miami. American Journal of Police, 14(1), 83100.

Dunworth, T. (2000). National evaluation of youth firearms violence initative. Research in brief. Washington, DC: National Institute of Justice.

*Dunworth, T., & Mills, G. (1999a). National evaluation of Weed and Seed: Akron, Ohio research report. Washington, DC: National Institute of Justice. Retrieved from http://www.weedandseed.info/docs/studies_national/akron-oh.pdf

Web End =http://www.weedandseed.info/docs/studies_national/akron-oh.pdf

*Dunworth, T., & Mills, G. (1999b). National evaluation of Weed and Seed: Hartford,

Connecticut research report. Washington, DC: National Institute of Justice. Retrieved from http://www.weedandseed.info/docs/studies_national/hartford-ct.pdf

Web End =http://www.weedandseed.info/docs/studies_national/hartford-ct.pdf

*Dunworth, T., & Mills, G. (1999c). National evaluation of Weed and Seed: Las Vegas,

Nevada research report. Washington, DC: National Institute of Justice. Retrieved from http://www.weedandseed.info/docs/studies_national/lasvegas-nv.pdf

Web End =http://www.weedandseed.info/docs/studies_national/lasvegas-nv.pdf

*Dunworth, T., & Mills, G. (1999d). National evaluation of Weed and Seed: Manatee and

Sarasota Counties, Florida research report. Washington, DC: National Institute of Justice. Retrieved from http://www.weedandseed.info/docs/studies_national/manatee-sarasota-fl.pdf

Web End =http://www.weedandseed.info/docs/studies_national/manatee- http://www.weedandseed.info/docs/studies_national/manatee-sarasota-fl.pdf

Web End =sarasota-fl.pdf

*Dunworth, T., & Mills, G. (1999e). National evaluation of Weed and Seed: Pittsburgh,

Pennsylvania research report. Washington, DC: National Institute of Justice. Retrieved from http://www.weedandseed.info/docs/studies_national/pittsburgh-pa.pdf

Web End =http://www.weedandseed.info/docs/studies_national/pittsburgh-pa.pdf

*Dunworth, T., & Mills, G. (1999f). National evaluation of Weed and Seed: Salt Lake City,

Utah research report. Washington, DC: National Institute of Justice. Retrieved from http://www.weedandseed.info/docs/studies_national/saltlakecity-ut.pdf

Web End =http://www.weedandseed.info/docs/studies_national/saltlakecity-ut.pdf

*Dunworth, T., & Mills, G. (1999g). National evaluation of Weed and Seed: Seattle,

Washington research report. Washington, DC: National Institute of Justice. Retrieved from http://www.weedandseed.info/docs/studies_national/seattle-wa.pdf

Web End =http://www.weedandseed.info/docs/studies_national/seattle-wa.pdf

*Dunworth, T., & Mills, G. (1999h). National evaluation of Weed and Seed: Shreveport,

Louisiana research report. Washington, DC: National Institute of Justice. Retrieved from http://www.weedandseed.info/docs/studies_national/shreveport-la.pdf

Web End =http://www.weedandseed.info/docs/studies_national/shreveport-la.pdf

*Eckert, R. (2009). Community policing as procedural justice: An examination of Baltimore residents after the implementation of a community policing strategy. (Masters thesis, Villanova University). Retrieved from ProQuest Dissertations & Theses (UMI Number: 1462400)

Feder, L., Jolin, A., & Feyerherm, W. (2000). Lessons from two randomized experiments in criminal justice settings. Crime & Delinquency, 46(3), 380400.

Friday, P. C., Lord, V. B., Exum, M. L., & Hartman, J. L. (2006). Evaluating the impact of a specialized domestic violence police unit. Charlotte, NC: Department of Criminal Justice, University of North Carolina,.

Gladstone, F. J. (1980). Co-ordinating crime prevention efforts. Home Office Research Study

No. 62. London: Home Office Crime Policy Planning Unit.

112 The Campbell Collaboration | www.campbellcollaboration.org

Goldkamp, J. S., & Vilcica, E. R. (2008). Targeted enforcement and adverse system side effects: The generation of fugitives in Philadelphia. Criminology, 46(2), 371409.

Goldstein, H. (1990). Problem-oriented policing. New York: McGraw-Hill.

Grossman, A. H. (2004). Attitudes and perceptions regarding community policing programs in metropolitan police agencies. (Masters thesis, California State University). Retrieved from ProQuest Dissertations & Theses (UMI Number: 1424804)

Guzik, K. W. (2006). Governing domestic violence: The power, practice, and efficacy of presumptive arrest and prosecution against the violent subjectivities of intimate abusers. (Doctoral dissertation, University of Illinois). Retrieved from ProQuest Dissertations & Theses (UMI Number: 3223604)

Hakkert, A. S., Gitelman, V., Cohen, A., Doveh, E., & Umansky, T. (2001). The evaluation of effects on driver behavior and accidents of concentrated general enforcement on interurban roads in Israel. Accident; analysis and prevention, 33(1), 4363.

*Hall, P. A. (1987). Neighborhood Watch and participant perceptions. (Doctoral dissertation,

University of Southern California). Retrieved from ProQuest Dissertations & Theses (UMI Number: 0560502)

Hallas, H. B. (2004). City of Youngstown Police Departments Weed and Seed Strategy Year

Four Evaluation Report. Youngstown, OH: Center for Human Services Development, Youngstown State University. Retrieved from http://www.weedandseed.info/docs/studies_local/youngstown-oh.pdf

Web End =http://www.weedandseed.info/docs/studies_local/youngstown-oh.pdf

Hanewicz, W. B., Fransway, L. M., & O'Neill, M. W. (1982). Improving the linkages between community mental health and the police. Journal of Police Science and Administration, 10(2), 218223.

Harrington, M. P. (2002). An inquiry into the effects of diversity training on police policy makers in Fairfax County, Virginia. (Doctoral dissertation, University of Virginia), Retrieved from ProQuest Dissertations & Theses (UMI Number: 3044904)

Harris, R. J., O'Connell, J. P., & Mande, M. J. (1998). City of Wilmington Operation Weed and Seed 1992 to 1996: An evaluation. Dover, DE: Delaware Statistical Analysis Center. Retrieved fromhttp://www.weedandseed.info/docs/studies_local/wilmington3.pdf

Web End = http://www.weedandseed.info/docs/studies_local/wilmington3.pdf

Harris, R. J., O'Connell, J. P., & Wilhite, S. (2005). Evaluation of Operation Weed and Seed in

Wilmington, Delaware 2001 to 2004. Dover, DE: Delaware Statistical Analysis Center. Retrieved from http://sac.omb.delaware.gov/publications/documents/2004_Weed_and_Seed_Evaluation.pdf

Web End =http://sac.omb.delaware.gov/publications/documents/2004_Weed_and_Seed_Evaluat http://sac.omb.delaware.gov/publications/documents/2004_Weed_and_Seed_Evaluation.pdf

Web End =ion.pdf

*Hartstone, E. C., & Richetelli, D. M. (2003). Final assessment of the Strategic Approaches to Community Safety initiative in New Haven. Washington, DC: National Institute of Justice. Retrieved from https://www.ncjrs.gov/pdffiles1/nij/grants/208859.pdf

Web End =https://www.ncjrs.gov/pdffiles1/nij/grants/208859.pdf

Henderson, C., Papagapiou, P., Gains, A., & Knox, J. (2004). Driving crime down: Denying criminals the use of the road. London: PA Consulting Group.

113 The Campbell Collaboration | www.campbellcollaboration.org

Henig, J. R. (1984). Citizens against crime: An assessment of the neighborhood watch program in Washington, DC. Occasional Paper No. 2. Washington, DC: Center for Washington Area Studies, George Washington University.

*Hinds, L. (2009). Youth, police legitimacy and informal contact. Journal of Police and

Criminal Psychology, 24, 1021.

*Holland, R. C. (1996). Informal resolution: Dealing with complaints against police in a manner satisfactory to the officer and the complainant. International Journal of Comparative and Applied Criminal Justice, 20(1), 8393.

Holmberg, L. (2005). Policing and the feeling of safety: The rise (and fall?) of community policing in the Nordic countries. Journal of Scandinavian Studies in Criminology and Crime Prevention, 5(2), 205219.

Holmberg, U. (2004). Crime victims' experiences of police interviews and their inclination to provide or omit information. International Journal of Police Science and Management, 6(3), 155170.

Honess, T., Maguire, M., & Charman, E. (1993). Vehicle watch and car theft: An evaluation

(Paper No. 50). London: Police Research Group, Crime Prevention Unit, Home Office Police Department.

Hovell, M. F., Seid, A. G., & Liles, S. (2006). Evaluation of a police and social services domestic violence program: Empirical evidence needed to inform public health policies. Violence Against Women, 12(2), 137159.

Hoyle, C., Young, R., & Hill, R. (2002). Proceed with caution: An evaluation of the Thames

Valley Police initiative in restorative cautioning. York, UK: Joseph Rowntree Foundation.

Johnson, R. A. (1997). Integrated patrol: Combining aggressive enforcement and community policing. FBI Law Enforcement Bulletin, 66(11), 610.

Jones, R., Joksch, H., Lacey, J., Wiliszowski, C., & Marchetti, L. (1995). Summary report: Field test of combined speed, alcohol, and safety belt enforcement strategies. Winchester, MA: Mid-America Research Institute.

Josi, D. A., Donahue, M. E., Magnus, R. E., & Bennett, K. J. (2000). Savannah's Weed and

Seed program evaluation (1999): Final report. Savannah, GA: Statistical Analysis Center, Criminal Justice Coordinating Council of Georgia. Retrieved from http://www.weedandseed.info/docs/studies_local/savannah-ga.pdf

Web End =http://www.weedandseed.info/docs/studies_local/savannah-ga.pdf

Kaiser, M. (1995). 1995 CAPS training evaluation report. Chicago, IL: Institute for Policy

Research, Northwestern University.

Katz, C. M. (1997). Police and gangs: A study of a police gang unit. (Doctoral dissertation, University of Nebraska). Retrieved from ProQuest Dissertations & Theses (UMI Number: 9820701)

Keaton, S., Ward, R., & Burke, C. (2009). North County Gang Enforcement Collaborative final evaluation report. San Diego, CA: San Diego Association of Governments

114 The Campbell Collaboration | www.campbellcollaboration.org

(SANDAG). Retrieved from http://www.sandag.org/uploads/publicationid/publicationid_1478_10693.pdf

Web End =http://www.sandag.org/uploads/publicationid/publicationid_1478_10693.pdf

Kennedy, D. M., Braga, A. A., Piehl, A. M., & Waring, E. J. (2001). Reducing gun violence: The

Boston Gun Project's Operation Ceasefire. Washington, DC: National Institute of Justice. Retrieved from https://www.ncjrs.gov/pdffiles1/nij/188741.pdf

Web End =https://www.ncjrs.gov/pdffiles1/nij/188741.pdf

Kent, D. R., Donaldson, S. I., Wyrick, P. A., & Smith, P. J. (2000). Evaluating criminal justice programs designed to reduce crime by targeting repeat gang offenders. Evaluation and Program Planning, 23(1), 115124.

Kerley, K. R., & Benson, M. L. (2000). Does community-oriented policing help build stronger communities? Police Quarterly, 3(1), 4669.

*Kerstetter, W. A., & Rasinski, K. A. (1994). Opening a window into police internal affairs:

Impact of procedural justice reform on third-party attitudes. Social Justice Research, 7(2), 107127.

Kessler, D. A., & Borella, D. (1997). Taking back Druid Hills: An evaluation of a community policing effort in Birmingham, Alabama. Law & Policy, 19(1), 95115.

Korander, T., & Torronen, J. (2005). Zero confidence in youths?: Experiences of the zero tolerance experiment in Finland. Young: Nordic Journal of Youth Research, 13(1), 4771.

Koss, M. P., Bachar, K. J., Hopkins, C. Q., & Carlson, C. (2004). Expanding a community's justice response to sex crimes through advocacy, prosecutorial, and public health collaboration: Introducing the RESTORE program. Journal of Interpersonal Violence, 19(12), 14351463.

Kreisel, B. W. (1998). An evaluation of police officers' perceptions of internal and external systems of citizen complaint procedures of police misconduct. (Doctoral dissertation, University of Nebraska). Retrieved from ProQuest Dissertations & Theses (UMI Number: 9901210)

Lardner, R. (1992). Factors affecting police/social work inter-agency co-operation in a child protection unit. The Police Journal, July, 213228.

Lawrenz, F., Lembo, J. F., & Schade, T. (1988). Time series analysis of the effect of a domestic violence directive on the number of arrests per day. Journal of Criminal Justice, 16(6), 493498.

Loveday, B. (2007). Basic Command Units (BCUs) and local authorities: Future mechanisms of police accountability and service delivery. International Journal of Police Science and Management, 9(4), 324335.

Lurigio, A., Davis, R., Regulus, T., Gwisada, V., Popkin, S., Dantzker, M., Smith, B., & Ouellet, A. (1993). An evaluation of the Cook County states attorneys office narcotics nuisance abatement program. Chicago, IL: Loyola University Department of Criminal Justice.

Matthews, R. (1993). Kerb-crawling, prostitution and multi-agency policing. Crime Prevention

Unit Series Paper No. 43. London: Home Office Police Department, Police Reasearch Group.

115 The Campbell Collaboration | www.campbellcollaboration.org

Mazerolle, L. G., & Terrill, W. (1997). Problem-oriented policing in public housing: Identifying the distribution of problem places. Policing: An International Journal of Police Strategies & Management, 20(2), 235255.

McCold, P. (2003). An experiment in police-based restorative justice: The Bethlehem

(Pennsylvania) Project. Police Practice & Research, 4(4), 379390.

McCold, P., & Wachtel, B. (1998). The Bethlehem Pennsylvania Police Family Group

Conferencing Project. Pipersville, PA: Community Service Foundation. Retrieved from http://www.realjustice.org/uploads/article_pdfs/BPD.pdf

Web End =http://www.realjustice.org/uploads/article_pdfs/BPD.pdf

McDevitt, J., Farrell, A., & Wolff, R. (2008). COPS Evaluation Brief No. 3: Creating a culture of integrity. Washington, DC: Office of Community Oriented Policing Services, US Department of Justice.

*McGarrell, E. F., & Chermak, S. (2004). Strategic approaches to reducing firearms violence: Final report on the Indianapolis violence reduction partnership. Washington, DC: National Institute of Justice.

Meerbaum, M. L. (1980). Role-playing involvement and the effects of a crisis intervention training program for police recruits (Doctoral dissertation). Retrieved from ProQuest Dissertations and Theses database (UMI No. 8109826).

Mesko, G., & Lobnikar, B. (2005). The contribution of local safety councils to local responsibility in crime prevention and provision of safety. Policing: An International Journal of Police Strategies & Management, 28(2), 353.

Moffett, L. (1996). An evaluation of selected variables of the Truancy Reduction Impact

Program. (Doctoral dissertation, Texas A&M University). Retrieved from ProQuest Dissertations & Theses (UMI Number: 9704363)

Moon, B., McCluskey, J., & Lee, S. (2005). Korean police officers' attitude toward the efficacy of mini-police stations. Journal of Criminal Justice, 33(5), 441449.

Morrison, S. (2000). Evaluation of Albuquerque Weed and Seed Sites: Trumbull and La Mesa

Neighborhoods: Final Report. Albuquerque, NM: Statistical Analysis Center, Institute for Social Research, University of New Mexico. Retrieved from http://www.weedandseed.info/docs/studies_local/albuquerque-nm.pdf

Web End =http://www.weedandseed.info/docs/studies_local/albuquerque-nm.pdf

*Murphy, K., Hinds, L., & Fleming, J. (2008). Encouraging public cooperation and support for police. Policing and Society, 18(2), 136155.

Murray, B. J. (2003). Perceptions of principals, school resource officers, and school resource officer supervisors of the School Resource Officer Program in comprehensive high schools in Riverside County, California. (Doctoral dissertation, University of La Verne). Retrieved from ProQuest Dissertations & Theses (UMI Number: 3090259)

North Central Florida Health Planning Council. (1999). City of Ocala Weed and Seed evaluation final report. Ocala, FL: Author. Retrieved from http://www.weedandseed.info/docs/studies_local/ocala.pdf

Web End =http://www.weedandseed.info/docs/studies_local/ocala.pdf

116 The Campbell Collaboration | www.campbellcollaboration.org

Novak, K. J., Hartman, J. L., Holsinger, A. M., & Turner, M. G. (1999). The effects of aggressive policing of disorder on serious crime. Policing: An International Journal of Police Strategies & Management, 22(2), 171190.

*Panetta, M. J. (2000). Identifying and assessing citizen perceptions of police and community policing practices. (Doctoral dissertation, Michigan State University). Retrieved from ProQuest Dissertations & Theses (UMI Number: 9971979)

Part, D. (2006). A flexible response to domestic abuse: Findings from an evaluation. Practice,

18(1), 4758.

Pate, A. M., & Annan, S. O. (1989). The Baltimore community policing experiment: Summary report. Washington, DC: Police Foundation.

Pearce, J. B., & Snortum, J. R. (1983). Police effectiveness in handling disturbance calls: An evaluation of crisis intervention training. Criminal Justice and Behavior, 10(1), 7192.

Penfold, C., Hunter, G., Campbell, R., & Barham, L. (2004). Tackling client violence in female street prostitution: Inter-agency working between outreach agencies and the police. Policing and Society, 14(4), 365379.

Perez, D. (1992). Police review systems. MIS Report, 24(8), 112.

Pickering, S., Wright-Neville, D., McCulloch, J., & Lentini, P. (2007). Counter-terrorism policing and culturally diverse communities. Melbourne, Australia: Monash University.

Pierce, K. M., & Vandell, D. L. (1997). Safe Haven program evaluation (199697). Madison,

WI: Office of Community Services, City of Madison.

*Ren, L., Cao, L., Lovrich, N., & Gaffney, M. (2005). Linking confidence in police with the performance of the police: Community policing can make a difference. Journal of Criminal Justice, 33, 5566.

Richter, K. K. P. (1998). Three case studies evaluating a community-based initiative to reduce adolescent substance abuse. (Doctoral dissertation, University of Kansas). Retrieved from ProQuest Dissertations & Theses (UMI Number: 9904809)

Rigotti, N. A., DiFranza, J. R., Chang, Y., Tisdale, T., Kemp, B., & Singer, D. E. (1997). The effect of enforcing tobacco-sales laws on adolescents' access to tobacco and smoking behavior. The New England Journal of Medicine, 337(15), 10441051.

*Robinson, A. L., & Chandek, M. S. (2000). Philosophy into practice? Community policing units and domestic violence victim participation. Policing: An International Journal of Police Strategies & Management, 23(3), 280302.

Rodabough, T., & Young, A. R. (2002). Fairness in the application of a curfew ordinance: A case study in applied sociology. Michigan Sociological Review, 16, 1538.

Roehl, J. A., & Cook, R. F. (1984). Evaluation of the Urban Crime Prevention program.

Reston, VA: Institute for Social Analysis.

Rohe, W. M., Adams, R. E., & Arcury, T. A. (2001). Planner's notebook: Community policing and planning. Journal of the American Planning Association, 67(1), 7890.

117 The Campbell Collaboration | www.campbellcollaboration.org

Rosenbaum, D. P. (1987). Coping with victimization: The effects of police intervention on victims' psychological readjustment. Crime & Delinquency, 33(4), 502519.

Rosenthal, S., & Reimer, H. (1997). Final evaluation report for the Philadelphia Weed and

Seed project. Philadelphia, PA: Center for Social Policy and Community Development, Temple University. Retrieved from http://www.weedandseed.info/docs/studies_local/philadelphia97.pdf

Web End =http://www.weedandseed.info/docs/studies_local/philadelphia97.pdf

Roth, J. A., Ryan, J. F., Gaffigan, S. J., Koper, C. S., Moore, M. H., Roehl, J. A., . . . Thacher, D. (2000). National evaluation of the COPS ProgramTitle I of the 1994 Crime Act. Washington, DC: National Institute of Justice.

Rubenser, L. L. (2000). Special units in policing: A case study of a nuisance task force.

(Doctoral dissertation, University of Nebraska). Retrieved from ProQuest Dissertations & Theses (UMI Number: 9975584)

Ryan, B. J. (2007). What the police are supposed to do: Contrasting expectations of community policing in Serbia. Policing and Society, 17(1), 120.

Sagar, T. (2005). Street Watch: Concept and practice. Civilian participation in street prostitution control. British Journal Of Criminology, 45(1), 98112.

Schmitz, R., Wassenberg, P. S., Gransky, L., Hagner, P. G., & Sims, P. L. (1998). An evaluation of the Homicide and Violent Crime Strike Force program in Madison and St. Clair counties. Springfield, IL: Center for Legal Studies, Institute for Public Affairs, University of Illinois.

Sellers, C. S., Taylor, T. J., & Esbensen, F.-A. (1998). Reality check: Evaluating a school-based gang prevention model. Evaluation Review, 22(5), 590608.

Shaler, G. (2004). Lincoln County Weed & Seed: An implementation evaluation of a rural crime prevention project. Portland, ME: Maine Statistical Analysis Center, Muskie School of Public Service, University of Southern Maine. Retrieved from http://www.jrsa.org/programs/maine.pdf

Web End =http://www.jrsa.org/programs/maine.pdf

*Shapland, J., Atkinson, A., Atkinson, H., Chapman, B., Dignan, J., Howes, M., Johnstone, J.,

Robinson, G., & Sorsby, A. (2007). Restorative justice: The views of victims and offenders. Ministry of Justice Research Series (3). United Kingdom: Ministry of Justice.

*Shapland, J., Atkinson, A., Atkinson, H., Dignan, J., Edwards, L., Hibbert, J., . . . Sorsby, A.

(2008). Does restorative justice affect reconviction? The fourth report from the evaluation of three schemes. Ministry of Justice Research Series (Vol. 10). London: Ministry of Justice.

Sharkey, S. M., & Stoke, C. B. (1985). Evaluation of 1983 selective speed enforcement projects in Virginia. Charlottesville, VA: Virginia Highway & Transportation Research Council.

Sharp, D., Atherton, S., & Williams, K. (2008). Civilian policing, legitimacy and vigilantism:

Findings from three case studies in England and Wales. Policing & Society, 18(3), 245 257.

Shepherd, J., & Lisles, C. (1998). Towards multi-agency violence prevention and victim support. British Journal Of Criminology, 38(3), 351370.

118 The Campbell Collaboration | www.campbellcollaboration.org

*Sherman, L. W., Strang, H., Barnes, G. C., Braithwaite, J., Inkpen, N., & Teh, M. M. (1998).

Experiments in restorative policing: A progress report on the Canberra Reintegrative Shaming Experiments (RISE). Canberra: Australian Federal Police and Australian National University.

Shernock, S. (2004). The MJTF as a type of coordination compatible with both the police consolidation and community policing movements. Police Practice and Research, 5(1), 6785.

Simmons, J., Legg, C., & Hosking, R. (2003). National Crime Recording Standard (NCRS): An analysis of the impact on recorded crime. Part One: The national picture. London: Home Office.

*Singer, L. (2004). Reassurance policing: An evaluation of the local management of community safety. Home Office Research Studies (Vol. 228). London: Home Office.

*Skogan, W. G., & Steiner, L. (2004). CAPS at Ten: Community policing in Chicago - An evaluation of Chicago's alternative policing strategy. Chicago, IL: The Chicago Community Policing Evaluation Consortium. Retrieved from http://www.ipr.northwestern.edu/publications/policing_papers/Yr10-CAPSeval.pdf

Web End =http://www.ipr.northwestern.edu/publications/policing_papers/Yr10-CAPSeval.pdf

Smith, B. W., Novak, K. J., Frank, J., & Travis, L. F. I. (2000). Multijurisdictional drug task forces: An analysis of impacts. Journal of Criminal Justice, 28(6), 543556.

South Australian Office of Crime Statistics and Research. (2005). Evaluation of the Fake ID project. Adelaide, Australia: Author.

Stokes, R. J., Lawton, B., Henderson, K., & Greene, J. R. (2006). Co-producing commercial safety services in Philadelphia. Philadelphia, PA: Center for Public Policy, Temple University.

Sun, I. Y. (2003). Officer proactivity: A comparison between police field training officers and non-field training officers. Journal of Criminal Justice, 31(3), 265277.

Swanson, C., & Owen, M. (2007). Building bridges: Integrating restorative justice with the school resource officer model. Working Paper No. 1. International Police Executive Symposium. Retrieved from http://www.ipes.info/WPS/WPS%20No%201.pdf

Web End =http://www.ipes.info/WPS/WPS%20No%201.pdf

Taylor, N., & Charlton, K. (2005). Police shopfronts and reporting to police by retailers.

Trends and Issues in Crime and Criminal Justice No. 295. Canberra, Australia: Australian Institute of Criminology.

Tennant, W. A. (1986). Police management of dangerous mentally disturbed individuals through proactive intervention. (Doctoral dissertation, Wright Institute). Retrieved from ProQuest Dissertations & Theses (UMI Number: 8706397)

Terpstra, J., & Trommel, W. (2009). Police, managerialization and presentational strategies.

Policing: An International Journal Of Police Strategies & Management, 32(1), 128143.

Tita, G., Riley, K. J., Ridgeway, G., Abrahamse, A. F., & Greenwood, P. W. (2003). Reducing gun violence: Results from an intervention in East Los Angeles. Santa Monica, CA: Rand Corporation.

119 The Campbell Collaboration | www.campbellcollaboration.org

*Tuffin, R., Morris, J., & Poole, A. (2006). An evaluation of the impact of the National

Reassurance Policing programme. Home Office Research Study 296. London: Development and Statistics Directorate, Home Office Research.

Turnbull, P. J., Webster, R., & Stiliwell, G. (1996). Get it while you can: An evaluation of an early intervention project for arrestees with alcohol and drug problems. London: Home Office Drugs Prevention Initiative.

Urban, L. S. (2005). The deterrent effect of curfew enforcement: Operation Nightwatch in St.

Louis. (Doctoral dissertation, University of Missouri). Retrieved from ProQuest Dissertations & Theses (UMI Number: 3173535)

Virginia Department of Criminal Justice Services. (2000). Evaluation of Virginia's Weed and

Seed Initiative. Richmond, VA: Author. Retrieved from http://www.dcjs.virginia.gov/research/documents/weedandseed/

Web End =http://www.dcjs.virginia.gov/research/documents/weedandseed/

Virta, S. (2002). Local security management. Policing: An International Journal of Police

Strategies & Management, 25(1), 190200.

Volpe, M. R., & Phillips, N. (2003). Police use of mediation. Conflict Resolution Quarterly,

21(2), 263267.

Walker, S. (2001). Domestic violence: Analysis of a community safety alarm system. Child

Abuse Review, 10(3), 170182.

Wallis, A., & Ford, D. (Eds.) (1980). Crime prevention through environmental design: The commercial demonstration in Portland, Oregon. Washington, DC: National Institute of Justice.

*Weisburd, D., Morris, N. A., & Ready, J. (2008). Risk-focused policing at places: An experimental evaluation. Justice Quarterly, 25(1), 163200.

Weisel, D. L. (1990). Playing the home field: A problem-oriented approach to drug control.

American Journal of Police, 9(1), 75.

White, M. D., Fyfe, J. J., Campbell, S. P., & Goldkamp, J. S. (2001). The school-police partnership: Identifying at-risk youth through a Truant Recovery Program. Evaluation Review, 25(5), 507532.

Willis, J. J., Mastrofski, S. D., & Weisburd, D. (2007). Making sense of COMPSTAT: A theory-based analysis of organizational change in three police departments. Law & Society Review, 41(1), 147188.

Winfree, L. T. Jr, Lynskey, D. P., & Maupin, J. r. (1999). Developing local police and federal law enforcement partnerships: G.R.E.A.T. as a case study of policy implementation. Criminal Justice Review, 24(2), 145168.

*Young, R., Hoyle, C., Cooper, K., & Hill, R. (2005). Informal resolution of complaints against the police: A quasi-experimental test of restorative justice. Criminal Justice, 5(3), 279 317.

*Zevitz, R. G., Palazzari, T., Frinzi, J. N., & Mallinger, A. (1997). Milwaukee Weed and Seed program evaluation final report. Milwaukee, WI: Marquette University. Retrieved from http://www.weedandseed.info/docs/studies_local/milwaukee.pdf

Web End =http://www.weedandseed.info/docs/studies_local/milwaukee.pdf

120 The Campbell Collaboration | www.campbellcollaboration.org

Appendix 4: Coding Sheet for Legitimacy in Policing Review

I. ELIGIBILITY CHECK SHEET INSTRUCTIONS

1. Document ID: _____________

List the unique number assigned to every document retrieved and reviewed

(Top right corner of document).

2. First Author Last Name:

Last name only of the first named study author

Title of the journal article, report, thesis or book in which the study is described.

4. Publication Summary: Enter the Journal Name, Volume and

Issue, date of publication of article, book , report or thesis, publisher (if any).

5. Coders name: Enter your name

6. Date eligibility determined:_______/_______/_______

Enter the date the eligibility of this study was determined.

3. Study Title:

7. A study MUST meet the following criteria to be eligible for the review. Answer each question with a YES or NO:

a. Study is written after 1980: Y/N

b. Study is written up in English, French or German: Y/N

c. Study includes an evaluation of a police intervention (involving a directive or training)

Refer to the Legitimacy in Policing Protocol (LPP) for further information regarding each eligibility criteria (section 3: 3.1.1 to 3.1.8).

121 The Campbell Collaboration | www.campbellcollaboration.org

aimed at encouraging legitimacy in policing: Y/N

d. One can extract data from the study to calculate

effect size: Y/N

e. Study includes a comparison group: Y/N

f. Study reports at least one of the following outcomes:

o Satisfaction

o Confidence in police

o Cooperation

o Compliance with police

o Respect for the law

o Establishment of social ties

o Reoffending

o Crime

o Disorder

Note: Study is not eligible if it does not utilize a comparison group from:

Randomized experiment

Quasi-experimental comparison group, with or without a pre-test

Quasi-experimental interrupted time series

Other. Note that Other studies are NOT eligible for analysis but may be eligible for literature review and inventory of interventions aimed at increasing legitimacy in policing.

8. a. Document is eligible for review: Yes_______ No_______

b. Document is eligible for meta-analysis: Yes_______ No_______

This is a summary of the eligibility decision. The coder should work through the determination of document eligibility in section II before responding to the questions above.

9. Does document contain a review of the literature, pertinent background information, references that might be relevant or any other information that might be of interest to the review? Y/N

There are many different ways that an article might be worthwhile to the review even if it does not meet the eligibility criteria. Please summarize the type of information contained in the article, book, report or thesis (with page numbers) that is of relevance to our review.

122 The Campbell Collaboration | www.campbellcollaboration.org

Section II. Determination of document eligibility. Refer to section 3: 3.1.1 to 3.1.8 in LPP for further information regarding each eligibility criteria.

Study is written after 1980 and is written in English, French or German

Ineligible for review

Study utilizes comparison/control group in: Randomized experiment Quasi-experimental comparison group, Quasi-experimental interrupted time series Other studies are NOT eligible for analysis

Able to calculate effect size from study (or acquire missing information)

Refer to section 3.1.2 in protocol for more info

Refer to section 3.1.3 & 3.4 in protocol for info

Yes

No

Yes

No Ineligible for meta-analysis

Study includes police intervention (involving a directive or training) aimed at encouraging legitimacy in policing through the use of procedural approaches

Intervention involves police interacting

with The individual (e.g. citizen, victim, offender) The group (e.g. community, group gathering) Third party (e.g. religious leader, community advisor)

Study reports at least one of the following outcomes: Satisfaction, compliance, cooperation, respect for law and establishment of social ties and/or reduction in reoffending, crime and disorder

Yes

No

Ineligible for review

Yes

No Ineligible for meta-analysis

Yes

No

Ineligible for review

Yes

No Ineligible for meta-analysis

Relevant for Review

Stop! If ineligible, do not complete any further sections.

Eligible for Meta-Analysis

Check box if relevant to the review

Check box if eligible for analysis

Ineligible for Review or Meta-Analysis

Describe here why the study is ineligible:

123 The Campbell Collaboration | www.campbellcollaboration.org

III. SEARCH INFORMATION INSTRUCTIONS

1. Searchers name:

Person who conducted search

2. Date of search: _______/_______/_______

Date the study was identified

3. Date the study was retrieved: ________/__________/_________

Date that the study was downloaded, printed, obtained via email, mail etc

The unique identification number of the study

4. Document ID number:

5. Datasource:

a. Electronic database:_______________________________________

b. Specify search terms:______________________________________

c. Bibliographic search:_______________________________________

d. Reference search: _______________________________________

e. Other: _______________________________________

Circle the datasource used to retrieve the document being reviewed.

a. Name the database where study found

b. If an electronic database was used to locate the study, specify the exact search terms used (e.g. police AND procedural justice).

c. Where a bibliographic search was conducted, specify the name of expert whose bibliography was reviewed.

d. If reference search, list the author, year and title of publication from which the study was identified.

e. If none of the above, list the source by which the study was identified.

124 The Campbell Collaboration | www.campbellcollaboration.org

IV. REFERENCE INFORMATION INSTRUCTIONS

Enter all names of contributing authors, in order that they appear

Author(s):

1. Date (Day, Month, Year) of publication: _______/_____/____

Enter day, month, year (or just year or just month or year)

Write out the entire title of the study, article, book, chapter, thesis or report

2. Title:

3. Document type:

a. Book

b. Book chapter

c. Journal

i. Peer reviewed

ii. Not peer reviewed

d. Conference paper

e. Dissertation or thesis

f. Government publication (local)

g. Government publication (federal)

h. Police department report

i. Technical report

j. Other:

Circle the document type that applies

If other publication type, list the relevant reference details (e.g. book in the case of a book chapter, governmental organization, university in the case of a dissertation etc.)

5. Volume: 6. Issue

7. Country of publication:

4. Journal Name:

125 The Campbell Collaboration | www.campbellcollaboration.org

8. When was the research undertaken? From/Start date: Unclear To/Finish date:

9. What are the primary aims or hypotheses of this article. Copy and paste the aims section from the source

V. INTERVENTION INFORMATION

INSTRUCTIONS Page / paragrap h referenc e

1

1. Intervention Description:

a. Describe the experimental intervention:

b. Describe the comparison/control group intervention:

Provide detail on the police intervention with emphasis on key differences between the experimental (treatment) and comparison/cont rol groups.

2. Is legitimacy in policing part of the experimental intervention via

a. A directive from supervisor(s)? Y/N

Provide detail of directive

b. Training? Y/N

Provide detail of training________________________________

Answer yes, no (or dont know). Provide details as to the directive or training involved in the intervention. If both directive and training are part of the experimental intervention, provide details on both.

126 The Campbell Collaboration | www.campbellcollaboration.org

c. Organisational structure innovation? Y/N

d. Stated in another paper (Provide name of experiment and first authors name) :

3. Does the intervention include elements of:

a. citizen participation? Y/N

b. neutrality in decision making? Y/N

c. demonstration of dignity and respect throughout the interaction? Y/N

d. other (please specify):

Answer yes, no or dont know. These are all elements of procedural justice that are discussed in the literature as ways to bring about enhanced legitimacy in policing.

4. At what unit of analysis was the treatment delivered/intervention directed towards?

Describe the unit at which the intervention is directed.

a. Individual(s)

i. Citizens/public

ii. Offenders

iii. Victims

iv. Students

b. Group(s)

i. Communities

ii. Beats

iii. Neighborhoods

iv. Organizations

c. Third Party(ies)

i. Religious leaders

ii. Community leaders

iii. Government

127 The Campbell Collaboration | www.campbellcollaboration.org

v. Young people

vi. Other:______

v. Groups

vi. Gatherings/even ts

vii. Other:______

bodies

iv. Business owners or managers

v. Parents/Guardian s

vi. Place Managers

vii. Other:_________

5. Intervention Context:

a. responding to calls for service

b. routine patrols

c. investigations

d. warrant searches, raids

e. field contacts, street pops

f. official cautions

g. suspect, witness, victim interviewing

h. traffic stops

i. reporting of an incident

j. problem oriented policing

k. conferencing

l. crackdowns

m. directed patrols

n. intelligence-led policing

o. crime prevention interventions (e.g. school based programs, Neighborhood Watch)

The intervention was conducted in which context/situation(s). Circle all that apply.

128 The Campbell Collaboration | www.campbellcollaboration.org

p. crowd control

q. Other (specify):

r. Unknown/not stated or unclear

6. Was the treatment assigned

a. Pre-police contact

b. Post-police contact

Were participants assigned to treatment/control groups before or after they first encountered police in this study (e.g. before/after initial stop, before/after arrest)

7. In what setting is the intervention delivered:

a. Home

b. School

c. Community

d. Prison or government facility

e. Other (specify):

f. Unknown/not stated or unclear

g. Stated in another paper

Where was the intervention delivered?

8. In addition to the police, who else is involved in the delivery of the intervention (tick all that apply):

a. No one else involved in the delivery of the intervention

b. Third Parties

c. Academic researchers

d. Individual(s)

e. Group(s)

f. Other (specify)

g. Unknown/not stated or unclear

Were other individuals or groups involved in delivering the intervention?

129 The Campbell Collaboration | www.campbellcollaboration.org

h. Stated in another paper

9. Was the duration of the intervention indicated (in minutes, days or months)

a. Yes (specify duration in minutes, days or months):

b. No

c. Unknown/not stated or unclear

d. Stated in another paper

Indicate the length of the intervention (as opposed to the research). For example, if the intervention is a conference averaging 2 hrs, list the duration as 120 minutes.

10. Does the study describe the intervention has being delivered in a regimented and/or structured way?

a. Yes

b. No

c. Unknown/not stated or unclear

d. Stated in another paper

Did the study refer to the intervention as being highly structured or regimented to ensure consistency of delivery?

11. Was the delivery of the intervention consistent over time?

a. Yes

b. No: Specify

c. Unknown/not stated or unclear

d. Stated in another paper

If the intervention was not consistently delivered, detail why, using additional pages if necessary.

INSTRUCTIONS Page

/paragra ph referenc e

VI. UNIT OF ANALYSIS

1. What was the unit of analysis for the evaluation of the intervention?

This the primary unit at which the

130 The Campbell Collaboration | www.campbellcollaboration.org

a. Individual(s)

i. Citizens/p ublic

ii. Offenders

iii. Victims

iv. Students

v. Young people

vi. Other:___ ___

b. Group(s)

i. Communities

ii. Beats

iii. Neighborhood s

iv. Organizations

v. Groups

vi. Gatherings/ev ents

vii. Other:______

c.Third Party(ies)

i. Religious leaders

ii. Community leaders

iii. Government bodies

iv. Business owners or managers

v. Parents/Guardi ans

vi. Place Managers

vii. Other:______

intervention was delivered and evaluated. Tick the one primary unit only.

2. Did the researchers collect nested data within the unit of analysis? Y/N

If yes, please indicate the nested unit of analysis?

For example, one could imagine a study where procedural justice principles are used by police in beats or neighborhoods. The unit of analysis may be police interventions in communities, but data are collected about individual citizens encounters with police.

3. Describe the manner in which the sample participants (i.e. individuals, groups or third parties) were recruited into the study:

a. Stated in another paper

Provide a detailed explanation as to how the cases in the study were recruited.

131 The Campbell Collaboration | www.campbellcollaboration.org

4.What was the initial sample size (N) recruited into the study and what was the final N

a. Initial N:

b. Final N:

c. Stated in another paper

5.Gender composition of sample:

b. Male: N__

c. Female: N___

d. Unknown/not stated or unclear

e. Stated in another paper

Indicate whether males and/or female participants were involved in the study. Where given, provide the breakdown of gender for sample. E.g. List the number of males/females involved in study.

83

6. Age composition of sample:

a. Only adults (18+)

b. Only juveniles (under 18)

c. Mix of both juveniles and adults (specify age range)

d. Unknown/not stated or unclear

e. Stated in another paper

Where given, circle the relevant age group(s) of sample involved in the intervention. Provide the N and percentage if available.

84

7. Other demographic characteristics of the sample: (e.g. socioeconomic status, employment profile,

If provided, list any other demographic characteristics of the target population.

132 The Campbell Collaboration | www.campbellcollaboration.org

8. Race/ethnicity of the sample. N or percentage

a. White: N percentage

b. Black: N______percentage______

c. Asian: N______percentage______

d. Indigenous: N______percentage_______

e. Other: N______percentage______

f. Unknown/unclear: N______percentage______

g. Stated in another paper

h. Not stated

Circle the race/ethnicity of sample and list the N and/or percentage if provided.

9. Were adjustments made for baseline differences?

a. Yes (specify)

b. No

c. Unknown/not stated or unclear

d. Stated in another paper

Were adjustments made for any baseline (e.g. gender, race, age) differences in the sample between experimental and control /comparison group. If yes, specify adjustments.

10. Was there sample attrition?

a. Yes (specify):

b. No

C. Stated in another paper

Did the study report any attrition of the sample? If yes, provide details of attrition.

11. Where adjustments made for attrition?

a. Yes (specify):

b. No

If attrition was a factor in the study, were adjustments made and if so what adjustments

133 The Campbell Collaboration | www.campbellcollaboration.org

c. Unknown/not stated or unclear

d. Not applicable

e. Stated in another paper

were made?

12. Where adjustments made for differential attrition?

a. Yes (specify):

b. No

c. Unknown/not stated or unclear

d. Not applicable

e. Stated in another paper

If there was differential attrition (e.g. more participants pulled out of the experimental than control condition) were adjustments made? If yes, specify what adjustments were made.

13. In what country did the study take place?

A. USA

b. Canada

c. South America

d. United Kingdom

e. Europe

f. Asia

g. Middle East

h. Africa

i. Australia

j. Pacific

k. Other

l. Stated in another paper

Circle the country in which the study took place. Note that this may be different to the country from which the Author(s) institution/organiz ation is based (e.g. study takes place in Australia but Author(s) live in USA).

VII. RESEARCH DESIGN & QUALITY ASSESSMENT OF STUDYS METHODOLOGY

INSTRUCTIONS Page / paragraph reference

1. Type of study:

a. Randomized experiment

Refer to LPP section 3.1.2 (types of study designs) for more information

134 The Campbell Collaboration | www.campbellcollaboration.org

b. Quasi-experimental comparison group, with pre-test

c. Quasi-experimental comparison group, without a pre-test

d. Quasi-experimental interrupted time series

e. Other:_______________________

Note, Other studies are NOT eligible for analysis but may be eligible for literature review and inventory of interventions aimed at increasing legitimacy in policing.

regarding the experimental and control conditions.

2. Did the author(s) identify any mistakes that occured in the way in which participants were assigned to the intervention (experimental) or comparison (control) group?

a. Yes (specify):

b. No

c. Unknown/not stated or unclear

d. Stated in another paper

If errors occurred (for example, an error occurred in the random assignment of sample to intervention or control condition), describe the mistake.

3. If a mistake did occur (yes to question 2), indicate how the mistake was corrected if at all

a. Not applicable

b. Mistake was corrected by (specify):

c. Mistake did occur but was not corrected

d. Unknown/not stated or unclear

e. Stated in another paper

Provide detail on how the mistake by which participants were assigned to the intervention/experimental or comparison group was dealt with by researchers in study.

4. Was there any sample bias in the experimental design?

a. Yes (specify):

b. No

c. Unknown/not stated or unclear

d. Stated in another paper

Did the study design suggest any bias in the way the sample was chosen and/or assigned (e.g. females assigned to the experimental group because researchers thought they would be more responsive)?

135 The Campbell Collaboration | www.campbellcollaboration.org

5. Was there any attrition bias?

a. Yes (specify):

b. No

c. Not applicable (e.g. there was not attrition listed in section VI, question 9)

d. Unknown/not stated or unclear

e. Stated in another paper

If attrition did occur, was it biased towards a specific group or condition (e.g. larger attrition amongst females than males)?

6. Was equivalency between groups established or discussed?

a. Yes (specify):

b. No

c. Unknown/not stated or unclear

d. Stated in another paper

Did researchers comment on whether equivalency between experimental and comparison group had been reviewed or established? For example, if the sample is small, it should not be assumed that randomly assigning groups created equivalency (but can be assumed it minimized selection bias).

Did researchers ensure that the experimental group ONLY received the intervention so that meaningful comparisons could be made between experimental and control (e.g. were there contamination effects because people in the experimental group were interacting with people in the comparison group?

7. Was there integrity in maintaining intervention delivery?

a. Yes (specify):

b. No

c. Unknown/not stated or unclear

d. Stated in another paper

8. Were steps taken to ensure treatment integrity?

a. Yes (specify):

b. No

Did researchers strive to ensure that the experimental condition and comparison group received the intended intervention? If so, detail

136 The Campbell Collaboration | www.campbellcollaboration.org

c. Unknown/not stated or unclear

d. Stated in another paper

the actions taken.

9. Was there a level of monitoring of treatment delivery?

a. Yes (specify):

b. No

c. Unknown/not stated or unclear

d. Stated in another paper

Did researchers strive to ensure the intervention was delivered as intended throughout the experiment (e.g. if police were required to deliver a message in the experimental group, how did researchers check to make sure the message was delivered throughout the study)?

10. Was there consistency of intervention periods and follow-up post intervention time frames both within and between experimental and comparison group?

a. Yes (specify):

b. No

c. Unknown/not stated or unclear

d. Stated in another paper

11. Was analysis conducted as intention to treat?

a. Yes (specify):

b. No

c. Unknown/not stated or unclear

d. Stated in another paper

Was analysis conducted on the intent to treat (e.g. on the actual treatment participants received as opposed to the treatment participants were randomly assigned to.

12. Were research standards adhered to in terms of gathering outcome data?

a. Yes :

b. No (specify):

c. Unknown/not stated or unclear

137 The Campbell Collaboration | www.campbellcollaboration.org

d. Stated in another paper

VIII. OUTCOME(S) REPORTED

INSTRUCTIONS Page / paragraph reference

1. How many evaluation outcome measures are reported in the study?

List the total number of outcomes reported

2. Is the primary outcome:

a. A direct outcome of legitimacy policing (e.g. satisfaction, cooperation, compliance)

b. An indirect outcome of legitimacy policing (e.g. re-offending, crime, disorder)

Are the main effects of the study describing direct outcomes of interventions aimed to increase legitimacy or indirect outcomes?

3. List the direct outcome(s) presented in the study (circle all that apply)

a. No direct outcomes presented

b. Satisfaction

c. Compliance

d. Cooperation

e. Establishment of social ties

f. Procedural fairness

g. Perceived legitimacy

List only the outcomes that are of interest to the review (i.e. listed here)

4. List the indirect outcome(s) presented in the study

a. No indirect outcomes presented

b. Reduction in reoffending

c. Reduction in crime

d. Reduction in disorder

List only the outcomes that are of interest to the review (i.e. listed here)

5. Were the outcomes initially intended as the outcomes of the study

a. Yes

Did the authors indicate that any of the outcomes were not initially part of

138 The Campbell Collaboration | www.campbellcollaboration.org

b. No, explain why

c. Unclear/not stated or unknown:

the original design.

6. List any other outcomes measured in the study that are not of interest to the review

No other outcomes

List any outcomes measured by the authors that are not included in the list of direct or indirect outcomes in questions 3 and 4 of this section

IX. OUTCOME VARIABLES

Code Separately for EACH outcome variable

OUTCOME NUMBER:________

( PLEASE ASSIGN EACH OUTCOME MEASURE NUMBER INCREMENTALLY SO THAT EACH OUTCOME REPORTED IN A STUDY HAS ITS OWN UNIQUE IDENTIFIER)

INSTRUCTIONS

NOTE: Print out separate tables for this section, insert the outcome measure number.

Page / paragraph reference

1. What ONE outcome/dependent variable is being described in this section

a) Direct Y/N

Specify type of direct:

Cooperation? Y/N

Satisfaction? Y/N

Compliance? Y/N

Social Ties? Y/N

Other? (specify)______________

b) Indirect Y/N

Specify type of indirect:

Specify which outcome is being referred to in this section (e.g. direct: satisfaction).

139 The Campbell Collaboration | www.campbellcollaboration.org

Crime? Y/N

Disorder? Y/N

Reoffending? Y/N

Other? (specify)_______________

2. What is the in-text definition used for this outcome?

Copy the definition of this outcome used in the text of the article

3. Describe other measurement issues related to this outcome measure: (i.e. the items in the survey that were used in an index; the alpha reliability of the index etc)

Page number depicting the operationalization of the outcome construct:

Provide as much information as possible about the way the construct was measured. Some of this information may be found in a footnote or in an endnote.

Include a list of items used to measure the outcome if possible.

4. If direct, does this outcome relate to:

a. Police as an institution

b. An individual police officer

c. A process initiated and controlled by police

For example, is satisfaction with police as an institution, satisfaction with a police officer, or satisfaction with a police-run process measured?

5. What type of data was used to measure this outcome?

a. Official data

i. Police calls for service

ii. Police incidents

iii. Police arrests

iv. Complaints against police

v. Courts

vi. Prison

vii. Other official data (specify):

Provide details on the type of data used to report on this outcome. E.g. if reoffending data were collected from police systems.

140 The Campbell Collaboration | www.campbellcollaboration.org

b. Survey data

c. Interview data

d. Self-reports

e. Other data (specify):

6. What is the level of measurement for the variable used to assess THIS outcome:

a. Dichotomous

b. Continuous

c. Ordinal

d. Combination

e. Other (specify):

Variable refers to the data used to measure the outcome listed in question 2 and 3.

7. Did the researcher(s) assess the quality of the data and/or variable collected?

a. Yes (specify):

b. No

c. Unclear/not stated or unknown

Did the study researcher indicate that the quality of the data used to measure this outcome was assessed? For example, did authors describe a process of data cleaning or cross checking of data?

8. Did the researcher(s) express any concern regarding the quality of the data and/or variable?

a. Yes (specify):

b. No

c. Unclear/not stated or unknown

Did authors describe any problems regarding data used to measure this outcome variable?

9. Does the study variable correspond to the initial stated research question?

a. Yes

b. No (describe the discrepancy):

c. Unclear/unknown

141 The Campbell Collaboration | www.campbellcollaboration.org

X. EFFECT SIZE/REPORTS OF STATISTICAL SIGNIFICANCE

Code separately for EACH outcome variable

INSTRUCTIONS

The effect size/reports of statistical significance should correspond to the outcome described in Section IX ONLY.

Page / paragraph reference

1. What is the total sample size (N) in the analysis? N=______________

List the total sample size relating to the outcome listed in section IX, Question 2. This may be different than the sample size listed in section VI (Sample characteristics)

2. What is the total sample size of the experimental group? N=

As above but relating to the experimental/treatment group.

3. What is the total sample size of the control/comparison group? N=_______

As above but relating to the control/comparison group.

4. Was attrition a problem for this outcome?

a. Yes (provide details):

b. No

c. Unclear/not stated or unknown

Indicate whether attrition was a problem for this outcome as opposed to the sample generally (as detailed in section VI: Sample characteristics).

5. Raw difference favors (e.g. shows more success for):

a. Experimental group

b. Comparison group

.

Do the results favor the experimental or control group

6. Did a test of statistical significance indicate statistically significant differences between either control or treatment group or the pre and post tested treatment group?

a. Yes (specify significance level (e.g. p=.05):

142 The Campbell Collaboration | www.campbellcollaboration.org

b. No

c. Unclear/unknown

7. Was a standardized effect size reported

a. Yes

i. What was the effect size measure used:

ii. What was the effect size:

iii. What page was the effect size found

b. No

i. Are data available to calculate an effect size (effect size can be calculated form mean and standard deviations, t or F value, Chi-square, frequencies or proportions (dichotomous or polychotomous), pre and post etc.

1. Yes

2. No (proceed to requesting missing information from study authors)

c. Unclear/unknown

8. Did the evaluation control for validity by using multivariate methods (i.e. regression) to assess the impact of an intervention/program on a given outcome

a. Yes

b. No

c. Unclear/unknown

This is only an issue for quasi-experimental designs but is being coded for all designs.

XI. DATA INSTRUCTIONS Page /

143 The Campbell Collaboration | www.campbellcollaboration.org

paragraph reference

1. Treatment group N=

2. Control group N=

3. Treatment Group mean

4. Control group mean

5. Treatment group standard deviation

6. Control group standard deviation

Proportions or frequencies

1. n of treatment group with a successful outcome _______

2. n of control group with a successful outcome ________

3. Proportion of treatment group with a successful outcome _______

4. Proportion of control group with a successful outcome ________

Significance tests

1. t-value ________

2. F-value _________

3. Chi-square value _______

4. Calculated effect size _________

Regression output

Copy and paste regression output here

144 The Campbell Collaboration | www.campbellcollaboration.org

XII. CONCLUSIONS BY AUTHORS

Note: This section provides detail by authors regarding the effectiveness of the intervention with respect to the outcome/problem being addressed on this coding sheet.

INSTRUCTIONS Page / paragraph reference

1. Conclusion about the direct and indirect impact of the intervention

a. Authors conclude no enhanced direct effect

b. Unclear/no conclusion provided

c. Authors conclude problem (e.g. crime and disorder) declined

d. Authors concluded problem (e.g. crime and disorder) did not decline

Circle option that describes authors conclusions regarding the direct and indirect impact of the intervention.

2. Did the authors conclude that the intervention was beneficial?

a. Yes

b. No

c. Unknown/not stated or unclear

Circle option that denotes authors conclusions regarding if the intervention was beneficial

3. Did the authors conclude that a relationship existed between the intervention and a stated problem (e.g. satisfaction, cooperation, compliance, crime and disorder)

a. Yes

b. No

c. Unknown/not stated or unclear

Circle the option that denotes authors conclusions regarding whether a relationship exists between the intervention and a stated problem.

145 The Campbell Collaboration | www.campbellcollaboration.org

XIII. Notes and Questions

Note This section provides a working space for reviewers to note questions and comments for issues that may need detailed revision and discussion

INSTRUCTIONS Page / paragraph reference

Write any comments on the article focus, methodology, or outcome measures that may affect analysis or interpretation.

Write any questions or areas of the coding where information was uncertain.

Indicate the main aims of the article if

146 The Campbell Collaboration | www.campbellcollaboration.org