Police performance measurement: an annotated bibliography: [Doc 11]

Abstract

This study provides information to assist those involved in performance measurement in police organisations. The strategies used to identify the literature are described. Thematic sections cover; general overviews; methodological issues; performance management in other industries; national, international and cross-national studies; frameworks (e.g. Compstat; the Balanced Scorecard); criticisms (particularly unintended consequences); crime-specific measures; practitioner guides; performance evaluation of individual staff; police department plans and evaluations; annotated bibliographies in related areas, and; other literature. Our discussion offers two conclusions: the measures best aligned with performance are typically more expensive, while most operational data should only provide contextual information; the philosophy of open governance should be pursued to promote transparency, accountability and communication to improve police performance.

Tiwana et al. Crime Science (2015) 4:1 DOI 10.1186/s40163-014-0011-4

RESEARCH Open Access

Police performance measurement: an annotated bibliography

Neena Tiwana*, Gary Bass and Graham Farrell

Abstract

This study provides information to assist those involved in performance measurement in police organisations. The strategies used to identify the literature are described. Thematic sections cover; general overviews; methodological issues; performance management in other industries; national, international and cross-national studies; frameworks (e.g. Compstat; the Balanced Scorecard); criticisms (particularly unintended consequences); crime-specific measures; practitioner guides; performance evaluation of individual staff; police department plans and evaluations; annotated bibliographies in related areas, and; other literature. Our discussion offers two conclusions: the measures best aligned with performance are typically more expensive, while most operational data should only provide contextual information; the philosophy of open governance should be pursued to promote transparency, accountability and communication to improve police performance.

Keywords: Policing; Performance management; Police performance management; Performance indicators; Compstat; Balanced scorecard

Background

Performance measurement is an issue of growing importance in the public and private sectors in many countries.It is a complex area in which there is no consensus over either the form or nature of what should be measured, how measurement should take place, what different indicators mean, and how they are best used to promote improved performance. However, there is consensus on the fact that performance measures are a potentially powerful policy instrument, with potentially tremendous impact on police work. At the same time there is an emerging consensus that the development and use of performance measures can be problematic. It is this paradox the difficulty and potential pitfalls of wielding a powerful instrument - that makes the topic problematic.

The nature of policing means that, while its performance measures overlap with those in other industries, they also raise significantly different methodological and substantive issues. Hence the justification for this annotated bibliography is twofold. First, there is a large body of specialised work on this topic that has not, to our knowledge, been systematically collated. Second,

there are issues particular to policing that are not addressed in other annotated bibliographies on performance measurement.

The concluding section contains our brief reflections on what works. It distills the evidence from the studies contained herein in an effort to identify the key ingredients of a recipe for success in the development and use of performance measures for policing. We recommend caution as there are few good indicators of performance and those that exist tend to be the more difficult and expensive to collect, often based on surveys. The more readily available and cheaper indicators based on routinely-collected operational and administrative data can provide informative contextual indicators but are often partial, at best, indicators of performance. There is also evidence that performance measures can lead to unexpected side-effects, including manipulation of statistics and resources by police officers in an effort to meet performance expectations. Hence our conclusion is that the philosophy of open governance should be pursued, making many contextual indicators widely available to promote transparency, comparison, accountability and communication, alongside a relatively small set of core performance indicators.

* Correspondence: mailto:[email protected]

Web End [email protected] School of Criminology, Institute for Canadian Urban Research Studies, Simon Fraser University, 8888 University Drive, Burnaby V5A 1S6, Canada

2014 Tiwana et al.; licensee Springer. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0

Web End =http://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited.

Tiwana et al. Crime Science (2015) 4:1 Page 2 of 28

Methods

The present study was undertaken as part of a collaboration with the Police Services Division of the Ministry of Justice of British Columbia. It emerged from work examining the feasibility of developing performance measures for British Columbia. The main literature searches were undertaken between July 2011 and November 2012, with more recent studies added when they were identified during the further stages of preparation.

We sought to capture the English language literature by all appropriate means. Multiple search strategies were employed. Online search engines were used, but particularly the excellent meta-search resources of the university library. Key search terms used in combination with police and policing and with each other were: performance, measurement, productivity, Compstat, and indicator. However the majority of the documents were identified through an iterative process of snowballing from existing reference lists.

Variation in terminology, and in whether or not performance was the main or secondary focus of a study, meant the identification of studies was not straightforward. For instance, some earlier studies prefer the term productivity to performance. This is one reason why snowballing was an important component of the search method.

The bibliography is organised into sections. This is preferable to an alphabetical listing by author surname because it groups related studies. However, some sections overlap and many studies could have been in more than one section. Each section has a brief introduction that provides some context for the individual studies.

Results

The annotated studies are included here as the findings of the search. They are followed by a short discussion and conclusion.

General OverviewsThis section identifies key overviews of the topic. They are probably the studies to read first if you are new to the topic. One overview relating to criminal justice is included. The reviews indicate the complexity of the area. Davis (2012) provides the broadest geographical coverage, and synthesises best practices internationally. Maguires (2003 and 2004) two part article provides a chronology of police performance measurement and introduction to key issues. Book includes chapters addressing various features of policing that need to be considered when developing performance measures. Maguire and Uchidas (2000) review essay examines the measurement and explanations of the various features of police organizations. The Vera Institute of Justice (2003) review locates policing within justice sector performance measures.

Davis, R.C. (2012). Selected international best practices in police performance measurement. Santa Monica, CA: RAND Corporation

This thoughtful study provides an excellent and accessible introduction. It covers key definitions such as outputs and outcomes, the ambiguity that exists in interpreting some indicators, contextual factors, and the issue of performance incentives for police departments. It gives useful and sometimes contrasting examples of performance measures used in England and Wales, Northern Ireland, New Zealand, and South Africa.

Maguire, E.R. (2003, September). Measuring the performance of law enforcement agencies: Part1 of a 2-part article. CALEA Update Magazine, 83. http://www.calea.org/calea-update-magazine/issue-83/measuring-performance-law-enforcement-agencies-part-1of-2-oart-articl

Web End =http://www. http://www.calea.org/calea-update-magazine/issue-83/measuring-performance-law-enforcement-agencies-part-1of-2-oart-articl

Web End =calea.org/calea-update-magazine/issue-83/measuring- http://www.calea.org/calea-update-magazine/issue-83/measuring-performance-law-enforcement-agencies-part-1of-2-oart-articl

Web End =performance-law-enforcement-agencies-part-1of-2-oart http://www.calea.org/calea-update-magazine/issue-83/measuring-performance-law-enforcement-agencies-part-1of-2-oart-articl

Web End =articl (Accessed: 14 October 2013).

This is Part 1 of Maguires very readable review, and Part 2 is listed next. It provides a brief history of police performance measurement in the United States and reviews criticisms of traditional performance indicators:1. Crime rates, 2. Arrests and citations, 3. Clearance, 4. Response times. It reviews the dimensions of policing (the categories or domains of police departments) that it suggests should inform the development of performance measures.

Maguire, E.R. (2004, February). Measuring the performance of law enforcement agencies: Part 2. CALEA Update Magazine, 84. http://www.calea.org/calea-update-magazine/issue-84/measuring-performance-law-enforcement-agencies-part-2-2-part-article

Web End =http://www.calea. http://www.calea.org/calea-update-magazine/issue-84/measuring-performance-law-enforcement-agencies-part-2-2-part-article

Web End =org/calea-update-magazine/issue-84/measuring- http://www.calea.org/calea-update-magazine/issue-84/measuring-performance-law-enforcement-agencies-part-2-2-part-article

Web End =performance-law-enforcement-agencies-part-2-2-part- http://www.calea.org/calea-update-magazine/issue-84/measuring-performance-law-enforcement-agencies-part-2-2-part-article

Web End =article . (Accessed: 14 October 2013).

This is Part 2 to Maguire (2003) and addresses the development, pilot-testing, and a step-by-step implementation process for police performance measures. It suggests that the goals of the police organisation and dimensions of policing must be established before performance measures. It identifies sources in addition to police administrative data that can be used including: general community surveys, contact surveys, employee surveys, direct observation, and independent testing or simulation studies (e.g. the mystery shopper methodology used in market research to examine staff-public interaction). It recommends comparing police forces through stratification or grouping of similar forces.

Golding, B. and Savage, S. (2008). Leadership and performance management. In T. Newburn (Ed.),

Tiwana et al. Crime Science (2015) 4:1 Page 3 of 28

Handbook of Policing. (2nd ed., pp. 725-759). Portland, Oregon: Willan.

This book chapter gives a wide-ranging introduction, insightful history and assessment of police leadership and performance management in Britain. That country has, due to over two decades of government-backed police performance measurement regimes, produced a disproportionate amount of published results, transparency, studies and debate.

T. Prenzler (Ed.) (2012). Policing and security in practice: challenges and achievements. New York, NY: Palgrave MacMillan. (pp. 1-19).

Four of this edited books 12 chapters examine police performance measures: 1. Policing to a Different Beat: Measuring Police Performance; 2. Legitimacy and Policing;6. The Effectiveness of Traffic Policing in Reducing Traffic Crashes, and; 7. Approaches to Improving Organizational Effectiveness: The Impact of Attraction, Selection and Leadership Practices in Policing. While perhaps going beyond the level of the novice reader, together these chapters address many contemporary issues of relevance.

Maguire, E.R. and C.D. Uchida (2000). Measurement and Explanation in the Comparative Study of American Police Organizations. In D. Duffee (ed.), Criminal Justice 2000: Vol 4. Measurement and Analysis of Crime and Justice. Rockville, MD: National Criminal Justice Reference Service.

This is an academic review with a methodological focus on measurement and explanation of differences between US police forces. It examines the structure and organisation of police organisations and their function. It is included here because measurement issues, in the context of policing, are its focus.

Vera Institute of Justice. (2003). Measuring progress toward safety and justice: A global guide to the design of performance indicators across the justice sector. New York, NY: Vera Institute of Justice.

This study locates policing within a broader examination of performance measurement in the justice sector. It examines strategic, institutional, and activity-based indicators and the process for developing performance indicators. Its target audience is programme managers in policing, ministries of justice (prosecution and legal aid), judicial performance, non-custodial sentencing, prisons, and accountability mechanisms (e.g. ombudsmen). It aims to address both public and private sector components of the justice system.

Early Works: Pre-1990sThe study of police performance measurement expanded rapidly from the early 1990s. Hence the contribution of this section is to identify key earlier studies. While there are many studies of the history of policing in general, the focus here remain on performance measures.

Hoffman (1971) criticises what even then appear as limited traditional police performance measures such as clearance rates, response times and examines public satisfaction. Ostrom (1973) proposes indicators to measure police output and efficiency. Skogan (1976) defines efficiency and effectiveness and how to achieve them. Bouza (1978) touches on the various facets of performance assessment, and Larson (1978) examines police unionism. Decker (1980) examines police productivity using indicators of output, while Stevens, Webster and Stipak (1980) focus on response times. Carlson and Sutton (1981) and Stevens and McDavid (1981) compare how police and citizens define good performance. Similarly, Brown and Coulter (1983) examine the influence of subjective and objective measures in assessing police performance and Parks (1984) looks at the link between the objective and subjective indicators by studying response times. Pollitt (1986) locates policing in the broader context of the history of public sector performance management.

Hoffman, R.B. (1971). Performance measurements in crime control. Journal of Research in Crime and Delinquency, 8(2), 165-174. doi:10.1177/002242787100800205.

This study focuses on clearance rates as performance measures and proposes a broader conceptual framework that shifts the focus away from enforcing laws and arresting offenders. The framework suggests indicators in five areas: economic disincentives to criminal activities and markets; crime prevention; direct measures on operating performance and individual officer performance; community assistance and control, and; quality of life.

Ostrom, E. (1973). On the meaning and measurement of output and efficiency in the provision of urban police services. Journal of Criminal Justice, 1, 93-112.

The study examines terminology and defines output and efficiency in relation to policing. It then develops indicators relating to output and efficiency that are based on policing activities.

Skogan, W.G. (1976). Efficiency and effectiveness in big-city police departments. Public Administration Review, 36(3), 278-286.

Tiwana et al. Crime Science (2015) 4:1 Page 4 of 28

The study defines effectiveness as an organizations ability to transform inputs to outputs, and efficiency as producing desired inputs while keeping costs low. It offers recommendations to ensure effective and efficient police organizations including recruiting ethnically representative officers, hiring civilians, and employing technology such as computers to assist with performing duties.

Bouza, A.V. (1978). Police administration: Organization and performance. Elmsford, New York: Pergamon Press Inc.

Written by the then Deputy Chief of New York City Transit Police, the first part of the book deals with police organisation and the second part with performance. Performance areas addressed include productivity, integrity, accountability, morale and police unions, community relations and the press, and the criminal justice system.

Larson, R.C. (1978). Police accountability: Performance measures and unionism. Cambridge, MA: Massachusetts Institute of Technology.

This is an edited book with chapters that address the historical development of police performance measures. It has a particular focus on issues relevant to unionism in policing which is important because they relate to whether and how performance measures are developed.

Decker, S.H. (1980). The effect of police characteristics on alternative measures of police output. Criminal Justice Review, 5(2),34-40. doi:10.1177/ 073401688000500206.

The study assesses police productivity using three output indicators: clearance ratio, detection ratio, and effectiveness ratio. Productivity is assessed by examining the relationship between five variables (expenditures, manpower, municipal police effort, technological innovations, and police reform) and nine crime categories under each output indicator (rape, robbery, aggravated assault, burglary, larceny, motor vehicle theft, property crime, violent crime, and overall crime).

Stevens, J.M., Webster, T.C., and Stipak, B. (1980). Response time: Role in assessing police performance. Public Productivity Review, 4(3), 210-230.

This article examines response time as an indicator of police performance in clearing reported crimes. It finds little relationship between response times and clearance rates. Some Part 1 and Part 2 crimes were more likely to

be cleared with faster response times, but it was unclear whether this was due to selection bias by responding officers it is suggested police may respond quicker when a clearance is more likely.

Carlson, H.M., and Sutton, M.S. (1981). A multi-method approach to community evaluation of police performance. Journal of Criminal Justice, 9(3), 227-234.

This innovative study examines whether the definition of good police performance is comparable across police officers, police supervisors, and citizens. It finds that negative ratings by police are similar to those of citizens. However, the definition of good performance held by police was not shared by citizens. This suggests a fundamental mis-match of importance for performance measurement.

Stevens, J.M., and McDavid, J.C. (1981). Urban police performance attitudes. Computers, Environment and Urban Systems, 6(4), 157-169.

This study examines police officers attitudes towards their performance. It seeks to identify factors influencing this attitudes, using a multivariate perspective. It finds that occupational and public factors influence police attitudes, that the police do not agree that citizen and victim input is valuable to police effectiveness, while citizen ratings and police officer self-ratings of performance are related.

Brown, K. and Coulter, P.B. (1983). Subjective and objective measures of police service delivery. Public Administration Review, 43(1), 50-58.

This study highlights the distinction between public satisfaction and quality of police service delivered. It uses objective administrative measures of effectiveness, efficiency, and equity of police agencies, and subjective measures of attitudes based on user surveys. It finds citizen satisfaction is not related to level of police service delivery.

Parks, R.B. (1984). Linking objective and subjective measures of performance. Public Administration Review, 44(2), 118-127.

Parks (1984) examines the relationship between (objective) indicators of police response capabilities and the publics (subjective) perception. Citizens with a recent experience with police response time were compared to citizens who did not. It found that in neighbourhoods with greater patrol car visibility, citizens perceived shorter

Tiwana et al. Crime Science (2015) 4:1 Page 5 of 28

police response times, and that a longer response time results in dissatisfaction with police actions. This implies that increased neighbourhood patrol contributes to higher satisfaction among citizens in their perception of police response times.

Pollitt, C. (1986). Performance measurement in the public services: some political implications. Parliamentary Affairs, 39(3), 315-329.

This article provides an overview of public sector performance measurement in the United Kingdom from the late 1970s. The importance of the study is to locate the development of police performance measures in that broader historical context.

Methodological IssuesThere are a range of methodological issues and challenges associated with developing indicators to measure police performance. This section identifies key areas and readings.

GeneralAshby (2005) examines the importance of geography when developing police performance frameworks, and Charbonneau and Riccucci (2008) outline the importance of social equity indicators. Langworthy (1999) is an edited book with many chapters by leading US scholars working on measuring what matters in policing. Moore and Braga (2003a, b) present seven key dimensions of policing by which police performance should be measured. Vollaards (2006) dissertation argues for the implementation of incentives in policing according to performance. Coleman (2012) describes an innovative survey of stake-holders to solicit their views on what aspects of police work should be measured.

Ashby, D.I. (2005). Policing neighbourhoods: Exploring the geographies of crime, policing and performance assessment. Policing and Society: An International Journal of Research and Policy, 15(4), 413-447. doi:10.1080/10439460500310079.

This study examines geographical issues relating to performance measurement. In particular it considers area size and comparability on different dimensions.

Charbonneau, E., and Riccucci, N.M. (2008). Beyond the usual suspects: An analysis of the performance measurement literature on social equity indicators in policing. Public Performance and Management Review, 31(4), 604-620. doi:10.2753/ pmr153-9576310405.

The study suggest indicators of social equity meaning the fair treatment of persons, akin to what is elsewhere termed procedural justice - should be included in measures of the effectiveness and efficiency of police performance as it relates to community policing. It contains a useful summary table of performance measures used in other studies.

Langworthy, R.H. (Ed.). (1999). Measuring what matters: Proceedings from the policing research institute meetings. Rockville, MD: National Institute of Justice, Office of Justice Programs, U.S. Department of Justice.

This NIJ report is an edited book with chapters by leading US policing scholars and practitioners. It draws on their work and discussion papers developed for meetings hosted by the National Institute of Justice (NIJ) and the Office of Community Oriented Policing Services (COPS) resulting in the Policing Research Institute. Many of the chapters could really be separate entries in this bibliography.

Moore, M.H. and Braga, A. (2003). The bottom line of policing: What citizens should value (and measure!) in police performance. Washington, DC: Police Executive Research Forum

This insightful study highlights key issues to consider when assessing police performance. It identifies seven dimensions of policing which should be valued by the public, and measured; the victimization rate; calling offenders to account; fear and insecurity; safety in public spaces; fair use of force; economic use of public funds, and; quality of service.

Vollaard, B.A. (2006). Police effectiveness: Measurement and Incentives. (Doctoral dissertation). Retrieved from RAND Corporation.

This is a published doctoral dissertation. It examines how to measure police performance with a focus on how government might incentivise it using rewards.

Coleman, T.G. (2012). A model for improving the strategic measurement and management of policing: The police organizational performance index (POPI). Unpublished doctoral dissertation. University of Regina, Saskatchewan.

This is a doctoral dissertation introducing the Police Organizational Performance Index (POPI) to measure police organizational performance. It is based on a survey of policing stakeholders to solicit their views of what counts in police performance.

Tiwana et al. Crime Science (2015) 4:1 Page 6 of 28

Public Satisfaction and ConfidencePublic satisfaction with, and confidence in, policing have emerged as important performance measurement issues. This is particularly for the UK where many of the studies in this section originate. In 2008, the UK dropped all other police performance measures and used the single measure of public confidence, which Fitzgerald (2010) argues reflected a political agenda. Myhill and Beak (2008) outline how demographic factors and treatment during police encounters impact public confidence, and Jang, Joo, and Zhao (2010) examine socio-demographic factors. Stanko and Bradford (2009) outline the Metropolitan Police Service model for measuring public confidence. Myhill and Quinton (2010) find that service-oriented policing methods contribute to higher confidence in the police. Myhill, Quinton, Bradford, Poole, and Sims (2011) introduce alternative indicators to the single indicator devised from the British Crime Survey so that public confidence can be measured using local survey data, while Myhill and Bradford (2012) examine quality of police service and its impact on public confidence.

FitzGerald, M. (2010). A confidence trick? Policing: A Journal of Policy and Practice, 4(3), 298-301. doi:10.1093/police/paq025.

In 2008 the UK government eliminated all police performance targets except public confidence. This study suggests that change was driven largely by political factors. It argues that, while public confidence matters, the public must also do their part in reporting crime, co-operating in investigations, and testifying in court. It notes that as public confidence grows, the crime reporting rate increases, which reduces clearance rates in the absence of more police resources.

Myhill, A., and Beak, K. (2008). Public confidence in the police. London, UK: National Policing Improvement Agency.

This study develops models to examine public confidence in the police as measured by the British Crime Survey. It finds public confidence is strongly related to broader concerns of the local community. If the public perceived less anti-social behaviour and less crime in their communities, they were more likely to have higher confidence in police. It also highlighted procedural justice (being treated fairly and respectfully by the police in encounters) and quality of service. It did not find ethnicity generally related to confidence in the police though that of Black Caribbean citizens was significantly lower.

Jang, H., Joo, H-J., and Zhao, J. (2010). Determinants of public confidence in police: An international

perspective. Journal of Criminal Justice, 38, 57-68. doi:10.1016/j.jcrimjus.2009.11.008.

This study examines correlates of public confidence in policing. Findings include: Countries with high homicide rates have low rates of public confidence in police; Some citizens report greater confidence in police including older citizens, women, people satisfied with life and those with a conservative political orientation; Confidence was lowest among those in deviant subcultures and who viewed government as supporting elites; Satisfaction with current political developments was strongly related to satisfaction police, indicating that internationally, democracy is related to confidence in the police.

Myhill, A., and Bradford, B. (2012). Can police enhance public confidence by improving quality of service? Results from two surveys in England and Wales. Policing and Society: An International Journal of Research and Policy, 22(4), 397-425. doi:10.1080/ 10439463.2011.641551.

Conducting two surveys based on data from the National Reassurance Policing Programme (NRPP) and the British Crime Survey, the authors explore whether improvements in the quality of police service can enhance the publics trust and confidence in the police. The authors argue for a shift towards a process-based model of policing including police treatment of the public.

Myhill, A.,and Quinton, P. (2010). Confidence, neighbourhood policing, and contact: Drawing together the evidence. Policing: A Journal of Policy and Practice, 4(3), 273-281. doi:10.1093/police/paq026.

This study reviews the literature on confidence in the police. It finds that public confidence in the police is likely to increase if the police adopt a service model in their interactions with the public.

Myhill, A., Quinton, P., Bradford, B., Poole, A., and Sims, G. (2011). It depends what you mean by confident: operationalizing measures of public confidence and the role of performance indicators. Policing: A Journal of Policy and Practice, 5(2), 114-124. doi:10.1093/police/par027.

While the study supports measuring public confidence in the police, it suggests that questions from the British Crime Survey and a local police force survey do not accurately measure public confidence as institutional trust. Consequently, it is suggested that public confidence surveys should measure procedural fairness during

Tiwana et al. Crime Science (2015) 4:1 Page 7 of 28

encounters with the public, police legitimacy via whether the public feels that community values are represented by local police, and public intentions to co-operate, including the publics cooperation with providing intelligence and information and reporting crime.

Rosenbaum, D.P., Schuck, A., Lawrence, D., Hartnett,S., McDevitt, J., and Posick, C. (2011). Community-based indicators of police performance: Introducing the platforms public satisfaction survey. Washington, D.C.: National Police Research Platform, National Institute of Justice.

The study evaluates police performance via ongoing public satisfaction surveys of persons coming into contact with police (rather than a population sample). The National Police Research Platforms methodology includes police departments signing up to participate in research and receive feedback. Preliminary findings procedural justice, respondent characteristics, incident characteristics, [and] outcome vs. process (pp.7-8) are key influences upon public satisfaction.

Stanko, E.A., and Bradford, B. (2009). Beyond measuring how good a job police are doing: The MPS model of confidence in policing. Policing: A Journal of Policy and Practice, 3(4), 322-330. doi:10.1093/police/ pap047.

The authors developed the Metropolitan Police Service (MPS) model based on four key public confidence factors:1. Effectiveness - police officers ability to perform duties and responsibilities effectively; 2. Fair treatment - in their demeanour and interactions with the public; 3. Community commitment/engagement attentiveness of officers to community concerns; 4. Disorder - the extent to which police respond to disorder within communities. The authors adopted questions from the Metropolitan Polices Public Attitude Survey.

Crime RatesPolicing aims to reduce crime, but the relationship between the two remains uncertain. Eck and Maguires (2000) important review concludes that policing did not induce the crime drop in US violence of the 1990s. Kovandzic and Sloan (2002) and Worrall and Kovandzic (2010) suggest greater police number are inversely related to crime. Loveday (2000) examines clearance rates, and Levitt (1998) focuses on reporting bias that can occur when police numbers change. Seeks to pursuade politicians against using crime statistics as evidence of police performance. Shepherd and Sivarajasingam (2005) argue for the use of emergency department data.

Eck, E.J.and Maguire, E.R. (2000). Have changes in policing reduced violent crime? Assessment of the evidence. In A. Blumstein and J. Wallman (Eds.). The crime drop in America (pp. 207-265). New York, NY: Cambridge University Press.

This chapter of Blumstein and Wallmans landmark collection examines whether policing strategies contributed to the dramatic drop in US violence in the 1990s. It first examines general policing efforts that have been suggested to reduce in violent crime. It then examines focused policing strategies. It concludes that there is no real evidence that policing caused the decline in violence.

Kovandzic, T.V. and Sloan, J.J. (2002). Police levels and crime rates revisited: A county-level analysis from Florida (1980-1998). Journal of Criminal Justice, 30, 65-76.

This study examines the relationship between the number of police officers and crime in Florida counties. It finds a negative relationship between them and suggests this is consistent with two earlier studies elsewhere.

Worrall, J.L., Kovandzic, T.V. (2010). Police levels and crime rates: An instrumental variables approach. Social Science Research, 39, 506-516. doi:10.1016/j. ssresearch.2010.02.001.

This study explores the relationship between police numbers and crime levels. It finds a modest inverse association between policing levels and four crime types: homicide, robbery, assault, and burglary.

Loveday, B. (2000). Managing crime: Police use of crime data as an indicator of effectiveness. International Journal of the Sociology of Law, 28, 215-237. doi:10.1006/ijsl.2000.0124.

This study highlights limitations of crime and clearance rates as measures of police performance. It concludes that when crime and clearance rates are used to assess police performance they become a sophisticated artifice - both can be massaged and manipulated. Hence when crime is high, which could be taken to indicate poor performance, low clearance rates are used to claim lack of police resources.

Levitt, S.D. (1998). The relationship between crime reporting and police: Implications for the use of uniform crime reports. Journal of Quantitative Criminology, 14(1), 61-81.

Tiwana et al. Crime Science (2015) 4:1 Page 8 of 28

This study examines the impact of reporting bias on the relationship between the police and crime rates. Reporting bias occurs when an increase in police numbers results in an increase in victimization reports despite decreases in victimization rates.

Shepherd, J., and Sivarajasingam, V. (2005). Injury research explains conflicting violence trends. Injury Prevention, 11(6), 324-325. doi:10.1136/ip.2005.009761.

This commentary argues that police recorded crime data is unreliable. It suggests that hospital emergency department data offer a useful alternative.

Effectiveness and EfficiencyA key role of performance measurement is assessing effectiveness and efficiency. The literature in this section determine how these two concepts can be accurately captured. Drake and Simper (2001) examine factors impacting policing efficiency in the UK. Drake and Simper (2003) discuss the challenges associated with measuring police efficiency. Drake and Simper (2005) assess the efficiency of Basic Command Units in the UK. Garca-Snchez (2009) assesses police efficiency in Spain in public safety and road safety. Garca-Snchez , Rodrguez-Domnguez, Parra-Domnguez (2013a, b, c) assess police efficiency in handling major crimes in Spain. Garca-Snchez, Rodrguez-Domnguez, and Parra-Domnguez (2013a, b, c) measure police efficiency in Spain using data envelopment analysis for the specific period ranging from 2001-2006. Heatons (2010) article summarizes the academic literature on the costs of crime and policing effectiveness.

Drake, L.M., and Simper, R. (2001). The economic evaluation of policing activity: An application of a hybrid methodology. European Journal of Law and Economics, 12, 173-192.

The authors apply a hybrid methodology to assess police efficiency in the United Kingdom. The authors find that one of the main variables to assess police efficiency is the violent crime clearance rate in the UK.

Drake, L. M., and Simper, R. (2003). An evaluation in the choice of inputs and outputs in the efficiency measurement of police forces. Journal of Socio-Economics, 32, 701-710.

The authors examine the difficulties associated with the development of input and output indicators and the challenges associated with measuring police efficiency.

Drake, L.M., and Simper, R. (2005). Police efficiency in offences cleared: An analysis of English Basic command units. International Review of Law and Economics, 25, 186-208. doi:10.1016/j.irle.2005.06.003.

This study ranks UK Basic Command Units (small areas) based on their clearance rates. It compares two ranking methods: nonparametric data envelopment analysis (DEA) and the parametric stochastic output distance frontier (SODF). It finds a strong correlation between the methods, suggesting similar validity for assessing BCUs and police forces efficiency in detection and prevention. The work identifies a non-linear relationship between crime (the input) and clearances (the output).

Garca-Snchez, I-M. (2009). Measuring the efficiency of local police force. European Journal of Law and Economics, 27(1), 5977.

The authors assess police efficiency in public safety and road safety in Spain. The authors define efficiency as optimum use of inputs to achieve the outputs.

Garca-Snchez I-M, Rodrguez-Domnguez, L., Parra-Domnguez, J. (2013). Evaluation of the efficacy and effectiveness of the Spanish security forces. European Journal of Law and Economics, 36(1), 57-75. doi:10.1007/ s10657-011-9265-4

The authors assess the effectiveness and efficiency of Spanish police forces in dealing with major crimes. Efficacy is divided into two categories: operational efficacy and goals efficacy.

Garca-Snchez, I-M., Rodrguez-Domnguez, L., and Parra-Domnguez, J. (2013). Yearly evolution of police efficiency in Spain and explanatory factors. Central European Journal of Operations Research, 21(1), 31-62. doi:10.1007/s10100-011-0207-6.

The authors assess the efficiency of police forces in Spain for the period 2001 to 2006. The authors look at cleared crimes and police numbers in addition to external factors outside the influence of the police.

Heaton, P. (2010). Hidden in plain sight: What cost-of-crime research can tell us about investing in police. Santa Monica, CA: RAND Corporation.

This study summarises the literature on costs of crime and police effectiveness. Based on the assumption that more police numbers reduces crime, it offers a crude calculator of the benefit of reduced crime from increasing police spending.

Tiwana et al. Crime Science (2015) 4:1 Page 9 of 28

Qualitative IndicatorsWhereas most of the literature focuses on quantification, this section contains literature on the importance of qualitative indicators. Fielding and Innes (2006) argue for qualitative methodology, Shilston (2008) finds that certain aspects of police activity more than quantitative assessment, and Shilston (2011) applies the Black Box method to measure police service delivery.

Fielding, N., and Innes, M. (2006). Reassurance policing, community policing and measuring police performance. Policing and Society: An International Journal of Research and Policy, 16(2), 127-145. doi:10.1080/10439460600662122.

The authors argue that community policing and reassurance policing is essentially too nuanced for quantitative measures, and argues for assessment using qualitative indicators.

Shilston, T.G. (2008). One, two, three, what are we still counting for? Police performance regimes, public perceptions of service delivery and the failure of quantitative measurement. Policing: A Journal of Policy and Practice, 2(3), 359-366. doi:10.1093/police/pan043.

Quantitative measurement of police performance is considered crucial to assess police performance. However, the study argues that to measure public perceptions, quantitative is not an appropriate method because it does not capture all aspects of policing. The study argues for a qualitative approach.

Shilston, T.G. (2011). Black box: A qualitative method for improving public confidence in policing through micro-analysing service delivery. Policing: A Journal of Policy and Practice, 5(2), 125-131, doi:10.1093/police/ par003.

The author introduces Black Box as a qualitative method for assessing police service delivery. It draws on the terminology and method of aviation crash investigations. The study proposes that this will enable the police to examine events chronologically from the time of reported through the actions resulting from police attending an incident.

Inconsistent Police DataDocuments in this section highlight the need for consistency of police data, particularly crime recording practices. Burrows, Tarling, Mackie, Lewis, and Taylor (2000) examine police recording practices in 10 police forces in the UK. Hallam (2009) looks at factors influencing crime recording. A Home Office (2011) document

provides background information on the National Crime Recording Standard (NCRS), and a North Yorkshire Police (n.d.) document provides information on NCRS for that areas police force. Simmons, Legg, and Hosking (2003) examine the impact of NCRS on crime statistics. The Thin Blue Line (2011) document, which we feel lacks rigour, examines discrepancies in crime recording.

Burrows, J., Tarling, R., Mackie, A., Lewis, R., and Taylor, G. (2000). Review of police forces crime recording practices. London, UK: Home Office.

This study examines crime recording practices in 10 English police forces. It examines police decisions underpinning whether to record a public complaint, designate as a crime or an incident. The particular issue is the not recording of crimes reported to the police as observed in the gaps between the crime statistics report by the British Crime Survey and police recorded crime. The study was also undertaken to understand the criteria applied by police forces to differentiate a crime from an incident.

Hallam, S.A. (2009). Policing in the iron cage: The tensions between the bureaucratic mandate and street level reality. Unpublished doctoral dissertation. University of Bedfordshire, Luton, Bedford, UK.

This thesis examines how deviation from the National Crime Recording Standard (NCRS) may occur as either a result of situational factors or performance management. It finds measuring police performance based on crime rates is far from perfect, and the recording of crimes by the police is dependent on the situational, social, political and organizational contexts of how the statistics are compiled (p.229). It suggests that, if anything, the NCRS has contributed to an increase in workload of the police officers.

Home Office. (2011). The national crime recording standard (NCRS): What you need to know. London, UK: Home Office.

This two page document prepared by the Home Office provides brief background information on the National Crime Recording Standard (NCRS) and the Home Office Counting Rules for recorded crime (HOCR). It outlines the purposes of recording crime by police officers and addresses frequently asked questions regarding these two standards.

North Yorkshire Police. ( n.d.). Safer neighbourhoods: National crime recording standards (NCRS) procedure. North Yorkshire, UK: North Yorkshire Police.

Tiwana et al. Crime Science (2015) 4:1 Page 10 of 28

This document prepared by the North Yorkshire Police provides guidelines to its officers regarding the NCRS and the process and proper procedures to follow regarding recording of crimes, classification and re-classification, no crimes, one crime per victim, the finished incident rule, the principle crime rule, location of crimes, detections, and other agencies. The document also provides the responsibilities of various units and personnel within the police force.

Simmons, J., Legg, C., and Hosking, R. (2003). National crime recording standard (NCRS): An analysis of the impact on recorded crime. Part one: The national picture. London, UK: Home Office.

This study examines the impact of the NCRS which was introduced in 2002 in England and Wales to improve the consistency of recorded crime. It finds the NCRS changes explain why recorded crime appeared to increase and suggests that, when the recording change is accounted for, crime rates had continued to fall.

The Thin Blue Line. (2011). Crime of the century: A chilling look at crime statistics, police recorded crime and offences brought to justice in the UK. West Midlands, UK: Nice 1 Ltd.

In our opinion this is a poor quality study but its authors know enough to be dangerous. It offers a conspiracy theory wherein the fall in UK crime (and, we infer, that elsewhere) is a big misleading lie and the result of a pernicious web of deceit, presumably spun by government, police, academic researchers, the media, and most everyone else. The study seems to represent vested interests of those whose resources are threatened by falling crime. We think its bonkers but its out there, so now youve been warned.

Performance Measurement in Other SectorsThis section provides insight into the extensive broader (non-police or criminal justice) literature on performance measurement. Bevan and Hood (2006) examine targets in the English public health care system. Bird et al. (2005) outline guidelines for performance monitoring in the public sector in the UK. The book by Carter, Klein, and Day (1992) covers the history and use of performance measurement across all UK public and private sectors. Chew, Swanson, Stine, Bartol, Brown, and Robinson (2008) provide a guide to performance measures in information security agencies. Dooren (2011) examines challenges associated with performance measurement. Harrell et al. (n.d.) discuss performance measurement in human service organizations, while Kennerley and Neely (2003) acknowledge that organizations evolve and

that performance measures must adapt to change. Michell and Neely (2010) examine the impact of Public Service Agreements (PSA) on the health care system and police services in the UK. We include reports from the Treasury Board of Canada but it is likely similar ones exist for other countries: the Treasury Board of Canada Secretariat (2007, 2009, 2010,n.d.) provides submission guidelines for federal agencies, how to complete a Performance Measurement and Evaluation Plan, performance measurement strategies for federal programs and, guidelines for completing related reports. The guideline published by the U.S. Department of Energy (1996) outlines how to develop performance measures for the U.S. Department of Energy.

Bevan, G., and Hood, C. (2006). Whats measured is what matters: Targets and gaming in the English public health care system. Public Administration, 84(3), 517-538.

This study examines the validity of the assumptions of governance by targets in the English public health care system. It identifies areas where targets result in gaming and inadvertent negative effects on practice.

Bird, S.M., Cox, D., Farewell, V.T., Goldstein, H., Holt,T., and Smith, P.C. (2005). Performance indicators: Good, bad, and ugly. Journal of the Royal Statistical Society. Series A (Statistics in Society), 168(1), 1-27.

This is a report by the Royal Statistical Societys Working Party on Performance Monitoring in the Public Services, outlining recommendations for performance monitoring in UK public services, with guidelines for the development and implementation of performance monitoring. It outlines the history of performance management, examines the design and setting of targets and protocols, data analysis, presentation of performance indicators, the benefits of monitoring performance, evaluating performance, and the Integrity, confidentiality and ethics of performance monitoring.

Carter, N., Klein, R., and Day, P. (1992). How organisations measure success: The use of performance indicators in government. New York, NY: Routledge.

This study provides an overview of the historical development of the use of performance indicators in the United Kingdom in public and private sectors. It has a chapter devoted to the introduction of performance measures in the criminal justice system.

Chew, E., Swanson, M., Stine, K., Bartol, N., Brown,A., Robinson, W. (2008). Performance measurement

Tiwana et al. Crime Science (2015) 4:1 Page 11 of 28

guide for information security. Gaitherburg, MD: National Institute of Standards and Technology, U.S. Department of Commerce.

This is a guide prepared by the National Institute of Standards and Technology in the U.S. Department of Commerce. It outlines how to [develop], [select], and [implement]measures to assess performance in information security agencies.

Dooren, W.V. (2011). Better performance management: Some single and double-loop strategies. Public Performance and Management Review, 34(3), 420-433. doi:10.2753/PMR1530-957634030.

This study reviews performance measurement and the associated challenges. It categorizes solutions to the challenges according to single-loop learning and double-loop learning. Single-loop learning strategies are solutions which target the implementation of performance management. Double-loop learning solutions propose revisions and changes to performance management rather than just implementing better performance management.

Harrell, A., Burt, M., Hatry, H., Rossman, S., Roth, J., and Sabol, W. (n.d.). Evaluation strategies for human services programs: A guide for policymakers and providers. Washington, DC: The Urban Institute.

This report is a practitioner guide. It provides a set of principles for evaluating human service programs.

Kennerley, M., and Neely, A. (2003). Measuring performance in a changing business environment. International Journal of Operations and Production Management, 23(2), 213-229. doi:10.1108/ 01443570310458465.

This study suggests that, as organizations evolve, performance measurement systems should reflect the changes. It addresses two main research questions: (1) How do performance measurement systems evolve? and (2) How can performance measurement systems be managed in a way that accounts for organizational change?

Michell, P., and Neely, A. (2010). Performance measurement in the public sector in England: Searching for the golden thread. Public Administration Review, 70(4), 591-600. doi:10.1111/j.1540-6210.2010.02180.x.

This study examines the impact of Public Service Agreements (PSAs) which introduced performance measurement across the UKs public sector. Health care and policing are used as case studies. A key intention of

the PSAs was for national policies to be accurately reflected in services delivered locally: labelled the Golden Thread. The study describes issues arising in this top-down performance measurement system.

Treasury Board of Canada Secretariat. (2007). A guide to preparing treasury board submissions. Ottawa, ON: Treasury Board of Canada Secretariat.

This guide details submission guidelines for federal agencies seeking approval for new initiatives from Canadas Treasury Board. This includes details on clear objectives, defined outcomes, how value for money is ensured, and how results will be reported to parliament.

Treasury Board of Canada Secretariat. (2009). Handbook for regulatory proposals: Performance measurement and evaluation plan. Ottawa, ON: Treasury Board of Canada Secretariat.

This is a handbook for government officials and analysts in the Regulatory Affairs Sector of the Treasury Board of Canada Secretariat who are given responsibility for developing and implementing a Performance Measurement and Evaluation Plan (PMEP). The PMEP is defined as a statement that continually assesses whether the initially identified objectives for regulatory activities are being met. It offers guidance on how to complete a PMEP.

Treasury Board of Canada Secretariat. (2010). Supporting effective evaluations: A Guide to developing performance measurement strategies. http://www.tbs-sct.gc.ca/cee/dpms-esmr/dpms-esmrtb-eng.asp

Web End =http://www.tbs http://www.tbs-sct.gc.ca/cee/dpms-esmr/dpms-esmrtb-eng.asp

Web End =sct.gc.ca/cee/dpms-esmr/dpms-esmrtb-eng.asp (Accessed: 26 October 2013).

This is an online guide for program managers and others tasked with evaluation. It aims to assist with the development of performance measurement for federal programs.

Treasury Board of Canada Secretariat. (n.d.). Guide to the preparation of part III of the 2006-2007 estimates: Reports on plans and priorities and departmental performance reports. Ottawa, ON: Treasury Board of Canada Secretariat.

The purpose of this guide is to assist federal departments with their reports on plans and priorities (RPPs) and departmental performance reports (DPR). The guide is intended to ensure that RPPs and DPRs are consistent across federal departments. RPPs and DPRs are prepared for parliament and outline departmental plans to achieve expected results and perform well.

Tiwana et al. Crime Science (2015) 4:1 Page 12 of 28

U.S. Department of Energy. (1996). Guidelines for performance measurement. Washington, DC: U.S. Department of Energy.

This is a guide for organizations within the Department of Energy. It outlines how to develop and implement performance measurement.

National ApproachesThis section contains studies grouped by country that have not been listed elsewhere. Depending largely on the governance structure of the country, a national performance regime may or may not even be feasible. Literature from the UK contributes disproportionately to this section due to that countrys well-documented national regime that began over two decades ago, whereas the US literature features in many of the other sections because it is less thematic, reflecting that countrys decentralised governance of policing.

AustraliaAlach and Crous (2012) address the need for distinct and tailored performance measured for specialist policing units. Cuganesan and Lacey (2011) assess a return on investment (ROI) performance measurement framework and specifically developing an ROI metric. Dadds and Scheide (2000) examine the activity measurement method used in the South Australia Police. Fleming and Scott (2008) provide an overview of Operational Performance Reviews (OPRs), which is a form of Australian Compstat, and Mazerolle, Rombouts, and McBroom (2006) study the impact of those OPRs on crime reduction.

Alach, Z., and Crous, C. (2012). A tough nut to crack: Performance measurement in specialist policing. Canberra, ACT: Australian Institute of Criminology.

This report outlines how to develop performance measurement for a specialist police unit. It uses the Auckland Metropolitan Crime and Operational Support (AMCOS) specialist unit of the New Zealand Police as a case study. It is predicated on the assumption that specialist policing units face different issues to general policing units and that performance measurement should reflect the differences. The authors define specialist units as technical units, niche units or investigative units.

Cuganesan, S., and Lacey, D. (2011). Developments in public sector performance measurement: A project on producing return on investment metrics for law enforcement. Financial Accountability and Management, 27(4), 458-479.

This article details the introduction of a Return on Investment (ROI) performance measurement framework at the Victoria Police in Australia, focusing on a single measure: the ROI metric. In 2005, the Victoria Police introduced the Organized Crime Strategy. The ROI metric calculated the monetary value of outputs/impacts from service delivery function and the monetary value of inputs consumed in service delivery.

Dadds, V., and Scheide, T. (2000). Police performance and activity measurement. Trends and Issues in Crime and Criminal Justice No. 180. Canberra, ACT: Australian Institute of Criminology.

The authors provide an overview of South Australia Polices (SAPOL) activity measurement approach. Activity measurement seeks to gauge what police do across the workday, based on the argument that traditional performance measures do not accurately portray what is done, particularly because many activities are not related to law enforcement.

Fleming, J. and Scott, A. (2008). Performance measurement in Australian police organizations. Policing: A Journal of Policy and Practice, 2(3), 322-330. doi:10.1093/police/pan038.

This study provides a good overview of police performance measurement in Australia. It has a focus on Operational Performance Reviews (OPRs), which is a version of Compstat.

Mazerolle, L., Rombouts, S., and McBroom, J. (2006). The impact of operational performance reviews on reported crime in Queensland. Trends and Issues in Crime and Criminal Justice, 313, 1-6.

This study evaluates the impact upon crime of Operational Performance Reviews (OPRs). OPRs are found to have a significant impact in reducing property crime, particularly in the unlawful entry category. However other explanations are also acknowledged to have contributed to the Australian crime drop. The authors acknowledge the role of differences between districts in their different crime rates. The study identifies the importance of examining local (small area) crime trends in order to understand the aggregate trend.

New ZealandHeyer (2012) reviews cost-saving efforts by New Zealand police to identify potential lessons for the US. The New Zealand Police (2013) published their Statement of Intent for 2013 to 2016 which includes priority areas. The State Services Commission (2012) document highlights areas in the New Zealand Police requiring improvement, and

Tiwana et al. Crime Science (2015) 4:1 Page 13 of 28

the fact sheet published by the State Services Commission describes the national Performance Improvement Framework.

Heyer, G.D. (2012). Cost-saving efforts of the New Zealand Police: Opportunities for the United States. The Police Chief, 79(12), 60-63.

This is a magazine article by a Chief Inspector of the New Zealand Police. It discusses techniques employed by New Zealand Police to increase efficiency and effectiveness, which may assist policing services in the United States.

New Zealand Police. (2013). Statement of Intent 2013/2014 2015/2016. Wellington, New Zealand: New Zealand Police.

This Statement of Intent document, prepared by the New Zealand Police, covers 2013 to 2016. It outlines how police goals align with those of the New Zealand government. Four priorities are identified: less crime, improved road safety, protected communities, more valued services.

State Services Commission. (2012). Performance improvement framework: Formal review of the New Zealand Police. Wellington, New Zealand: State Services Commission.

This document details a review undertaken by the State Services Commission of the New Zealand Police. The purpose of the review was to identify areas requiring improvement in police performance.

State Services Commission. (2013). PIF fact sheet 1: Introducing the performance improvement framework. http://www.ssc.govt.nz/pif-factsheet1

Web End =http://www.ssc.govt.nz/pif-factsheet1 (Accessed: 08 August 2013).

This fact sheet provides information on the performance improvement framework (PIF). The

State Services Commission used the PIF to review New Zealand Police. The PIF seeks to assess whether the agency under review is situated to address issues that may arise in the medium-term future, and identifies areas for improvement.

United Kingdom: Two Decades of National Performance MeasurementThe UKs extensive literature on police performance measurement reflects over two decades of national government focus and regimes . This has spurred a mix of

government reviews and guides alongside academic critiques and contributions.

General OverviewsBarton (2013) argues for the adoption of lean policing practices. Boyd, Geoghegan, and Gibbs. (2011) examine police expenditure and impact on performance. Butterfield Edwards, and Woodall (2005) look at the impact of New Public Management on police sergeants. Hale, Uglow, and Heaton (2005) examine performance measurement through the comparison of families of similar police forces. Hunton, Jones, and Baker (2009) propose a new performance framework that integrates what were then the three existing frameworks, while the Independent Police Commission (2013) provides a new vision for policing the UK, and the Police Federation (2013) detail numerous articles on recent police reforms. Is included as an overview of policing in Britain to that date, which may provide context and be of historical interest.

Barton, A. (2013). Lean policing? New approaches to business process improvement across the UK police service. Public Money and Management, 33(3), 221-224.

This article examines adopting Lean thinking practices in policing. Adopting the Lean philosophy in policing would ensure that internal costs and wasteful activities would be reduced thus increasing public satisfaction and confidence in the police. It is suggested that police resources should be refocused on improving public satisfaction with policing activities.

Boyd, E., Geoghegan, R., and Gibbs. B. (2011). Cost of the cops: Manpower and deployment in policing. London, UK: Policy Exchange.

This review details how major increases in UK police spending were poorly used in the decade before, spent on more police officers but without improved performance. It makes a series of recommendations including increased civilianisation, better sick-leave management, increased visibility, reduced expenditures and increased clearances.

Butterfield, R., Edwards, C., and Woodall, J. (2005). The new public management and managerial roles: The case of the police sergeant. British Journal of Management, 16, 329-341. doi:10.1111/j.1467-8551.2005.00466.x.

This article discusses how the philosophy of new public management influenced how police sergeants performed their roles. The rank of Sergeant was

Tiwana et al. Crime Science (2015) 4:1 Page 14 of 28

selected because of their responsibility in managing service delivery.

Hale, C.M., Uglow, S.P., and Heaton, R. (2005). Uniform styles II: Police families and policing styles. Policing and Society: An International Journal of Research and Policy, 15(1), 1-18. doi:10.1080/1043946042000338904.

Cross-jurisdictional comparison of performance implies apples being compared to apples. Since police jurisdictions often differ considerably this can be unfair. So the attempt was made to develop families of similar UK police forces so each could be compared to its closest peers. This journal article examines the methodology underlying the groupings.

Hunton, P., Jones, A., and Baker, P. (2009). New development: Performance management in a UK police force. Public Money and Management, 29(3), 195-200.

This study proposes a performance measurement framework that encompasses the three prior ones: the National Policing Plan, the Police Performance Assessment Framework (PPAF), and the National Intelligence Model. The integrated framework includes key components of assessing police performance such as performance measurement, monitoring, and review.

Independent Police Commission. (2013). Policing for a better Britain: Report of the Independent Police Commission (The Stevens Report). Essex, UK: The Lord Stevens of Kirkwhelpington QPM.

This is a major independent review of UK policing that was chaired by Lord Stevens, former chief of London police, and involved many leading police scholars, with written input from many parties involved in policing. The report aims to be comprehensive. It goes back to basics to identify and update the aims of policing, and progresses to suggest a new framework for policing.

Police Federation. (2013). Policing UK 2013. Priorities and pressures: A year of transformation. Surrey, UK: Police Federation.

This document contains short statements by three dozen experts from policing, politics, academia, journalism, and elsewhere. Each reflects on aspects of the significant police reform that UK policing was undergoing at the time. It includes the Home Secretarys statement on performance indicators or, rather, their wholesale removal in favour of A singular focus on cutting crime(p.12-13). The value of this publication is to bring these

divergent views together in one accessible place, providing context to the wholesale police reform of which major change to the national performance measurement regime was a component.

HistoryThis section includes historical accounts of police performance measurement in the UK. Collier (2006) documents the various changes to performance measurement. Hale, Heaton, and Uglow (2004) examine the HMIC approved policing styles available to UK police service. Martin (2011) studies the shift from centralization to decentralization in public sector performance measurement. Newburn (2003) traces the history of performance measurement since 1945 and the polices various relationships. Savage (2007) also examines the history of performance measurement specifically focuses on policy change.

Collier, P.M. (2006). In search of purpose and priorities: Police performance indicators in England and Wales. Public Money and Management, 26(3), 165-172.

Collier (2006) examines the rapid evolution of UK police performance measurement from 1992 to 2004, culminating in the Policing Performance Assessment Framework (PPAF). It suggests changing political priorities have underpinned change. It details three key issues in performance measurement the complexity of policing, the definition of performance, and inconsistent use of the term indicator.

Hale, C., Heaton, R., and Uglow, S. (2004). Uniform styles? Aspects of police centralization in England and Wales. Policing and Society: An International Journal of Research and Policy, 14(4), 291-312

This article precedes Hale, Uglow, and Heaton (2005) in the General Overviews section. This study focuses on the reports of Her Majestys Inspectorate of Constabulary (HMIC). It examines UK approved policing styles including a focus on intelligence-led policing, and factors determining the extent to which approved policing styles are adopted.

Martin, S. (2011, March). What is localism and does it matter? Paper presented at the Commonwealth Local Government Forum Research Colloquium: Sustainable local governance for prosperous communities, Cardiff, UK.

This article traces the history of centralization of public sector performance measurement and more recent shifts towards decentralization in the UK. It outlines

Tiwana et al. Crime Science (2015) 4:1 Page 15 of 28

future considerations and interpretations of the UKs Localism Bill.

Newburn, T. (2003). Policing since 1945. In T. Newburn (Ed.), Handbook of policing (pp. 84-105). Portland, Oregon: Willan.

This is a chapter in the first edition of the Handbook of Policing. It focuses on relationships between the police and local communities, government, and other policing agencies, providing the context within which more recent performance management regimes have developed.

Savage, S.P. (2007). Police reform: forces for change. New York, New York: Oxford University Press.

This book provides the reader with an overview of police reforms, focusing on UK policy change from the late 1970s (when much change began to occur) to 2007. It is an extensive review providing a detailed history.

Official Reports and GuidesThis section includes key guides and official reports. The Audit Commission (n.d.) includes performance indicators for policing services for 1999/2000. The Department of the environment, transport and the regions (1999) document includes indicators for policing derived from the best value performance indicators (BVPIs) framework and Audit Commissions performance indicators (ACPIs). The Fellows Associates (2010) provide a summary of two key documents outlining how to increase police effectiveness while keeping police expenditures low. Similarly, Her Majestys Inspectorate of Constabulary (2010) looks at how to ensure that British policing remains effective despite budget cuts. Flanagan (2008a) provides a review of the police recommendations for improving. The second article by Flanagan (2008b) includes a Equality Impact Statement (EIA) which is used to determine the impact the review of policing has on citizens. Home Office (2008a) introduces the replacement for the Police Performance Assessment Framework, the Assessments of Policing and Community Safety (APACS) framework. Home Office (2008b) introduced the Statutory Performance Indicators for year 2008/2009. Home Office (2008c) provides the best value performance indicators implemented on 01 April 2008. The Home Office, Association of Police Authorities (2006) document guides police authorities on how to wmanage performance. Leigh, Mundy, and Tuffin (1999) discuss the implementation of best value across the UK.

Audit Commission. (n.d.). Local authority performance indicators. 1999/00 Audit guide. London, UK: Audit Commission.

This guide details performance indicators for all Audit Commission programmes including police services for 1999/2000. The introductory section provides the reader with the historical background and information on then-current auditing and publication process for performance indicators.

Department of the environment, transport and the regions. (1999). DETR: Best value and audit commission performance indicators for 2000/2001. Volume One: The performance indicators. London, UK: Department of the environment, transport and the regions.

This guide details the UKs Best Value Performance Indicators (BVPIs), a set of performance measures for UK local services for 2000/2001, including education, social services, housing, emergency services (police, fire), and the environment (including transport). A set of measures with a common foundation allows comparison between agencies and on issues shared by more than one agency.

Fellows Associates. (2010). Sustaining value for money in the police service and valuing the police. London, UK: Fellows Associates.

This document provides a summary of two documents: Valuing the Police: Policing in an age of austerity published by Her Majestys Inspectorate of Constabulary (HMIC) and Sustaining value for money in the police services published by HMIC, Wales Audit Office, and the Audit Commission. These documents discuss increasing policing effectiveness while reducing policing expenditures.

Flanagan, R. (2008a). The review of policing: Final report. Surrey, UK: Police Federation of England and Wales.

Sir Ronnie Flanagan was commissioned by the UK government minister for internal affairs, the Home Secretary, to review policing in England and Wales. The report discusses challenges faced by the police and offers recommendations for improvement.

Flanagan, R. (2008b). The review of policing: Equality impact assessment report. London, UK: Home Office.

This report is an Equality Impact Statement (EIA) to determine what impact the review of policing (Flanagan 2008a, above) would have on citizens dependent upon the seven strands of diversity: age, disability, faith, gender, race, sexual orientation and transgender.

Tiwana et al. Crime Science (2015) 4:1 Page 16 of 28

Her Majestys Inspectorate of Constabulary. (2010). Valuing the police: Policing in an age of austerity. London, UK: Her Majestys Inspectorate of Constabulary.

This report examines how to ensure effective policing despite budget cuts. It identifies four key areas where savings could be made; reducing variation between forces on spending on some budget items; aligning staff shift patterns to need; preparing for budget cuts, and; system architecture - a broad term which includes the conclusion that performance measurement is cluttered and lacks incentives.

Home Office. (2008a). Definitions and survey guidance for APACS measures of user satisfaction 2008/9. London, United Kingdom: Home Office.

This is a Home Office guide to Assessments of Policing and Community Safety (APACS), the performance assessment framework developed to replace the Policing Performance Assessment Framework (PPAF). It describes how to conduct user satisfaction surveys to ensure consistency and promote best practices.

Home Office. (2008b). Guidance on statutory performance indicators for policing and community safety 2008/09. London, United Kingdom: Home Office.

The Statutory Performance Indicators (SPI) established for the 2008/2009 are outlined in this guide. For each SPI, it details the SPI number, full title, formula(e), definitions, and how the data is to be reported to the Home Office.

Home Office. (2008c). Local government, England and Wales police, England and Wales. The police authorities (best value) performance indicators order 2008. London, UK: Home Office.

This document usefully lists and details the Statutory Performance Indicators (SPI) to be used for the Assessments of Policing and Community Safety (APACS) in England and Wales as of April 1, 2008. A table at the start lists the SPIs and the page numbers on which they are described, with a page or two for each including definitions and formulae.

Home Office, Association of Police Authorities. (2006). Police performance management: Practical guidance for police authorities. London, UK: Association of Police Authorities, Home Office.

This is a practitioner guide prepared for police authorities. It describes the role of police authorities and provides guidance on performance management.

Leigh, A., Mundy, G., and Tuffin, R. (1999). Best value policing: Making preparations. Police Research Series Paper 116. London, UK: Home Office.

This guide details research on how police forces have been preparing to implement best value policing on April 1, 2000, and draws together preliminary findings about progress and issues arising. The research included a telephone survey of all 43 police forces plus face-to-face interviews with key police personnel.

United StatesNote that entries relating to the US tend to be spread throughout this annotated bibliography rather than listed here.

Manning, P.K. (2008). Performance rituals. Policing: A Journal of Policy and Practice, 2(3),284-293. doi:10.1093/police/pan050.

This article discusses the introduction of police performance evaluation in the United States and its implications for the policing mandate. It bemoans the focus on individual performance, suggesting this is largely responsible for the absence of performance measurement relating to organisations.

CanadaKiedrowski, J., Petrunik, M., MacDonald, T., and Melchers, R. (2013).Canadian police board views on the use of police performance metrics. Ottawa, ON: Public Safety Canada.

This study, commissioned for the Ministry of Public Safety (with responsibility for policing), details a survey of police boards across Canada. It examines their use of performance measurement tools. It highlights the under-utilisation of performance measures, and, in essence, the lack of consistency and rigour in their application.

Lithopoulos, S., and Rigakos, G.S. (2006). Neoliberalism, community, and police regionalization in Canada: A critical empirical analysis. Policing: An International Journal of Police Strategies and Management, 28(2), 337-362. doi:10.1108/13639510510597942.

This study examines the regionalization of policing in Canada using basic empirical measures of police performance. It finds no evidence in favour of regionalisation, suggesting this may be because policing is primarily a local concern.

Pare, P-P., and Ouimet, M. (2004). A measure of police performance: Analyzing police clearance and

Tiwana et al. Crime Science (2015) 4:1 Page 17 of 28

charge statistics. The Canadian Review of Policing Research. at http://crpr.icaap.org/index.php/crpr/article/view/15/14

Web End =http://crpr.icaap.org/index.php/crpr/art http://crpr.icaap.org/index.php/crpr/article/view/15/14

Web End =icle/view/15/14 , accessed August 2012.

This review argues that exogenous variables which impact the clearance rate can be used to develop a more appropriate indicator to assess police performance.

IndiaKumar, S., and Kumar, S. (2013). Does modernization improve performance: Evidence from Indian police (MPRA Paper No. 45801) . Retrieved from the Municipal Personal RePEc Archive website: http://mpra.ub.uni-muenchen.de/45801/1/MPRA_paper_45801.pdf

Web End =http://mpra. http://mpra.ub.uni-muenchen.de/45801/1/MPRA_paper_45801.pdf

Web End =ub.uni-muenchen.de/45801/1/MPRA_paper_45801.pdf (accessed September 2013).

This paper develops statistical models to explore how modernization in Indian police is impacting performance. It finds that among Indian police departments, those which have adopted new communications technology and spend more on officer training, are performing better than previously.

South AfricaLeggett, T. (2003). What do the police do? Performance measurement and the SAPS. Institute for Security Studies, 66, 1-15.

The purpose of this document is to determine which performance indicators should be chosen to accurately assess the performance of the South African Police Service (SAPS). To develop performance indicators, the authors examine the police role in society.

International and Cross-national StudiesThis section includes international and cross-national comparative studies. Lelandais and Bodson (2007) cover five continents and the range of experiences in adopting performance measurement systems. Maillard and Savage (2012) study performance measurement reforms in Britain and France. Sluis and Ringeling (2009) analyze changes and recent reform in policing in three countries.

Lelandais, I., and Bodson, J. (2007). Measuring police performance: International experiences. Montreal, QC: International Centre for the Prevention of Crime.

This study reviews experiences with police performance measurement in different regions of the world, including North America, South America, Europe, and Oceania. It describes the respective police services, evaluating the method of measuring performance, and its strengths and challenges.

Maillard, J.D., and Savage, S.P. (2012). Comparing performance: the development of police performance management in France and Britain. Policing and Society: An International Journal of Research and Policy, 22(4), 363-383.

This study is a comparative review of police performance management reforms in Britain and France. It compares the structure of policing in terms of centralisation and localisation and the role of police organisations. Key differences include the citizen focus and the public sharing of information on police performance in the British model.

Sluis, A.V., and Ringeling, A. (2009). Evolving patterns in the police systems of North Rhine-Westphalia, The Netherlands and England and Wales. German Policy Studies, 5(2), 145-168.

The authors compare the organization and governance of policing services in three countries: North-Rhine Westphalia, The Netherlands, and England and Wales. The paper is organized into six sections which provide the reader with an overview of some of the reforms and shifts that have occurred in these police services.

FrameworksThis section details literature on three frameworks relating to performance measurement, each of different orientation. The first is Compstat which is a performance management framework that has taken different forms in different times and places, and is now utilised outside of policing as data-driven performance management. The second, the Balanced Scorecard, is a framework for identifying groups of data for analysis as a performance management system, and has been quite widely adopted in the business world and to a lesser extent in policing. Some of the classic and more recent literature from the business sector is included for the Balanced Scorecard as well as documents relating to its extensive use by the Royal Canadian Mounted Police. The third is Data Envelopment Analysis, which is a framework for the analysis of performance management.

CompstatThe word Compstat is a concatenation of the term computer statistics (or comparative statistics, depending on the source). It is an approach to performance management that was introduced in the New York Police Department in the 1990s but has been adapted elsewhere. Bratton and Malinowski (2008) explore the impact on crime reduction. Moore (2003) provides an overview of Compstat and its impact on police agencies. Moore and Braga (2003a, b) explore the implementation of Compstat

Tiwana et al. Crime Science (2015) 4:1 Page 18 of 28

in six police departments. Weisburd et al. (2003) find that prior to the introduction of Compstat in American policing, policing was already moving towards strategic practices. Hatry and Davies (2011) details the extension of Compstat to other sectors under the banner of data-driven performance review.

Bratton, W.J., and Malinowski, S.W. (2008). Police performance management in practice: Taking COMPSTAT to the next level. Policing: A Journal of Policy and Practice, 2(3), 259-265. doi:10.1093/police/pan036.

Compstat is said to have developed under, and receive its first main use by, Police Chief William Bratton in New York City. In this study, the authors argue that it can contribute to crime reduction. The study also suggests future directions in performance management.

Moore, M.H. (2003). Sizing up COMPSTAT: An important administrative innovation in policing. Criminology and Public Policy, 2(3), 469-494. doi:10.1111/j.1745-9133.2003.tb00009.x.

This article provides a particularly informed overview of Compstat and details its origins, use and impact on police agencies.

Moore, M.H. and Braga, A.A. (2003). Measuring and improving police performance: The lessons of Compstat and its progeny. Policing: An International Journal of Police Strategies and Management, 26(3), 439 453.

This study explains why it is important to measure police performance including reasons such as both external and internal accountability. The authors also outline dimensions of policing that should be measured to assess performance. Six police departments in the U.S. are studied to shed light on the implementation of Compstat.

Weisburd, D., Mastrofski, S.D., McNally, A., Greenspan,R., and Willis, J.J. (2003). Reforming to preserve: COMPSTAT and strategic problem solving in American policing. Criminology and Public Policy, 2(3), 421 456.

This paper explores the implementation of COMPSTAT in the United States. The authors found that American policing was already shifting towards more strategic policing practices prior to the introduction of COMPSTAT.

Hatry, H. and Davies, E. (2011). A guide to data-driven performance reviews. Improving Performance

Series. Washington, DC: IBM Centre for The Business of Government.

This guide argues that Compstat, as developed by the New York City police department, is a performance management process that is widely adopted in federal programs across all sectors. It refers to this process as data-driven performance reviews.

Balanced ScorecardIn BusinessThe Balanced Scorecard, which has become influential in policing, was developed for the business sector by Kaplan and Norton (1992, 1993). Kaplan and Norton (2007) provide updates on its application for different aspects of management. Behn (2003) identifies eight purposes for measuring performance. Humphreys and Trotman (2011) provide a more recent update and critique focused on common measures bias.

Kaplan, R.S., and Norton, D.P. (1992). The balanced scorecard Measures that drive performance. Harvard Business Review, 71-79.

Kaplan and Nortons classic study introducing the balanced scorecard. The balanced scorecard groups measures into substantive areas and assigns scores that can be summed and compared across and within groups.

Kaplan, R.S., and Norton, D.P. (1993). Putting the balanced scorecard to work. Harvard Business Review, 71(5), 134-147.

This study documents the experiences of private sector organizations with using the balanced scorecard.

Kaplan, R.S., and Norton, D.P. (2007). Using the balanced scorecard as a strategic management system. Harvard Business Review, 1-14.

This article analyzes how businesses have applied the balanced scorecard as a strategic management system to reach long-term objectives.

Norton, D.P., and Coffey, J. (2007). Building an organized process for strategy communication. Balanced Scorecard Report, 9(3), 1-5.

This article looks at communicating an organizations strategy to its employees.

Behn, R.D. (2003). Why measure performance: Different purposes require different measures. Public Administration Review, 63(5), 586-606.

Tiwana et al. Crime Science (2015) 4:1 Page 19 of 28

This article explores why measuring performance of any agency (public or private) is important. It identifies eight purposes for measuring performance: to evaluate, control, budget, motivate, promote, celebrate, learn, and improve.

Humphreys, K.A., and Trotman, K.T. (2011). The balanced scorecard: The effect of strategy information on performance evaluation judgments. Journal of Management Accounting Research, 23, 81-98. doi:10.2308/jmar-10085.

Common measures bias that arises via the balanced scorecard is examined. This occurs when measures common to two units instead of being unique to one division are present in performance evaluations. The authors explore whether strategic information can be used to combat the common measures bias.

In PolicingGomes, Mendes and Carvalho (2006) and Carmona and Gronlund (2003) detail use of the balanced scorecard approach to police performance management in Spain and Sweden respectively. The balanced scorecard was used extensively across Canada by the Royal Canadian Mounted Police, as detailed in the reports listed here (RCMP 2006, 2012) though we understand this was discontinued in 2013.

Gomes, P.S., Mendes, S.M., and Carvalho, J.B. (2006). Performance measurement of the Portuguese police force using the balanced scorecard. Paper presented at the 4th International Conference on Accounting, Auditing and Management in Public Sector Reforms, Siena, Italy.

This study examine the factors determining the Portuguese polices attitudes towards the implementation of the use of the balanced scorecard to measure police performance.

Carmona, S., and Gronlund, A. (2003). Measures VS. actions: The balanced scorecard in Swedish law enforcement. International Journal of Operations and Production Management, 23(12), 1475-1496. doi:10.1108/01443570310506722.

This article discusses the implementation of the balanced scorecard in Swedish Law Enforcement and the associated challenges.

Royal Canadian Mounted Police. (2006, 2007, 2008, 2009, 2010, 2011, 2012). Royal Canadian Mounted Police Report on Plans and Priorities [year covered]. Ottawa: RCMP.

This is an annual RCMP report, available online for 2006 to 2012. It shows the balanced scorecard in use at

a national level, with activities linked to outcomes. The RCMP discontinued use of the balanced scorecard in 2013.

Data Envelopment AnalysisData envelopment analysis (DEA) is an analytic approach for assessing efficiency. Carrington, Puthucheary, Rose, and Yaisawarng (1997) use DEA to assess the New South Wales Police Service. Gracia-Sanchez (2007) use DEA to assess the performance of the Spanish National Force. Gorman and Ruggiero (2008) apply DEA to assess performance of 49 U.S. continental states police services. Verma and Gavirneni (2006) apply this method on Indian State Police units. Finally, Wu, Chen, and Yeh, (2010) apply this method on Taiwanese police precincts.

Carrington, R.; Puthucheary, N.; Rose, D., and Yaisawarng,S. (1997). Performance Measurement in Government Service Provision: The Case of Police Services in New South Wales. Journal of Productivity Analysis, 8(4), 415-430.

This article describes the implementation of data envelopment analysis in the New South Wales (NSW) Police Service and assessing the efficiency of patrols.

Gracia-Sanchez, I-M. (2007). Evaluating the effectiveness of the Spanish police force through data envelopment analysis. European Journal of Law and Economics, 23(1), 43-57. doi:10.1007/s10657-007-9004-z.

This article assesses the performance of the Spanish National Police Force using the data envelopment analysis. The authors were specifically interested in examining the factors that characterize effective police units.

Gorman, M.F., and Ruggiero, J. (2008). Evaluating US state police performance using data envelopment analysis. International Journal of Production Economics, 113(2), 1031-1037.

The authors assess the performance of 49 U.S. continental state police services using data envelopment analysis while considering demographic variables.

Verma, A., and Gavirneni, S. (2006). Measuring police efficiency in India: An application of data envelopment analysis. Policing: An International Journal of Police Strategies and Management, 29(1), 125-145. doi:10.1108/13639510610648520.

This article assesses policing efficiency in Indian State police units using data envelopment analysis. The study identifies four inputs (expenditure, staffing, investigators, and cases investigated) and four outputs (number of

Tiwana et al. Crime Science (2015) 4:1 Page 20 of 28

arrests, charges, convictions and trials) and compares across Indian police units. It identifies some police units as performing better than others and identifies possible implications for resource allocation.

Wu, T-H., Chen, M-S., and Yeh, J-Y. (2010). Measuring the performance of police forces in Taiwan using data envelopment analysis. Evaluation and Program Planning, 33(3), 246-254. doi:10.1016/j. evalprogplan.2009.09.001.

This article assesses the efficiency of police precincts in Taiwan using data envelopment analysis (DEA). The DEA also included environmental factors and found that although these factors impacted the differences in efficiencies among the precincts, however the effects were not statistically significant.

Garca-Snchez, I-M., Rodrguez-Domnguez, L., and Parra-Domnguez, J. (2013). Yearly evolution of police efficiency in Spain and explanatory factors. Central European Journal of Operations Research, 21(1), 31-62. doi:10.1007/s10100-011-0207-6.

This study uses data envelopment analysis to assess police efficiency in Spains 52 provinces. It uses police officer numbers as the input and solved crimes (clearances) as the output while considering exogenous variable including GDP, unemployment rate, immigration, population, youth population, and province size.

CriticismsWhile criticisms of performance management regimes are present in many of the studies elsewhere in this bibliography, this section contains studies not listed elsewhere.

Survey MeasuresFeilzer, M.Y. (2009). Not fit for purpose! The (ab-) use of the British Crime survey as a performance measure for individual police forces. Policing: A Journal of Policy and Practice, 3(2), 200-211. doi:10.1093/police/ pap009

This study fires a shot across the bows of the use of the British Crime Survey for developing measures of police performance. Its focus is methodological issues, particularly what it argues are the lack of validity and reliability of the survey questions.

Unintended ConsequencesMeasuring performance can have negative side-effects. Collier (2001) argues that performance measurement conflicts with basic human rights. Collier (2007) argues

that performance measurement is used as a disciplinary measure by the government. Patrick (2009) investigates the impacts of gaming on police performance. Eterno and Silverman (2012) examine negative effects of Compstat in New York.

Collier, P.M. (2001). Police performance measurement and human rights. Public Money and Management, 21(3), 35-39.

In this article, the author argues that human rights may be compromised as police officers strive to meet performance measurement targets. It suggests that greater officer autonomy and a values based learning paradigm based on the professional clinician model would be preferable.

Collier, P.M., (2007). Police performance: sovereignty, discipline and governmentality. In A. Neely (Ed.), Business performance measurement (2nd ed.)

(pp.363-382). Cambridge, UK: Cambridge University Press.

This is a chapter in a book about performance measurement in business. The chapter argues that performance measurement is knowledge and that knowledge is power. Collier examines the work of Michael Foucault to support his arguments. It argues that assessing police performance is a disciplinary measure applied by the government to control the police.

Patrick, R. (2009). Performance management, gaming and police practice. (Unpublished doctoral dissertation). University of Birmingham, United Kingdom.

This dissertation focuses on how police performance measurement can result in gaming such as the manipulation of statistics. It suggests that this in turn negatively affects the measures that result.

Eterno, J. A. and E. B. Silverman. 2012. The Crime Numbers Game: Management by Manipulation. Boca Raton, Florida: Taylor and Francis.

This book is a detailed criticism of Compstat performance management on policing in New York. Its focus is providing evidence that police use a variety of gaming techniques to manipulate crime statistics.

TargetsThe literature included in this section examines the use of targets in measuring police performance. All the works included in this category find that targets are established arbitrarily with not rational reason for the

Tiwana et al. Crime Science (2015) 4:1 Page 21 of 28

numerical targets chosen. Guilfoyles (2011) blog post argues against the use of targets when measuring police performance considering nothing beneficial comes out of the use of arbitrary targets. Guilfoyle (2012) similarly discusses the use of targets in the public sector in general. Guilfoyle (2013) highlights the impacts of arbitrary numerical targets on policing and the unintentional consequences. Loveday (2008) observes the impacts of targets on leadership abilities in the public sector.

Guilfoyle, S. (2011, April). Crime in progress: The impact of targets on police service deliver [Blog post]. http://inspguilfoyle.wordpress.com/crime-in-progress-the-impact-of-targets-on-police-service-delivery/

Web End =http://inspguilfoyle.wordpress.com/crime-in-progress-the- http://inspguilfoyle.wordpress.com/crime-in-progress-the-impact-of-targets-on-police-service-delivery/

Web End =impact-of-targets-on-police-service-delivery/ (Accessed:25 November 2013).

This is a blog written by a police inspector who, as the publications below show, has published various studies on police performance measurement. This is a critique of the use of targets as arbitrary.

Guilfoyle, S. (2012). On target? Public sector performance management: Recurrent themes, consequences and questions. Policing: A Journal of Policy and Practice, 6(3), 250-260. doi:10.1093/police/ pas001.

This article provides an overview of the use of targets in the public sector. It provides the reader with an historical account and the reasons why they have been popular despite the negative consequences.

Guilfoyle, S. (2013). Intelligent policing: How systems thinking methods eclipse conventional management practice. Devon, UK: Triarchy Press.

This book outlines the systems thinking approach to police performance management. It includes critiques of other forms of performance management. Targets are the sole focus of one chapter (and argues they should be scrapped) but the book is valuable in excess of this critique.

Loveday, B. (2008). Performance management and the decline of leadership within public services in the United Kingdom. Policing: A Journal of Policy and Practice, 2(1), 120-130. doi:10.1093/police/pam070.

This article distinguishes between management (achieving set goals) and leadership (setting the right goals and vision). It argues that performance measurement targets lead to management-driven policing that lacks leadership, and that it results in gaming (particularly the manipulation of indicators). It proposes a

systems approach that encourages management discre

tion and autonomy.

Performance Evaluation of IndividualsThis section is included to clarify the distinction between the measurement of performance in organisations, and that of individual police officers. Butterfield, Edwards, and Woodall (2004) look at the individual performance of police sergeants in the era of New Public Management. Shane (2010) introduces the performance management framework to assess individual police performance.

Butterfield, R., Edwards, C., and Woodall, J. (2004). The new public management and the UK police service: The role of the police sergeant in the implementation of performance management. Public Management Review, 6(3), 395-415. doi:10.1080/1471903042000256556.

The focus of this article is the impact of New Public Management NPM) on individual performance management of the police sergeant in the UK. The authors found that the introduction of NPM brought about many challenges for the police sergeants and performance management.

Shane, J.M. (2010). Performance management in police agencies: A conceptual framework. Policing: An International Journal of Police Strategies and Management, 33(1), 6-29. doi:10.1108/13639511011020575.

The author introduces the performance management framework. This aim of this framework is to assess individual performance in terms of the process undertaken to achieve institutional objectives.

Shane, J.M. (2009). What every chief executive should know: Using data to measure police performance. New York: Looseleaf Law.

This book focuses on individual performance as part of organisational performance. It identifies five groups as its audience: police executives, crime analysts and police planners, consultants, and business.

Practitioner GuidesThis section includes guides for practitioners, mainly government documents on how to measure performance.

Roberts, D.J. (2006). Law enforcement tech guide for creating performance measures that work: A guide for executives and managers. Washington, DC: Office of Community Oriented Policing Studies, U.S. Department of Justice.

Tiwana et al. Crime Science (2015) 4:1 Page 22 of 28

This is a guide published by the Office of Community Oriented Policing Studies. This guide has been developed to assist in assessing and managing performance of individual programs in law enforcement.

Crime- Specific Performance MeasuresThis section includes literature outlining performance measures specific to crime type. The crime types included are drugs, crime prevention and reduction, and family violence.

Drug Law EnforcementThe literature included in this section argues for the development of specific performance indicators to measure the performance of drug law enforcement initiatives.

Dorn, N. (2000). Performance management, indicators and drug enforcement: In the Crossfire or at the Crossroads. Crime Prevention Studies, 11. 299-318.

This article does not highlight specific drug enforcement indicators. Instead, it argues for the development and importance of these drug specific indicators.

Willis, K., and Anderson, J. (2010). Foundations for an effective performance measurement system for drug law enforcement. Hobart, Tas: National Drug Law Enforcement Research Fund.

This is a guide for drug law enforcement practitioners. The authors have established a 12 step process on the development and implementation of a performance measurement framework for drug enforcement.

Crime Prevention and ReductionReducing crime is a key goal of the police and so arguably an important area in which performance should be measured. Tilley (1995) examines how indicators to measure police effectiveness in preventing crime can be developed. Farrell, Edmunds, Hobbs, and Laycock (2000) examine progress among UK police forces to develop policies and practices to prevent repeat victimization. Karn (2013) provides an overview of the relationship between policing activities and crime reduction. Similarly, Walker, Sauvageau, Johnson, and Walker (2001) consult stakeholders such as experts, the police, community, and business representatives to gain an understanding of how to evaluate crime prevention initiatives.

Tilley, N. (1995). Thinking about crime prevention performance indicators. Crime Detection and Prevention Series Paper 57. London, UK: Home Office.

The focus of this study is performance indicators for police that relate to crime prevention. It argues that police respond to crime, and so levels of repeat victimisation

might be a useful gauge of police effectiveness in that respect. It publication was related to the introduction of repeat victimisation as a key performance indicator for policing in the UK.

Farrell, G., Edmunds, A., Hobbs, L., and Laycock,G. (2000). RV Snapshot: UK policing and repeat victimization. Crime Reduction Research Series Paper 5. London, UK: Home Office.

Preventing repeat victimization was introduced as a key performance indicator for UK policing. This research study details progress by police forces in developing definitions, policy and practice to prevent repeat victimization, finding significant variation across police forces.

Karn, J. (2013). Policing and crime reduction: The evidence and its implications for practice. London, UK: The Police Foundation.

This document provides background information for the four year project, Police Effectiveness in a Changing World. It examines the relationship between the police and crime reduction.

Walker, J., Sauvageau, J., Johnson, C., and Walker,C. (2001). Crime prevention performance indicators. Ottawa, ON: Ottawa Police Service.

The purpose of the document is to identify indicators which adequately capture policing activity to reduce crime. This document introduces an evaluation Tool Kit to evaluate these programs.

Family ViolenceDepartment of Justice Canada. (2004). Project managers guide to performance measurement and evaluation. Family violence initiative. Ottawa, ON: Department of Justice Canada.

This is a guide developed by the Department of Justice Canada and prepared for staff involved with the Family Violence Initiative. It addresses various evaluation concerns and includes performance indicators to assess the Family Violence Initiative.

Police Department Plans and EvaluationsThe documents in this section are police department strategic plans which include performance measurement plans. The City of Toronto (2011) document highlights six key areas in which policing effectiveness and efficiency strategies are recommended for improvement. The Toronto Police Service (2013) identifies the key priorities for the year and associated indicators to assess

Tiwana et al. Crime Science (2015) 4:1 Page 23 of 28

performance. The Vancouver Police Department (2013) annual business plan for 2013 describes the strategies to be undertaken to achieve long term goals for 2016 and the corresponding performance measures.

City of Toronto. (2011). Toronto police service: Service efficiency study. Final report to city manager. Toronto, ON: Ernst and Young.

The purpose of this report was to assess the six key areas of policing which were highlighted by the City Manager to increase the Tonto Police Service efficiency and effectiveness in. These areas included: staffing level, shift schedule, call taking and dispatch, emergency management, towing and pounds management, and school crossing guard program.

Toronto Police Service. (2013). Toronto police service 2013 business plan. Toronto, ON: Toronto Police Service.

This is the 2013 business plan for the Toronto Police Service. It includes nine service priorities and corresponding performance indicators.

Vancouver Police Department. (2013). Vancouver police department 2013 annual business plan. Vancouver, BC: Vancouver Police Department.

This is the Vancouver Police Department annual business plan for 2013. It outlines the goals set in the Departments five year strategic plan (2012-2016) and the key strategies undertaken in 2013 including the associated activities, outcomes, and target measures for each of the strategies.

Annotated Bibliographies in Related Areas Performance measurement is widespread. This section lists annotated bibliographies in related areas. Marteache and Maxfield (2011) overlaps with the present study and provides an excellent annotated bibliography of selected readings on performance measures in criminal justice. The Centre for Problem-Oriented Policing (n.d.) provides an annotated bibliography focused on problem-oriented policing, Whitfields (2007) selected annotated bibliography covers 12 studies of Compstat, Cabula (2009) focuses on corrections and related areas. The Centre for Educator Compensation Reform (2010) study is the only one included here that is from outside the criminal justice sector, by means of illustration.

http://www.oxfordbibliographies.com.proxy.lib.sfu.ca/ViewContributor/document/obo-9780195396607/obo-9780195396607-0106.xml?id=con0600#_self

Web End =Marteache , N. and Maxfield, M. (2011). Performance measurement and accountability systems. Oxford Bibliographies Online. doi:10.1093/OBO/ 9780195396607-0106.

This is an excellent annotated bibliography including references to general literature on organizational level performance measurement across the criminal justice sector.

Center for Problem-Oriented Policing. (n.d.). Annotated bibliography of studies implementing problem-oriented policing. http://www.popcenter.org/library/reading/PDFs/Annotated-Bibliography-of-Studies-of-Implementing-Problem-Oriented-Policing.pdf

Web End =http://www.popcenter.org/library/reading/PDFs/ http://www.popcenter.org/library/reading/PDFs/Annotated-Bibliography-of-Studies-of-Implementing-Problem-Oriented-Policing.pdf

Web End =Annotated-Bibliography-of-Studies-of-Implementing- http://www.popcenter.org/library/reading/PDFs/Annotated-Bibliography-of-Studies-of-Implementing-Problem-Oriented-Policing.pdf

Web End =Problem-Oriented-Policing.pdf (Accessed: 14 December 2013).

This annotated bibliography includes many studies on the implementation of problem-oriented policing.

Whitfield, K. (2007). CompStat: A selected bibliography. The Police Chief, 74(2).

This annotated bibliography is published online by The Police Chief. The selected bibliography includes 12 references ranging from The Police Chief, the FBI Law Enforcement Bulletin, Washington Crime News Service, and NIJ Journal. These documents address CompStat process, implementation, and other key areas of CompStat important to acknowledge.

Cabula, N. (2009). Organization performance annotated bibliography. http://community.nicic.gov/blogs/hpco/archive/2009/05/31/organization-performance-annotated-bibliography.aspx

Web End =http://community.nicic.gov/blogs/ http://community.nicic.gov/blogs/hpco/archive/2009/05/31/organization-performance-annotated-bibliography.aspx

Web End =hpco/archive/2009/05/31/organization-performance- http://community.nicic.gov/blogs/hpco/archive/2009/05/31/organization-performance-annotated-bibliography.aspx

Web End =annotated-bibliography.aspx (Accessed: 14 December 2013).

This is published on the National Institute of Corrections, U.S. Department of Justice website and coves the literature on organizational performance in various setting related to corrections including counselling, psychology, and the justice sector.

Center for Educator Compensation Reform. (2010). Performance measures. http://www.cecr.ed.gov/pdfs/annotatedBibliographies/Bib_PerformMsrs.pdf

Web End =www.cecr.ed.gov/pdfs/anno http://www.cecr.ed.gov/pdfs/annotatedBibliographies/Bib_PerformMsrs.pdf

Web End =tatedBibliographies/Bib_PerformMsrs.pdf

This annotated bibliography was prepared and published for the Center for Educator Compensation Reform and focuses on performance measures and evaluations for educators and schools, especially as those evaluations relate to compensation (p.1).

Discussion and Conclusion

Over 200 studies relating to police performance measurement, including a few relating to performance measurement more generally, were included in this study. Our hope is that this annotated bibliography should prove of utility to those involved in work in this area, whether in policing, government, research or another sector. The authors would be happy to receive notice of

Tiwana et al. Crime Science (2015) 4:1 Page 24 of 28

key studies that were either omitted or that have arisen since the searches for this study were undertaken.

Performance measurement is emerging as an important topic in policing, as in many other areas of the public and private sector. The trajectory of its rise to prominence has differed from one country to the next. In many countries it remains in what might be termed the early stages with a few fairly traditional indicators being incorporated by larger police departments into their evolving strategic plans. In contrast, in the United Kingdom, performance measures for policing have been part of a national agenda for over two decades. In that country the measures evolved significantly over time, becoming a large set that were gradually centralised, systematised and integrated with other public sector measures and the national victimization survey. More recently that country has discarded many of those measures, though the extent to which those reforms were driven by political or substantive issues remains debated, and the future of performance measures for policing remains uncertain at the time of writing. The decentralised structure of policing in other countries, most notably the United States, means that performance measurement has evolved on a more piecemeal basis elsewhere.

A difficult issue is how performance measures should be used. They can be used to trigger all three of the mechanisms by which policy instruments work: as carrots (incentives or rewards for good performance), as sticks (disincentives or punishment for poor performance think early Compstat), or as sermons (providing information about good, bad and comparative performance). They are one of the few policy instruments with that flexibility and potential, which means that how they are used perhaps wielded is a better term is as important as what they gauge. This flexibility, combined with the fact that the evidence suggests performance measures can clearly be extremely powerful in affecting police behaviour in both positive and negative ways, means they should, like wild horses, be handled with care.

A key concern has arisen that results from the power of performance indicators. The concern is negative side-effects. This includes the manipulation of statistics by police, and even the unwarranted perversion of practice, in order to meet performance targets. This makes the whole topic controversial, as it then can become unclear whether the benefits of performance measurement have exceeded the costs. There is also a concern that police management focused on targets can, by reducing professional autonomy, reduce strategic leadership capability at the local level.

With respect to indicators themselves, the evidence suggests a paradox: to the extent that there are some useful performance indicators, they tend to be the least readily available and most expensive to produce. The

simple indicators that can be produced cheaply from routinely-collected administrative data the crimes rates, clearance rates, response times and their ilk are also typically the least informative (in terms of performance) and most prone to negative side-effects. The more informative indicators of public satisfaction and confidence in the police, of procedural justice (treatment by the police) and of reductions in crime and disorder due to policing (after all, thats why we have police), are more difficult and typically more expensive to capture, often requiring dedicated surveys.

While this concluding discussion is brief, it is our attempt to synthesis some of the information from the annotated bibliography. The evidence suggests to us that performance measurement for policing should be developed and conducted with caution. It may be preferable to frame many traditional measures as contextual indicators, or simply information for police managers, rather than performance indicators, in order to seek to maintain the benefits of the information they provide while mini-mising the risk of negative side-effects. This is particularly true for the range of traditional indicators (such as crime rates, clearance rates, and response times) that are, at best, indirect and partial indicators of performance. The cross-jurisdictional or over-time comparison of activities and outputs can be informative, and used to promote communication with respect to good and bad practice, without necessarily being used as a stick with which to beat those who appear to be below average; particularly since fifty percent of every group always falls below the mean. This provision of information sits well with the philosophy of open governance and transparency that is increasingly important in democratic society. If this police-related information can be set alongside indicators that better gauge performance (albeit always imperfectly), most notably those based on surveys of the public and police employees, and if this portfolio can be tailored and managed in a manner appropriate to its local context, then these, we suggest, may be the key ingredients of a recipe for success.

Additional References that are Not Annotated

This section provides a selection of documents that are not annotated either because the topic was largely covered above or because the main theme is related but outside policing. The alternative was to exclude them, but we suggest there is some value from including at least some of the broader picture.

Boston, J. (1993). Financial management reform: Principles and Practice in New Zealand. Public Policy and Administration, 8(1), 14-29. doi:10.1177/095207679300800102.

Fleming, J. and Lafferty, G. (2000). New management

techniques and restructuring for accountability in

Tiwana et al. Crime Science (2015) 4:1 Page 25 of 28

Australian police organizations. Policing: An International Journal of Police Strategies and Management, 23(2), 154-168.

Kaplan, R.S., and Norton, D.P. (2006). Why system, not structure, is the way toward strategic alignment: A historical perspective. Balanced Scorecard Report, 8(4), 1-16.

Kaplan, R.S., Norton, D.P., and Rugelsjoen, B. (2010). Managing alliances with the balanced scorecard. Harvard Business Review, 1-9.

Lawrie, G., and Cobbold, I. (2002). Development of the 3rd generation balanced scorecard: Evolution of the balanced scorecard into an effective strategic performance management tool. Oxford, UK: 2GC Active Management.

Lawrie, G., Kalff, D., and Andersen, H. (2005, September). Balanced scorecard and results based management: Convergent performance management systems. Paper presented at the 3rd Annual Conference on Performance Measurement and Management Control, The European Institute for Advanced Studies in Management (EIASM), Nice, France. http://2gc.eu/files/resources/2GC-CP-BSCRBM-090216.pdf

Web End =http://2gc.eu/files/resources/2GC-CP-BSCRBM- http://2gc.eu/files/resources/2GC-CP-BSCRBM-090216.pdf

Web End =090216.pdf (Accessed: 19 October 2013).

Nyhan, R.C., and Martin, L.L. (1999) Assessing the performance of municipal police services using data envelopment analysis: an exploratory study. State and Local Government Review, 31(1), 1830.

Orlikowsky, W.J. and Gash, D.C. (1994). Technological frames: Making sense of information technology in organizations. ACM Transactions on Information Systems, 12(2), 174-207.

Vickers, M.H., and Kouzmin, A. (2001). New managerialism and Australian police organizations: A cautionary research note. The International Journal of Public Sector Management, 14(1), 7-26.

Smith, P.C., and Street, A. (2005). Measuring the efficiency of public services: the limits of analysis. Journal of the Royal Statistical Society Series A (Statistics in Society), 168(2), 401-417.

Sun, S. (2002). Measuring the relative efficiency of police precincts using data envelopment analysis. Socio-Economic Planning Sciences, 36, 51-71.

Competing interestsThe authors declare that they have no competing interests.

Authors contributionsNT wrote the background, methods, results, discussion, conclusion, references, and collected and reviewed the literature. GB developed the outline for the paper, collected and reviewed the literature, and reviewed the results. GF wrote the background, discussion, conclusion, developed the outline for the paper, and collected and reviewed the literature. All authors read and approved the final manuscript.

Received: 1 April 2014 Accepted: 17 October 2014

ReferencesAlach, Z, & Crous, C. (2012). A tough nut to crack: Performance measurement in specialist policing. Canberra, ACT: Australian Institute of Criminology.

Ashby, DI. (2005). Policing neighbourhoods: Exploring the geographies of crime, policing and performance assessment. Policing and Society: An International Journal of Research and Policy, 15(4), 413447. doi:10.1080/10439460500310079. Barton, A. (2013). Lean policing? New approaches to business process improvement across the UK police service. Public Money and Management, 33(3), 221224.

Behn, RD. (2003). Why measure performance: Different purposes require different measures. Public Administration Review, 63(5), 586606.

Bevan, G, & Hood, C. (2006). Whats measured is what matters: Targets and gaming in the English public health care system. Public Administration,84(3), 517538.

Bird, SM, Cox, D, Farewell, VT, Goldstein, H, Holt, T, & Smith, PC. (2005).Performance indicators: Good, bad, and ugly. Journal of the Royal Statistical Society. Series A (Statistics in Society), 168(1), 127.

Boston, J. (1993a). Financial management reform: Principles and Practice in NewZealand. Public Policy and Administration, 8(1), 1429. doi:10.1177/ 095207679300800102.

Boston, J. (1993b). Financial management reform: Principles and Practice in NewZealand. Public Policy and Administration, 8(1), 1429. doi:10.1177/ 095207679300800102.

Bouza, AV. (1978). Police administration: Organization and performance. Elmsford,New York: Pergamon Press Inc.

Boyd, E, Geoghegan, R, & Gibbs, B. (2011). Cost of the cops: Manpower and deployment in policing. London, UK: Policy Exchange.

Bratton, WJ, & Malinowski, SW. (2008). Police performance management in practice: Taking COMPSTAT to the next level. Policing: A Journal of Policy and Practice, 2(3), 259265. doi:10.1093/police/pan036.

Brown, K, & Coulter, PB. (1983). Subjective and objective measures of police service delivery. Public Administration Review, 43(1), 5058.

Burrows, J, Tarling, R, Mackie, A, Lewis, R, & Taylor, G. (2000). Review of police forces crime recording practices. London, UK: Home Office.

Butterfield, R, Edwards, C, & Woodall, J. (2004). The new public management and the UK police service: The role of the police sergeant in the implementation of performance management. Public Management Review, 6(3), 395415. doi:10.1080/1471903042000256556.

Butterfield, R, Edwards, C, & Woodall, J. (2005). The new public management and managerial roles: The case of the police sergeant. British Journal of Management, 16, 329341. doi:10.1111/j.1467-8551.2005.00466.x.

Cabula, N. (2009). Organization performance annotated bibliography. http://community.nicic.gov/blogs/hpco/archive/2009/05/31/organization-performance-annotated-bibliography.aspx

Web End =http:// http://community.nicic.gov/blogs/hpco/archive/2009/05/31/organization-performance-annotated-bibliography.aspx

Web End =community.nicic.gov/blogs/hpco/archive/2009/05/31/organization- http://community.nicic.gov/blogs/hpco/archive/2009/05/31/organization-performance-annotated-bibliography.aspx

Web End =performance-annotated-bibliography.aspx (Accessed: 14 December 2013). Carlson, HM, & Sutton, MS. (1981). A multimethod approach to community evaluation of police performance. Journal of Criminal Justice, 9(3), 227234. Carmona, S, & Gronlund, A. (2003). Measures VS. actions: The balanced scorecard in Swedish law enforcement. International Journal of Operations and Production Management, 23(12), 14751496. doi:10.1108/01443570310506722. Carrington, R, Puthucheary, N, Rose, D, & Yaisawarng, S. (1997). Performance

Measurement in Government Service Provision: The Case of Police Servicesin New South Wales. Journal of Productivity, 8(4), 415430.

Carter, N, Klein, R, & Day, P. (1992). How organisations measure success: The use of performance indicators in government. New York, NY: Routledge.

Center for Educator Compensation Reform. (2010). Performance measures. http://www.cecr.ed.gov/pdfs/annotatedBibliographies/Bib_PerformMsrs.pdf

Web End =www. http://www.cecr.ed.gov/pdfs/annotatedBibliographies/Bib_PerformMsrs.pdf

Web End =cecr.ed.gov/pdfs/annotatedBibliographies/Bib_PerformMsrs.pdf . Charbonneau, E, & Riccucci, NM. (2008). Beyond the usual suspects: An analysis of the performance measurement literature on social equity indicators in policing. Public Performance and Management Review, 31(4), 604620. doi:10.2753/pmr153-9576310405.

Chew, E, Swanson, M, Stine, K, Bartol, N, Brown, A, & Robinson, W. (2008).Performance measurement guide for information security. Gaitherburg, MD: National Institute of Standards and Technology, U.S. Department of Commerce.

City of Toronto. (2011). Toronto police service: Service efficiency study. Final report to city manager. Toronto, ON: Ernst and Young.

Coleman, TG. (2012). A model for improving the strategic measurement and management of policing: The police organizational performance index (POPI). University of Regina, Saskatchewan: Unpublished doctoral dissertation. Collier, PM. (2001). Police performance measurement and human rights. Public

Money and Management, 21(3), 3539.

Tiwana et al. Crime Science (2015) 4:1 Page 26 of 28

Collier, PM. (2006). In search of purpose and priorities: Police performance indicators in England and Wales. Public Money and Management, 26(3), 165172.

Collier, PM. (2007). Business performance measurement. In A Neely (Ed.), Police performance: sovereignty, discipline and governmentality (2nd ed., p. 363). Cambridge, UK: Cambridge University Press.

Commission, SS. (2012). Performance improvement framework: Formal review of the New Zealand Police. Wellington, New Zealand: State Services Commission. Cuganesan, S, & Lacey, D. (2011). Developments in public sector performance measurement: A project on producing return on investment metrics for law enforcement. Financial Accountability and Management, 27(4), 458479. Dadds, V, & Scheide, T. (2000). Police performance and activity measurement

(Trends and Issues in Crime and Criminal Justice No. 180). Canberra, ACT: Australian Institute of Criminology.

Davis, RC. (2012). Selected international best practices in police performance measurement. Santa Monica, CA: RAND Corporation.

Decker, SH. (1980). The effect of police characteristics on alternative measures of police output. Criminal Justice Review, 5(2), 3440. doi:10.1177/073401688000500206. Department of the environment, transport and the regions. (1999). DETR: Best value and audit commission performance indicators for 2000/2001. Volume One: The performance indicators. London, UK: Department of the environment, transport and the regions.

Dooren, WV. (2011). Better performance management: Some single and double-loop strategies. Public Performance and Management Review, 34(3), 420433. doi:10.2753/PMR1530-957634030.

Drake, LM, & Simper, R. (2001). The economic evaluation of policing activity: An application of a hybrid methodology. European Journal of Law and Economics, 12, 173192.

Drake, LM, & Simper, R. (2003). An evaluation in the choice of inputs and outputs in the efficiency measurement of police forces. Journal of Socio-Economics, 32, 701710.

Drake, LM, & Simper, R. (2005). Police efficiency in offences cleared: An analysis of English Basic command units. International Review of Law and Economics, 25, 186208. doi:10.1016/j.irle.2005.06.003.

Eck, EJ, & Maguire, ER. (2000). Have changes in policing reduced violent crime?

Assessment of the evidence. In A Blumstein & J Wallman (Eds.), The crime drop in America (pp. 207265). New York, NY: Cambridge University Press. Eterno, JA, & Silverman, EB. (2012). The Crime Numbers Game: Management by

Manipulation. Boca Raton, Florida: Taylor and Francis.

Farrell, G, Edmunds, A, Hobbs, L, & Laycock, G. (2000). RV Snapshot: UK policing and repeat victimization (Crime Reduction Research Series Paper 5). London, UK: Home Office.

Fellows Associates. (2010). Sustaining value for money in the police service and valuing the police. London, UK: Fellows Associates.

Fielding, N, & Innes, M. (2006). Reassurance policing, community policing and measuring police performance. Policing and Society: An International Journal of Research and Policy, 16(2), 127145. doi:10.1080/10439460600662122. FitzGerald, M. (2010). A confidence trick? Policing: A Journal of Policy and Practice,

4(3), 298301. doi:10.1093/police/paq025.

Flanagan, R. (2008a). The review of policing: Final report. Surrey, UK: PoliceFederation of England and Wales.

Flanagan, R. (2008b). The review of policing: Equality impact assessment report.London, UK: Home Office.

Fleming, J, & Scott, A. (2008). Performance measurement in Australian police organizations. Policing: A Journal of Policy and Practice, 2(3), 322330. doi:10.1093/police/pan038.

Garca-Snchez, I-M. (2009). Measuring the efficiency of local police force. EuropeanJournal of Law and Economics, 27(1), 5977.

Garca-Snchez, I-M, Rodrguez-Domnguez, L, & Parra-Domnguez, J. (2013a). Evaluation of the efficacy and effectiveness of the Spanish security forces. European Journal of Law and Economics, 36(1), 5775. doi:10.1007/s10657-011-9265-4.

Garca-Snchez, I-M, Rodrguez-Domnguez, L, & Parra-Domnguez, J. (2013b).Yearly evolution of police efficiency in Spain and explanatory factors. Central European Journal of Operations Research, 21(1), 3162. doi:10.1007/s10100-011-0207-6.

Garca-Snchez, I-M, Rodrguez-Domnguez, L, & Parra-Domnguez, J. (2013c).Yearly evolution of police efficiency in Spain and explanatory factors. Central European Journal of Operations Research, 21(1), 3162. doi:10.1007/s10100-011-0207-6.

Gomes, PS, Mendes, SM, & Carvalho, JB. (2006). Paper presented at the 4thInternational Conference on Accounting, Auditing and Management in Public

Sector Reforms. Siena: Italy. Performance measurement of the Portuguese police force using the balanced scorecard.

Gorman, MF, & Ruggiero, J. (2008). Evaluating US state police performance using data envelopment analysis. International Journal of Production Economics, 113(2), 10311037.

Gracia-Sanchez, I-M. (2007). Evaluating the effectiveness of the Spanish police force through data envelopment analysis. European Journal of Law and Economics, 23(1), 4357. doi:10.1007/s10657-007-9004-z.

Guilfoyle, S. (2011). Crime in progress: The impact of targets on police service deliver

[Blog post]. http://inspguilfoyle.wordpress.com/crime-in-progress-the-impact-of-targets-on-police-service-delivery/

Web End =http://inspguilfoyle.wordpress.com/crime-in-progress-the-impact- http://inspguilfoyle.wordpress.com/crime-in-progress-the-impact-of-targets-on-police-service-delivery/

Web End =of-targets-on-police-service-delivery/ (Accessed: 25 November 2013). Guilfoyle, S. (2012). On target? Public sector performance management:

Recurrent themes, consequences and questions. Policing: A Journal of Policy and Practice, 6(3), 250260. doi:10.1093/police/pas001.

Guilfoyle, S. (2013). Intelligent policing: How systems thinking methods eclipse conventional management practice. Devon, UK: Triarchy Press.

Hale, C, Heaton, R, & Uglow, S. (2004). Uniform styles? Aspects of police centralization in England and Wales. Policing and Society: An International Journal of Research and Policy, 14(4), 291312.

Hale, CM, Uglow, SP, & Heaton, R. (2005). Uniform styles II: Police families and policing styles. Policing and Society: An International Journal of Research and Policy, 15(1), 118. doi:10.1080/1043946042000338904.

Hallam, SA. (2009). Policing in the iron cage: The tensions between the bureaucratic mandate and street level reality. University of Bedfordshire, Luton, Bedford, UK: Unpublished doctoral dissertation.

Hatry, H, & Davies, E. (2011). A guide to data-driven performance reviews (Improving Performance Series). Washington, DC: IBM Centre for The Business of Government.

Heaton, P. (2010). Hidden in plain sight: What cost-of-crime research can tell us about investing in police. Santa Monica, CA: RAND Corporation.

Her Majestys Inspectorate of Constabulary. (2010). Valuing the police: Policing in an age of austerity. London, UK: Her Majestys Inspectorate of Constabulary. Heyer, GD. (2012). Cost-saving efforts of the New Zealand Police: Opportunities for the United States. The Police Chief, 79(12), 6063.

Hoffman, RB. (1971). Performance measurements in crime control. Journal of Research in Crime and Delinquency, 8(2), 165174. doi:10.1177/002242787100800205. Home Office. (2008a). Local government, England and Wales police, England and

Wales. The police authorities (best value) performance indicators order 2008. London, UK: Home Office.

Home Office. (2008b). Definitions and survey guidance for APACS measures of user satisfaction 2008/9. London, United Kingdom: Home Office.

Home Office. (2008c). Guidance on statutory performance indicators for policing and community safety 2008/09. London, United Kingdom: Home Office. Home Office, Association of Police Authorities. (2006). Police performance management: Practical guidance for police authorities. London, UK: Association of Police Authorities, Home Office.

Humphreys, KA, & Trotman, KT. (2011). The balanced scorecard: The effect of strategy information on performance evaluation judgments. Journal of Management Accounting Research, 23, 8198. doi:10.2308/jmar-10085. Hunton, P, Jones, A, & Baker, P. (2009). New development: Performance management in a UK police force. Public Money and Management, 29(3), 195200. Independent Police Commission. (2013). Policing for a better Britain: Report of the

Independent Police Commission (The Stevens Report). Essex, UK: The Lord Stevens of Kirkwhelpington QPM.

Jang, H, Joo, H-J, & Zhao, J. (2010). Determinants of public confidence in police:An international perspective. Journal of Criminal Justice, 38, 5768. doi:10.1016/j.jcrimjus.2009.11.008.

Kaplan, RS, & Norton, DP. (1992). The balanced scorecard Measures that drive performance. Harvard Business Review, 7179.

Kaplan, RS, & Norton, DP. (1993). Putting the balanced scorecard to work. HarvardBusiness Review, 71(5), 134147.

Kaplan, RS, & Norton, DP. (2007). Using the balanced scorecard as a strategic management system. Harvard Business Review, 114.

Karn, J. (2013). Policing and crime reduction: The evidence and its implications for practice. London, UK: The Police Foundation.

Kennerley, M, & Neely, A. (2003). Measuring performance in a changing business environment. International Journal of Operations and Production Management, 23(2), 213229. doi:10.1108/01443570310458465.

Kovandzic, TV, & Sloan, JJ. (2002). Police levels and crime rates revisited: A county-level analysis from Florida (19801998). Journal of Criminal Justice, 30, 6576.

Tiwana et al. Crime Science (2015) 4:1 Page 27 of 28

Langworthy, RH (Ed.). (1999). Measuring what matters: Proceedings from the policing research institute meetings. Rockville, MD: National Institute of Justice, Office of Justice Programs, U.S. Department of Justice.

Larson, RC. (1978). Police accountability: Performance measures and unionism.Cambridge, MA: Massachusetts Institute of Technology.

Leigh, A, Mundy, G, & Tuffin, R. (1999). Best value policing: Making preparations(Police Research Series Paper 116). London, UK: Home Office.

Lelandais, I, & Bodson, J. (2007). Measuring police performance: International experiences. Montreal, QC: International Centre for the Prevention of Crime. Levitt, SD. (1998). The relationship between crime reporting and police:

Implications for the use of uniform crime reports. Journal of Quantitative Criminology, 14(1), 6181.

Loveday, B. (2000). Managing crime: Police use of crime data as an indicator of effectiveness. International Journal of the Sociology of Law, 28, 215237. doi:10.1006/ijsl.2000.0124.

Loveday, B. (2008). Performance management and the decline of leadership within public services in the United Kingdom. Policing: A Journal of Policy and Practice, 2(1), 120130. doi:10.1093/police/pam070.

Maguire, ER. (2003). Measuring the performance of law enforcement agencies: Part1 of a 2-part article (CALEA Update Magazine, 83). http://www.calea.org/calea-update-magazine/issue-83/measuring-performance-law-enforcement-agencies-part-1of-2-oart-articl

Web End =http://www.calea.org/calea- http://www.calea.org/calea-update-magazine/issue-83/measuring-performance-law-enforcement-agencies-part-1of-2-oart-articl

Web End =update-magazine/issue-83/measuring-performance-law-enforcement- http://www.calea.org/calea-update-magazine/issue-83/measuring-performance-law-enforcement-agencies-part-1of-2-oart-articl

Web End =agencies-part-1of-2-oart-articl (Accessed: 14 October 2013).

Maguire, ER. (2004). Measuring the performance of law enforcement agencies: Part2. CALEA Update Magazine, 84. http://www.calea.org/calea-update-magazine/issue-84/measuring-performance-law-enforcement-agencies-part-2-2-part-article

Web End =http://www.calea.org/calea-update-magazine/ http://www.calea.org/calea-update-magazine/issue-84/measuring-performance-law-enforcement-agencies-part-2-2-part-article

Web End =issue-84/measuring-performance-law-enforcement-agencies-part-2-2-part- http://www.calea.org/calea-update-magazine/issue-84/measuring-performance-law-enforcement-agencies-part-2-2-part-article

Web End =article . (Accessed: 14 October 2013).

Maguire, ER, & Uchida, CD. (2000). Measurement and Explanation in theComparative Study of American Police Organizations. In D. Duffee (ed.), Criminal Justice 2000: Vol 4. Measurement and Analysis of Crime and Justice. Rockville, MD: National Criminal Justice Reference Service.

Maillard, JD, & Savage, SP. (2012). Comparing performance: the development of police performance management in France and Britain. Policing and Society: An International Journal of Research and Policy, 22(4), 363383.

Marteache, N, & Maxfield, M. (2011). Performance measurement and accountability systems. Oxford Bibliographies Online. doi:10.1093/OBO/ 9780195396607-0106.

Martin, S. (2011). What is localism and does it matter? (Paper presented at theCommonwealth Local Government Forum Research Colloquium: Sustainable local governance for prosperous communities). UK: Cardiff.

Mazerolle, L, Rombouts, S, & McBroom, J. (2006). The impact of operational performance reviews on reported crime in Queensland. Trends and Issues in Crime and Criminal Justice, 313, 16.

Michell, P, & Neely, A. (2010). Performance measurement in the public sector in England: Searching for the golden thread. Public Administration Review, 70(4), 591600. doi:10.1111/j.1540-6210.2010.02180.x.

Moore, MH. (2003). Sizing up COMPSTAT: An important administrative innovation in policing. Criminology and Public Policy, 2(3), 469494. doi:10.1111/j.1745-9133.2003.tb00009.x.

Moore, MH, & Braga, A. (2003a). The bottom line of policing: What citizens should value (and measure!) in police performance. Washington, DC: Police Executive Research Forum.

Moore, MH, & Braga, AA. (2003b). Measuring and improving police performance: The lessons of Compstat and its progeny. Policing: An International Journal of Police Strategies and Management, 26(3), 439453.

Myhill, A, & Beak, K. (2008). Public confidence in the police. London, UK: NationalPolicing Improvement Agency.

Myhill, A, & Bradford, B. (2012). Can police enhance public confidence by improving quality of service? Results from two surveys in England and Wales. Policing and Society: An International Journal of Research and Policy, 22(4), 397425. doi:10.1080/10439463.2011.641551.

Myhill, A, & Quinton, P. (2010). Confidence, neighbourhood policing, and contact: Drawing together the evidence. Policing: A Journal of Policy and Practice, 4(3), 273281. doi:10.1093/police/paq026.

Myhill, A, Quinton, P, Bradford, B, Poole, A, & Sims, G. (2011). It depends what you mean by confident: operationalizing measures of public confidence and the role of performance indicators. Policing: A Journal of Policy and Practice, 5(2), 114124. doi:10.1093/police/par027.

New Zealand Police. (2013). Statement of Intent 2013/20142015/2016. Wellington,New Zealand: New Zealand Police.

Newburn, T. (2003). Policing since 1945. In T Newburn (Ed.), Handbook of policing(pp. 84105). Portland, Oregon: Willan.

Office, H. (2011). The national crime recording standard (NCRS): What you need to know. London, UK: Home Office.

Ostrom, E. (1973). On the meaning and measurement of output and efficiency in the provision of urban police services. Journal of Criminal Justice, 1, 93112.

Parks, RB. (1984). Linking objective and subjective measures of performance.Public Administration Review, 44(2), 118127.

Patrick, R. (2009). Performance management, gaming and police practice (Unpublished doctoral dissertation). United Kingdom: University of Birmingham.

Police Federation. (2013). Policing UK 2013 (Priorities and pressures: A year of transformation). Surrey, UK: Police Federation.

Pollitt, C. (1986). Performance measurement in the public services: some political implications. Parliamentary Affairs, 39(3), 315329.

Savage, SP. (2007). Police reform: forces for change. Oxford: New York, New York. Shane, JM. (2010). Performance management in police agencies: A conceptual framework. Policing: An International Journal of Police Strategies and Management, 33(1), 629. doi:10.1108/13639511011020575.

Shepherd, J, & Sivarajasingam, V. (2005). Injury research explains conflicting violence trends. Injury Prevention, 11(6), 324325. doi:10.1136/ip.2005.009761. Shilston, TG. (2008). One, two, three, what are we still counting for? Police performance regimes, public perceptions of service delivery and the failure of quantitative measurement. Policing: A Journal of Policy and Practice, 2(3), 359366. doi:10.1093/police/pan043.

Shilston, TG. (2011). Black box: A qualitative method for improving public confidence in policing through micro-analysing service delivery. Policing: A Journal of Policy and Practice, 5(2), 125131. doi:10.1093/police/par003. Simmons, J, Legg, C, & Hosking, R. (2003). National crime recording standard

(NCRS): An analysis of the impact on recorded crime. Part one: The national picture. London, UK: Home Office.

Skogan, WG. (1976). Efficiency and effectiveness in big-city police departments.Public Administration Review, 36(3), 278286.

Sluis, AV, & Ringeling, A. (2009). Evolving patterns in the police systems of North Rhine-Westphalia, The Netherlands and England and Wales. German Policy Studies, 5(2), 145168.

Stanko, EA, & Bradford, B. (2009). Beyond measuring how good a job police are doing: The MPS model of confidence in policing. Policing: A Journal of Policy and Practice, 3(4), 322330. doi:10.1093/police/pap047.

Stevens, JM, & McDavid, JC. (1981). Urban police performance attitudes.Computers, Environment and Urban Systems, 6(4), 157169.

Stevens, JM, Webster, TC, & Stipak, B. (1980). Response time: Role in assessing police performance. Public Productivity Review, 4(3), 210230.

The Thin Blue Line. (2011). Crime of the century: A chilling look at crime statistics, police recorded crime and offences brought to justice in the UK. West Midlands, UK: Nice 1 Ltd.

Tilley, N. (1995). Thinking about crime prevention performance indicators (Crime

Detection and Prevention Series Paper 57). London, UK: Home Office. Toronto Police Service. (2013). Toronto police service 2013 business plan. Toronto,

ON: Toronto Police Service.

Treasury Board of Canada Secretariat. (2007). A guide to preparing treasury board submissions. Ottawa, ON: Treasury Board of Canada Secretariat.

Treasury Board of Canada Secretariat. (2009). Handbook for regulatory proposals: Performance measurement and evaluation plan. Ottawa, ON: Treasury Board of Canada Secretariat.

Treasury Board of Canada Secretariat. (2010). Supporting effective evaluations: A

Guide to developing performance measurement strategies. http://www.tbs-sct.gc.ca/cee/dpms-esmr/dpms-esmrtb-eng.asp

Web End =http://www.tbs-sct. http://www.tbs-sct.gc.ca/cee/dpms-esmr/dpms-esmrtb-eng.asp

Web End =gc.ca/cee/dpms-esmr/dpms-esmrtb-eng.asp (Accessed: 26 October 2013). Treasury Board of Canada Secretariat. Guide to the preparation of part III of the

20062007 estimates: Reports on plans and priorities and departmental performance reports. Ottawa, ON: Treasury Board of Canada Secretariat. U.S. Department of Energy. (1996). Guidelines for performance measurement.

Washington, DC: U.S. Department of Energy.

Vancouver Police Department. (2013). Vancouver police department 2013 annual business plan. Vancouver, BC: Vancouver Police Department.

Vera Institute of Justice. (2003). Measuring progress toward safety and justice: A global guide to the design of performance indicators across the justice sector. New York, NY: Vera Institute of Justice.

Verma, A, & Gavirneni, S. (2006). Measuring police efficiency in India: An application of data envelopment analysis. Policing: An International Journal of Police Strategies and Management, 29(1), 125145. doi:10.1108/13639510610648520. Vollaard, BA. (2006). Police effectiveness: Measurement and Incentives ((Doctoral dissertation)). RAND Corporation: Santa Monica, California.

Tiwana et al. Crime Science (2015) 4:1 Page 28 of 28

Walker, J, Sauvageau, J, Johnson, C, & Walker, C. (2001). Crime prevention performance indicators. Ottawa, ON: Ottawa Police Service.

Weisburd, D, Mastrofski, SD, McNally, A, Greenspan, R, & Willis, JJ. (2003). Reforming to preserve: COMPSTAT and strategic problem solving in American policing. Criminology and Public Policy, 2(3), 421456.

Whitfield, K. (2007). CompStat: A selected bibliography. The Police Chief, 74, 2. Worrall, JL, & Kovandzic, TV. (2010). Police levels and crime rates: An instrumental variables approach. Social Science Research, 39, 506516. doi:10.1016/j. ssresearch.2010.02.001.

Wu, T-H, Chen, M-S, & Yeh, J-Y. (2010). Measuring the performance of police forces in Taiwan using data envelopment analysis. Evaluation and Program Planning, 33(3), 254. doi:10.1016/j.evalprogplan.2009.09.001.

Submit your manuscript to a journal and benet from:

7 Convenient online submission7 Rigorous peer review7 Immediate publication on acceptance7 Open access: articles freely available online 7 High visibility within the eld7 Retaining the copyright to your article

Submit your next manuscript at 7 springeropen.com