Method overtness, forensic autopsy, and the evidentiary suicide note: A multilevel National Violent Death Reporting System analysis

Abstract

Objective

Higher prevalence of suicide notes could signify more conservatism in accounting and greater proneness to undercounting of suicide by method. We tested two hypotheses: (1) an evidentiary suicide note is more likely to accompany suicides by drug-intoxication and by other poisoning, as less violent and less forensically overt methods, than suicides by firearm and hanging/suffocation; and (2) performance of a forensic autopsy attenuates any observed association between overtness of method and the reported presence of a note.

Methods

This multilevel (individual/county), multivariable analysis employed a generalized linear mixed model (GLMM). Representing the 17 states participating in the United States National Violent Death Reporting System throughout 2011–2013, the study population comprised registered suicides, aged 15 years and older. Decedents totaled 32,151. The outcome measure was relative odds of an authenticated suicide note.

Results

An authenticated suicide note was documented in 31% of the suicide cases. Inspection of the full multivariable model showed a suicide note was more likely to manifest among drug intoxication (adjusted odds ratio [OR], 1.70; 95% CI, 1.56, 1.85) and other poisoning suicides (OR, 2.12; 1.85, 2.42) than firearm suicides, the referent. Respective excesses were larger when there was no autopsy or autopsy status was unknown (OR, 1.86; 95% CI, 1.61, 2.14) and (OR, 2.25; 95% CI, 1.86, 2.72) relative to the comparisons with a forensic autopsy (OR, 1.62, 95% CI, 1.45, 1.82 and OR, 2.01; 95% CI, 1.66, 2.43). Hanging/suffocation suicides did not differ from the firearm referent given an autopsy.

Conclusions

Suicide requires substantial affirmative evidence to establish manner of death, and affirmation of drug intoxication suicides appears to demand an especially high burden of proof. Findings and their implications argue for more stringent investigative standards, better training, and more resources to support comprehensive and accurate case ascertainment, as the foundation for developing evidence-based suicide prevention initiatives.

About the Authors:

Ian R. H. Rockett

Roles Conceptualization, Data curation, Formal analysis, Funding acquisition, Investigation, Methodology, Project administration, Resources, Supervision, Writing – original draft, Writing – review & editing

* E-mail: [email protected]

Affiliations Department of Epidemiology, West Virginia University, Morgantown, West Virginia, United States of America, Injury Control Research Center, West Virginia University, Morgantown, West Virginia, United States of America

ORCID http://orcid.org/0000-0002-4584-9337

Eric D. Caine

Roles Funding acquisition, Writing – original draft, Writing – review & editing

Affiliations Department of Psychiatry, University of Rochester Medical Center, Rochester, New York, United States of America, Injury Control Research Center for Suicide Prevention, University of Rochester Medical Center, Rochester, New York, United States of America

Steven Stack

Roles Writing – review & editing

Affiliations Department of Criminal Justice, Wayne State University, Detroit, Michigan, United States of America, Department of Psychiatry and Behavioral Neuroscience, Wayne State University, Detroit, Michigan, United States of America

Hilary S. Connery

Roles Writing – review & editing

Affiliations Division of Alcohol and Drug Abuse, McLean Hospital, Boston, Massachusetts, United States of America, Department of Psychiatry, Harvard Medical School, Boston, Massachusetts, United States of America

Kurt B. Nolte

Roles Writing – review & editing

Affiliation: Office of the Medical Investigator, University of New Mexico School of Medicine, Albuquerque, New Mexico, United States of America

Christa L. Lilly

Roles Writing – review & editing

Affiliation: Department of Biostatistics, West Virginia University, Morgantown, West Virginia, United States of America

Ted R. Miller

Roles Writing – review & editing

Affiliations Pacific Institute for Research and Evaluation, Calverton, Maryland, United States of America, Curtin University School of Public Health, Perth, Australia

Lewis S. Nelson

Roles Writing – review & editing

Affiliation: Department of Emergency Medicine, Rutgers New Jersey Medical School, Newark, New Jersey, United States of America

Sandra L. Putnam

Roles Writing – review & editing

Affiliation: Injury Control Research Center, West Virginia University, Morgantown, West Virginia, United States of America

Paul S. Nestadt

Roles Writing – review & editing

Affiliations Department of Mental Health, Johns Hopkins Bloomberg School of Public Health, Baltimore, Maryland, United States of America, Department of Psychiatry and Behavioral Sciences, Johns Hopkins University School of Medicine, Baltimore, Maryland, United States of America

ORCID http://orcid.org/0000-0002-2479-703X

Haomiao Jia

Roles Data curation, Formal analysis, Investigation, Methodology, Resources, Software, Validation, Writing – original draft, Writing – review & editing

Affiliations School of Nursing, Columbia University, New York, New York, United States of America, Department of Biostatistics, Mailman School of Public Health, Columbia University, New York, New York, United States of America

Introduction

Emerging as the leading cause of injury mortality in the United States (US) by 2009 [1], suicide is not a default manner-of-death determination for medical examiners and coroners [2–4]. A socially condemned and stigmatized phenomenon, suicides are undercounted [5], with false negativity posing a far greater threat than false positivity to valid certification [6,7]. National and international research indicates that undercounting is nonrandom across suicide methods [8,9]. The three leading methods of suicide in the US are firearm, hanging, and poisoning. Collectively, these methods accounted for 92% of total suicides in 2015 [10]. Suicides by poisoning are likely far more challenging for medical examiners and coroners to ascertain than those by shooting and hanging [4,11–14]. Gross discrepancies among these major methods in accounting for suicide would inhibit, or even preclude, accurate risk group delineation and risk factor identification, and hence impede the design, targeting, and implementation of effective clinical and population interventions.

Availability of a suicide note or an equivalent—whether written, typed, digital, or audio [15]—can serve as a pivotal piece of external forensic and psychological evidence for determining suicide as the manner of death [16]. Some countries even require a suicide note to record a death as suicide [17], as was true of some US coroners in the 1970s [18]. Thus, lack of an evidentiary note may induce suicide misclassification in the US [19]. Echoing its pivotal nature, a US study found that a suicide manner of death was associated with a far higher prevalence of notes than was undetermined intent, 32% versus 1.5% [20]. Two English studies also revealed large prevalence gaps [21,22]. Undetermined is the manner of death category most susceptible to obscuring suicides [21,23–25], in relative terms, with poisoning its predominant underlying cause-of-death sub-category in the US [14,26].

Two earlier American studies found an excess of note-leaving among suicides by poisoning relative to other methods [27,28], whereas a later one found an excess among hanging suicides [29]. A subsequent individual-level, multivariable analysis of National Violent Death Reporting System (NVDRS) data, for the period 2003–2006, showed excess note-leaving among suicides by poisoning, firearm, and hanging versus all other methods except jumping from a height and drowning, two other less forensically overt methods [30]. That study also showed note leaving was nonrandom across demographic characteristics such as age, sex, race/ethnicity, marital status, and urban residence. Finally, an Austrian study found no difference in the prevalence of note-leaving across methods of suicide or age, sex, family status, psychiatric care or motive [31]. Unlike the US, Austria is among rare countries whose suicide certification appears very accurate [32,33].

In this multilevel, multivariable study of NVDRS data for 2011–2013, we posited that a higher proportion of suicide notes in manner of death determinations involving fatal drug introxication, as compared to those involving a firearm or hanging, reflected their use as key evidence in the differential process of establishing suicide as the manner of death. This could signify stricter (i.e., more conservative) evidentiary standards when making these determinations, leading in turn to a greater potential for undercounting suicides.

We addressed two questions in our examination of the differential determination of suicide, with a view to informing and improving surveillance, etiologic understanding, and prevention of suicide and related injury mortality. Was an evidentiary note more common among suicide cases involving drug intoxication and other poisoning, less self-evident causes and manner of death, as compared to the violent methods of firearm and hanging, where apparent motivation is more overt? Did the performance of a forensic autopsy serve to mediate the association between the reporting of a note and the overtness of the suicide method? Although a forensic autopsy is not a personal window into decedent intent [34], together with toxicology it generates evidence that helps medical examiners and coroners identify injury mechanisms (i.e., causes of death) and decedent intent in suspicious or uncertain cases [35–37].

Salient to our study, forensic autopsy and toxicological testing rates vary greatly across states [38,39]. Since suicides are local events, and often socially stigmatized [40,41], we adjusted our analyses for both county-level and individual-level factors. They comprised characteristics of the medicolegal death investigations and the state-county investigation systems, decedent and areal demographics, and documented mental health antecedents and other death circumstances.

Data and methods

Individual-level data source and variables

The source for our individual-level variables was the Restricted Access Database from the NVDRS, which is administered by the Centers for Disease Control, and Prevention (CDC). This state, territory, and incident-based surveillance system employs public health informatics for making data linkages, primarily among death certificates, law enforcement records, and medical examiner and coroner records. Also variably incorporating such optional supplements as laboratory reports and hospital records, the NVDRS provides de-identified information about suicide and other violent deaths, including their geographic location, circumstances, and personal sociodemographics, in addition to investigation specifics [42–44]. Disaggregatable to county of death, the data analyzed in this study represented the 17 states that participated in the NVDRS throughout the 2011–2013 observation period. They were Alaska, Colorado, Georgia, Kentucky, Maryland, Massachusetts, New Jersey, New Mexico, North Carolina, Ohio, Oklahoma, Oregon, Rhode Island, South Carolina, Utah, Virginia, and Wisconsin. Enhancing study generalizability, these states mirrored the nation in their age and sex composition, manner-of-death distribution, and crude and age-adjusted all-cause, suicide, and undetermined intent death rates [20]. They overrepresented non-Hispanic Blacks and Whites and underrepresented Hispanics.

Our study population comprised registered suicides (ICD-10: U03, X60-X84, Y87.0), whose method and state and county of death were specified, and was further confined to decedents aged 15 years and older, since fewer than 1% of known suicides were younger. The number of decedents in our multivariable analyses totaled 32,151. As further background, our first table provides comparative data on suicide methods for the 17 NVDRS states and the US. These comparisons were based on separate counts for the selected demographic variables, not study population and corresponding national population counts. Similarity between the states and the nation would fortify the generalizability of the results. The data source was CDC WISQARS™ (Web-based Injury Statistics Query and Reporting System) [10].

The outcome variable in this study was suicide note (yes versus no or unknown). Our predictor distinguished suicide methods as follows: drug intoxication, other poisoning, jumping and drowning (combined owing to small sample size and similar corroborative challenges), hanging/suffocation, firearm (referent), and all other methods specified in the NVDRS suicide cases. Additional individual-level covariates were prior suicide attempt; primary mental diagnosis; current mental health treatment; blood alcohol concentration; number of other specified drug positives; physical health problem; number of intimate partner or legal or job or financial or school problems; crisis in past two weeks; emergency medical services at scene; region of death; age; sex; race/ethnicity; marital status; education; and military veteran status. The mediator was autopsy status (yes versus no or unknown).

County-level data sources and covariates

Linked to the individual-level data in this study, our county-level death investigation system covariates were mode of selection of the chief medical examiner or coroner for a given system (elected versus appointed) and accreditation status and type (accredited coroner, accredited medical examiner, unaccredited coroner, unaccredited medical examiner). Selection mode was identified through a CDC website [45], and accreditation status and system type through the respective websites of the relevant accrediting agencies, the National Association of Medical Examiners (NAME) [46] and the International Association of Coroners and Medical Examiners (IACME) [47]. We resolved outstanding questions by email or telephone communication with state, district, or county offices. County-level demographic covariates were urbanicity (5 categories representing the 12 ordinal categories of the 2013 Urban Influence Codes: large metropolitan/small metropolitan/metropolitan adjacent/micropolitan or adjacent/rural) and percent population below poverty as the measure of the local poverty burden. These two covariates served as proxies for external forces that potentially could inhibit or support medicolegal death investigations [48]. Their source was the County Area Health Resource File for 2014–2015 [49].

Hypotheses and statistical approach

We used a generalized linear mixed model (GLMM) to test the following two hypotheses: (1) an evidentiary suicide note is more likely to accompany suicides by drug-intoxication and by other poisoning than suicides by firearm and hanging/suffocation; and (2) performance of a forensic autopsy attenuates the observed association between overtness of method and the reported presence of a note. In testing the first hypothesis, we progressively applied four models, beginning with a univariable analysis, confined to suicide method, followed by multivariable analyses that cumulatively incorporated medicolegal system and investigation characteristics, mental health antecedents and precipitating circumstances, and decedent and areal demographics. Providing for a fifth multivariable model, we then tested our second hypothesis. The GLMM is a two-level model that is logistic at the individual level and linear at the county level. We also included a state-level random effect to incorporate the data structure of counties nested in a state and individuals nested in a county. The statistical software was SAS (Copyright(c) 2002–2010, SAS Institute Inc., Cary, NC).

Results

Table 1 describes the frequency of suicide by method across age, sex, and race/ethnicity for the NVDRS states and nationally during the observation period. Firearm, hanging, and drug intoxication were the leading methods of suicide. Generally, the states and nation showed very similar percentage distributions across age and sex, except at ages 14 years and under. The small numbers of the latter induced data instability; this age group was eliminated from our study population.

[Figure omitted. See PDF.]

Table 1. Percent suicide by method and selected demographic characteristics: 17 National Violent Death Reporting System States and the United States, 2011–2013a.

https://doi.org/10.1371/journal.pone.0197805.t001

Bivariable frequency data on suicide notes for the study population are presented in Table 2. An authenticated suicide note was documented in 31% of the suicide cases. Consistent with our principal hypothesis, an excess prevalence of notes manifested for suicides by drug intoxication and by other poisoning, at 42% and 48%, versus 29% and 31% for firearm and hanging/suffocation suicides, respectively. Pertinent to statistical adjustments in the multivariable analyses, note prevalence reflected some marked variability by selection mode of the chief medical examiner/coroner, blood alcohol concentration, region of death, and within the sociodemographic characteristics of sex, race/ethnicity, marital status, and education.

[Figure omitted. See PDF.]

Table 2. Prevalence of suicide notes by decedent demographics, death circumstances, and investigation system characteristics, 17 National Violent Death Reporting System States, 2011–2013.

https://doi.org/10.1371/journal.pone.0197805.t002

Partially affirming the principal hypothesis, the univariable analysis showed drug intoxication suicides and suicides by other poisoning were respectively 71% and 122% more likely than their referent, firearm suicides, to be associated with a note (Table 3). Neither hanging suicides, as consistent with our hypothesis, nor combined jumping and drowning suicides deviated from the referent. Findings remained robust as grouped death investigation variables (Model 2), precipitating circumstances (Model 3), and decedent and county-level demographics (Model 4) were cumulatively incorporated into the multivariable analyses. For drug intoxication suicides, percent increase in odds of a note relative to the firearm referent rose from 71% in the univariable analysis to 80% and 88%, respectively, in the multivariable analysis with application of Models 2 and 3, and diminution to 66% following addition of the demographics in Model 4. Corresponding changes were more modest for the other poisoning suicide category. However, under Models 1 and 4, percent increase in odds of a note relative to firearms was larger for other poisoning suicides than for drug intoxication suicides at 122% and 113%, respectively.

[Figure omitted. See PDF.]

Table 3. Odds ratios and 95% confidence intervals for an evidentiary note by suicide method, cumulatively adjusted for investigation characteristics, death circumstances, and demographics, 17 National Violent Death Reporting System States, 2011–2013.

https://doi.org/10.1371/journal.pone.0197805.t003

The stratified analysis supported our second hypothesis as performance of a forensic autopsy attenuated the association between suicide method and suicide note outcome (Table 4). Percent increase in odds of an evidentiary note for drug intoxication suicides, relative to firearm suicides, was 86% with no autopsy or unknown autopsy status versus 62% with autopsy. Corresponding figures for other poisoning suicides were 125% and 101%. With no autopsy or unknown autopsy status, hanging/suffocation suicides were less likely than firearm referents to be accompanied by a suicide note. Expanding Model 4 to include autopsy status showed increased percent odds of a note for drug intoxication suicides and other poisoning suicides of 70% and 112%, respectively, and a diminished effect for the residual group, all other suicide methods, compared to the firearm referent. Further inspection of the full model revealed that cases with no autopsy or with unknown autopsy status had 26% higher odds than autopsied cases of having a suicide note on record.

[Figure omitted. See PDF.]

Table 4. Odds ratios and 95% confidence intervals for an evidentiary note by suicide method, stratified by autopsy status and unstratified, adjusted for investigation characteristics, death circumstances, and demographics, 17 National Violent Death Reporting System States, 2011–2013.

https://doi.org/10.1371/journal.pone.0197805.t004

Discussion

Our findings caution users against uncritically taking suicide and other official data at face value. Under-ascertainment means suicide undercounting. Indicative of differential undercounting by method, increased odds of a suicide note manifested both for cases of suicides by drug intoxication and by other poisoning relative to their referent, firearm cases, as consonant with the first hypothesis concerning the relative violence and forensic overtness of methods. Hanging suicides did not deviate from the referent, nor did the combined category of jumping and drowning suicides. The observed note excess for both drug-intoxication and other poisoning suicides persisted through application of each of the multivariable models, including the full multilevel, multivariable model that incorporated autopsy status.

The study implication of growing suicide misclassification and suicide undercounting among drug intoxication deaths assumes added importance as an impediment to prevention in the face of the burgeoning opioid mortality epidemic [50] and severely under-resourced and overburdened emergency healthcare [51,52] and death investigation systems [53,54]. Although plausibly a gross underestimate, at 1.62 per 100,000 population, the drug intoxication suicide rate was 41% higher in 2015 than in 2000 [10]. However, suicides involving such poisons as gases, vapors, metals, pesticides, and household cleaners have remained relatively less common. At 0.50 per 100,000, the rate of non-drug poisoning suicides in 2015 was 12% lower than the rate in 2000. Moreover, determination of intent in the non-drug poisoning cases often rests on the unmistakable actions required to use these suicide methods (e.g., carbon monoxide poisoning where the decedent was found in a car with a hose from the tailpipe to the passenger compartment, windows sealed with duct tape, ignition key in the on position, and the gas tank empty), which diminishes the potential ambiguity they pose for medical examiners and coroners. On the other hand, the literature provides no evidence drug intoxication suicides are more planned and less impulsive than suicides by more overtly violent methods.

As hypothesized, performance of a forensic autopsy attenuated the observed associations between overtness of suicide method and odds of an evidentiary suicide note. Although an autopsy can help investigators identify the mechanism or cause of an injury death, it does not invariably enlighten decedent intentionality [55]. Indeed, performance of an autopsy diminished but did eliminate the note-effect we found when comparing mechanisms of deaths in the manner of death determination. Presence of a suicide note, similar to the effect of an overt suicide method, may even reduce autopsy occurrence [56]. During 2007, 97% of US homicides were autopsied compared with 60% of suicides, 81% of undetermined intent deaths, and 79% of unintentional (accidental) poisoning deaths, pointing to a less rigorous approach to suicide versus homicide determination and accounting [57].

The US National Association of Medical Examiners (NAME) Forensic Autopsy Performance Standards require an autopsy when “the death is by apparent intoxication by alcohol, drugs, or poison, unless a significant interval has passed, and the medical findings and absence of trauma are well documented” [58]. NAME does not require autopsy in other suicidal deaths where the cause is externally manifest (e.g., gunshot wound or hanging) and instead leaves autopsy performance to the discretion of the pathologist, if it is “necessary to determine cause or manner of death, or document injuries/disease, or collect evidence.” While this standard is expected for medical examiners, who work in some states, it does not apply to elected coroners–the norm for many states. Other jurisdictions have both coroners and medical examiners, varying by counties [39].

By comparison with the US, Finnish medicolegal authorities conducted forensic autopsies in 99% of suicides, 98% of homicides, 98% of unintentional poisoning deaths, and 97% of undetermined deaths in the period 2000–2003 [59]. Researchers viewed a lower autopsy rate and greater use of ill-defined and unknown cause-of-death codes as especially problematic for the quality of Danish suicide and other manner-of-death statistics relative to those of Finland [60,61]. During the period 1998–2007, Finland and Denmark had annualized suicide rates of 20 and 12 per 100,000 population, respectively [62]. Corresponding combined clinical and forensic autopsy rates for total deaths were 31% and 8%. This gap—as a plausible indicator of different standards of thoroughness in medicolegal death investigations [63]—may mean there is an artifactual component in the differential in suicide rates between these two Scandinavian countries. However, substantive factors contributing to a higher rate in Finland might include variable alcohol consumption and firearm ownership between Finland and Denmark.

More generally implicating an artifactual component in suicide rates and rate variation, an ecological or correlational study of 35 Eurasian countries reported spatial and temporal associations between the respective magnitudes of the combined clinical and forensic autopsy rate and the suicide rate [62]. Spatially or cross-sectionally, a 1% difference in autopsy rates was associated with a suicide rate difference of 0.49 per 100,000 population, and temporally or longitudinally a 1% decrease in the autopsy rate with a suicide rate decrease of 0.42 per 100,000.

Our research, examining registered or known suicides, complements another multilevel (individual/county), multivariable study which used NVDRS data from the same observation period to predict differential odds that suicides pooled with deaths of undetermined intent, included as possible suicides, would be classified by medical examiners and coroners as suicide if there was documentation of a suicide note and mental health antecedents [20]. One hypothesis in the complementary study addressed the association between an evidentiary suicide note and suicide classification. Underscoring a pivotal role an authenticated note can play in separating suicide from undetermined cases, presence was associated with 34-fold increased odds of a suicide classification. In addition, combined firearm and hanging/suffocation deaths showed 42-fold increased odds of a suicide classification relative to drug intoxication deaths or, alternatively expressed, 98% lower odds of an undetermined classification. Drug intoxication cases with an evidentiary note were 45 times more likely to be classified as suicide compared to corresponding cases with no note or unknown note status, and eight times more likely in firearm and hanging cases. A relative strength of the current multilevel study, since it focused on suicides rather than suicides and undetermined deaths, was greater granularity and specificity of suicide methods or injury causes/mechanisms than in the comparative NVDRS study.

Based on medicolegal and police investigations, which often involve family and friends of the decedent, NVDRS data are vulnerable to reporter bias, especially among persons who are ashamed or embarrassed by what transpired. This deficiency also may reflect a lack of willingness of investigators to collect data about decedent background and circumstances when suicide is readily apparent as the manner of death, or when as an elected local official, coroners are reluctant to probe potentially sensitive personal issues. Our study and the complementary NVDRS investigation jointly reveal a need for qualitative as well as quantitative research on whether family and friends of the decedents variably destroy or otherwise conceal suicide notes, and withhold other potential corroborative evidence from authorities. The field now would benefit from mixed methods investigations that integrate psychological autopsies, focus groups, surveys, sociocultural autopsies, content analysis, and thematic analysis to examine values and attitudes towards suicide that may continue to influence its reporting. Although high quality data regarding suicides are essential for planning, implementing, and evaluating suicide prevention programs, few resources have been devoted to improving fundamental data quality. Without such rigor, it will be difficult to accept the validity of future efforts to reduce suicide rates.

The enriched Restricted Access Database of the NVDRS provides the only population-based data in the US appropriate for evaluating our research questions about the relationship between suicide methods, a forensic autopsy, and an evidentiary suicide note. Besides reporter bias, a limitation of this study was restriction of the geographic domain to the 17 states that contributed data to the NVDRS throughout the observation period, 2011–2013. Nevertheless, high demographic concordance between these states and the nation [20] tempers our concern about reduced generalizability of study findings, which additionally is a by-product of system protocols that emphasize uniform definitions of manner of death and consistent data collection, entry, review, and coding [44]. Moreover, the similarity in the distribution of suicides by method across age, sex, and race/ethnicity, between the NVDRS states and the nation, enhanced our confidence in study generalizability. Since our observation period, the NVDRS has expanded to 40 states, the District of Columbia, and Puerto Rico, with a goal to cover all 50 states and the territories [64]. Further study limitations included the indirect nature of our assessment of differential suicide data quality by method, confinement of our study population to suicides whose death circumstances were captured by the NVDRS and whose methods were specified, and our inability to factor in medicolegal use of a computerized tomography scan in lieu of a forensic autopsy. Other limitations and strengths of the NVDRS have been reported [20,44,65–67].

Conclusions

Suicide requires substantial affirmative evidence to establish manner of death, and affirmation of drug intoxication suicides appears to demand an especially high burden of proof. Findings and their implications argue for more stringent investigative standards, better training, and more resources to support accurate and comprehensive case ascertainment, as the foundation for developing evidence-based suicide prevention initiatives.

Citation: Rockett IRH, Caine ED, Stack S, Connery HS, Nolte KB, Lilly CL, et al. (2018) Method overtness, forensic autopsy, and the evidentiary suicide note: A multilevel National Violent Death Reporting System analysis. PLoS ONE 13(5): e0197805. https://doi.org/10.1371/journal.pone.0197805

References

1. Rockett IRH, Regier MD, Kapusta ND, Coben JH, Miller TR, Hanzlick RL, et al. Leading causes of unintentional and intentional injury mortality: United States, 2000–2009. Am J Public Health 2012; 102(11): e84–92.

2. Fisher DJ. Equivocal cases of teenage suicide. Forensic Examiner 2000; 9(9–10): 24–27.

3. Timmermans S. Suicide determination and the professional authority of medical examiners. Am Sociol Rev 2005; 70(2): 311–333.

4. Stone DM, Holland KM, Bartholow B, Logan JE, LiKamWa McIntosh W, Trudeau A, et al. Deciphering suicide and other manners of death associated with drug intoxication: a Centers for Disease Control and Prevention consultation meeting summary. Am J Public Health 2017; Aug; 107(8): 1233–1239. pmid:28640689

5. Goldsmith SK, Pellmar TC, Kleinman AM, Bunney WEE. Reducing suicide: a national imperative. Washington, DC: National Academies Press, 2002.

6. O'Carroll PWA. A consideration of the validity and reliability of suicide mortality data. Suicide Life Threat Behav 1989; 19: 1–16.

7. Rockett IRH. Counting suicides and making suicide count as a public health problem. Crisis 2010; 31(5): 227–230. http://www.iasp.info/pdf/journal/2010_rockett_counting_suicides.pdf pmid:21134841

8. Warshauer ME, Monk M. Problems in suicide statistics for whites and blacks. Am J Public Health 1978; 68: 383–388. pmid:645985

9. Platt S, Backett S, Kreitman N. Social construction or causal ascription: distinguishing suicide from undetermined deaths. Soc Psychiatry and Psychiatr Epidemiol 1988; 23: 217–221.

10. Centers for Disease Control and Prevention. Web-based Injury Statistics Query and Reporting System, 2017. http://www.cdc.gov/ncipc/wisqars/

11. Ovenstone IMK. A psychiatric approach to the diagnosis of suicide and its effect upon the Edinburgh statistics. Br J Psychiatry 1973; 123: 15–21. pmid:4729869

12. Breiding MJ, Wiersema B. Variability of undetermined manner of death classification in the US. Inj Prev 2006; 12 Suppl 2: ii49–ii54.

13. Donaldson AE, Larsen GY, Fullerton-Gleason L, Olson LM. Classifying undetermined poisoning deaths. Inj Prev 2006; 12: 338–343. pmid:17018678

14. Rockett IRH, Wang S, Stack S, De Leo D, Frost JL, Ducatman AM, et al. Race/ethnicity and potential suicide misclassification: window on a minority suicide paradox? BMC Psychiatry 2010; 10:35. http://www.biomedcentral.com/content/pdf/1471-244x-10-35.pdf pmid:20482844

15. Byard RW, Heath KJ. Suicide notes, age and the impact of suicide guides. Scand J Forensic Sci 2007; 13: 10–11.

16. Parai JL, Krieger N, Tomlinson G, Adlaf EM. The validity of the certification of manner of death by Ontario coroners. Ann Epidemiol 2006; 16(11): 805–811. pmid:16621598

17. Stone DH, Chishti P, Roulston C. Final report of the European review of suicide and violence epidemiology (EUROSAVE) Project, 2002. http://ec.europa.eu/health/ph_projects/1999/injury/fp_injury_1999_frep_10_en.pdf

18. Nelson FL, Farberow NL, MacKinnon DR. The certification of suicide in eleven western states: an inquiry into the validity of reported suicide rates. Suicide Life Threat Behav 1978; 8(2): 75–88. pmid:694978

19. Centers for Disease Control and Prevention (CDC). Homicides and suicides—National Violent Death Reporting System, United States, 2003–2004. MMWR Morb Mortal Wkly Rep 2006; 55(26): 721–4. pmid:16826158

20. Rockett IRH, Caine ED, Connery HS, D’Onofrio G, Gunnell DJ, Miller TR, et al. Discerning suicide in drug intoxication deaths: paucity and primacy of suicide notes and psychiatric history. PLOS One 2018; Jan 10;13(1):e0190200. http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0190200 pmid:29320540

21. Linsley KR, Schapira K, Kelly TP. Open verdict v. suicide: importance to research. Br J Psychiatry 2001; 178: 465–468. pmid:11331564

22. Palmer BS, Bennewith O, Simkin S, Cooper J, Hawton K, Kapur N, et al. Factors influencing coroners' verdicts: an analysis of verdicts given in 12 coroners' districts to researcher-defined suicides in England in 2005. J Public Health (Oxf) 2015; 37(1): 157–165.

23. Cooper PN, Milroy CM. The coroner’s system and underreporting of suicide. Med Sci Law 1995; 35: 319–326. pmid:7500856

24. Jougla E, Pequignot F, Chappert J, Rossollin F, Le Toullec A, Pavillon G, et al. Sante Publique 2002; 50: 49–52 [in French].

25. Ohberg A, Lonnqvist J. Suicides hidden among undetermined deaths. Acta Psychiatr Scand 1998; 98(3): 214–218. pmid:9761408

26. Huguet N, Kaplan MS, McFarland BH. Rates and correlates of undetermined deaths among African Americans: results from the National Violent Death Reporting System. Suicide Life Threat Behav 2012; 42(2): 185–96. pmid:22486604

27. Cohen SL, Feidler JE. Content analysis of multiple messages in suicide notes. Suicide Life Threat Behav 1974; 4: 75–95.

28. Tuckman J, Kleiner RJ, Lavell M. Emotional content of suicide notes. Am J Psychiatry 1959; 116: 59–63. pmid:13661450

29. Callanan VJ, Davis MS. A comparison of suicide note writers with suicides who did not leave notes. Suicide Life Threat Behav 2009; 39: 558–568. pmid:19929155

30. Stack S, Rockett IRH. Are suicide note writers representative of all suicides? Analysis of the National Violent Death Reporting System. Suicide Life Threat Behav 2018; 48(1): 12–20. pmid:27982463

31. Eisenwort B, Berzlanovich A, Willinger U, Eisenwort G, Lindorfer S, Sonneck G, et al. Suicide notes and their importance to suicide research. The representativeness of suicide note writers. Nervenarzt 2006; 77(11): 1355–1356, 1358–1360,1362 [in German]. pmid:16028079

32. Rockett IRH, Thomas BM. Reliability and sensitivity of suicide certification in higher-income countries. Suicide Life Threat Behav 1999; 29: 141–149. pmid:10407967

33. Värnik P, Sisask M, Värnik A, Arensman E, Van Audenhove C, van der Feltz-Cornelis CM, et al. Validity of suicide statistics in Europe in relation to undetermined deaths: developing the 2–20 benchmark. Inj Prev 2012; 18(5): 321–325. pmid:22157205

34. Acinas MP, Robles JI, Peláez-Fernández MÁ. Suicide note and the psychological autopsy: associated behavioral aspects. Actas Esp Psiquiatr 2015; 43(3): 69–79. pmid:25999154

35. Croft PR, Lathrop SL, Zumwalt RE. Amended cause and manner of death certification: a six-year review of the New Mexico experience. J Forensic Sci 2006; 51(3): 651–656. pmid:16696715

36. Shields LBE, Hunsaker DM, Hunsaker JC, Ward MK. Toxicologic findings in suicide. A 10-year retrospective review of Kentucky medical examiner cases. Am J Forensic Med Pathol 2006; 27(2): 106–112. pmid:16738426

37. Hoyert DL, Kung HC, Xu J. Autopsy patterns in 2003. Vital Health Stat 2007; 20(32): 1–27.

38. Karch D, Crosby A, Simon T. Toxicology testing and results for suicide victims—13 states, 2004. MMWR Morb Mortal Wkly Rep 2006; 55(46): 1245–1248. pmid:17119531

39. Hanzlick R. A perspective on medicolegal death investigation in the United States: 2013. Acad Forensic Pathol 2014; 4(1): 2–9.

40. Scourfield J, Fincham B, Langer S, Shiner M. Sociological autopsy: an integrated approach to the study of suicide in men. Soc Sci Med 2012; 74(4): 466–473. pmid:20646811

41. Neuilly MA. Putting the public back in public health: an argument for the articulation of fatality reviews and coroners’ inquests. Homicide Stud 2013; 17(4): 339–352.

42. Paulozzi LJ, Mercy J, Frazier L Jr, Annest JL; Centers for Disease Control and Prevention. CDC's National Violent Death Reporting System: background and methodology. Inj Prev 2004; 10(1): 47–52. pmid:14760027

43. Steenkamp M, Frazier L, Lipskiy N, Deberry M, Thomas S, Barker L, et al. The National Violent Death Reporting System: an exciting new tool for public health surveillance. Inj Prev 2006; 12 Suppl 2: ii3–ii5.

44. Blair JM, Fowler KA, Jack SP, Crosby AE. The National Violent Death Reporting System: overview and future directions. MMWR Surveill Summ 2016; 65(10): 1–42. pmid:27537325

45. Centers for Disease Control and Prevention, 2015. Public Health Law Program. Coroner/Medical Examiner Laws by State. https://www.cdc.gov/phlp/publications/topic/coroner.html

46. National Association of Medical Examiners, 2017. https://netforum.avectra.com/eweb/DynamicPage.aspx?Site=NAME&WebCode=OrgSearch

47. International Association of Coroners & Medical Examiners, 2017. http://www.theiacme.com/accreditation.

48. Waldhoer T, Berzlanovich A, Vutuc C, Haidinger G. Rates of postmortem examination in Austria: the effect of distance between location of death and site of examination. J Clin Epidemiol 2003; 56(9): 891–895. pmid:14505775

49. Health Resources and Services Administration, Bureau of Health Workforce, National Center for Health Workforce Analysis. The County Area Health Resources File (AHRF) 2014–2015 release. Rockville: US Department of Health and Human Services, Health Resources and Services Administration; 2015.

50. Hedegaard H, Warner M, Minino AM. Drug overdose deaths in the United States, 1999–2015. NCHS Data Brief 2017; 273: 1–8.

51. Institute of Medicine. Emergency medical services at the crossroads. Washington, DC: The National Academies Press; 2006.

52. Institute of Medicine. Hospital-based emergency care: at the breaking point. Washington, DC: The National Academies Press; 2006.

53. Institute of Medicine; Board on Health Promotion and Disease Prevention; Committee for the Workshop on the Medicolegal Death Investigation System. Medicolegal death investigation system: workshop summary. Washington, DC: The National Academies Press; 2003.

54. Committee on Identifying the Needs of the Forensic Science Community; Committee on Science, Technology, and Law Policy and Global Affairs; Committee on Applied and Theoretical Statistics; Division on Engineering and Physical Sciences. Strengthening forensic science in the United States: a path forward. Washington, DC: The National Academies Press; 2009.

55. Sorenson SB, Shen H, Kraus JF. Undetermined manner of death. A comparison with unintentional injury, suicide, and homicide death. Eval Rev 1997; 21(1): 43–57. pmid:10183268

56. Carpenter BJ, Tait G, Barnes M, Adkins G, Naylor C, Begum N. Increasing the information available to coroners: the effect on autopsy decision making. Med Sci Law 2009; 49(2): 101–109. pmid:19537447

57. Hoyert DL. The changing profile of autopsied deaths in the United States, 1972–2007. NCHS Data Brief 2011; (67): 1–8. http://www.cdc.gov/nchs/data/databriefs/db67.pdf

58. National Association of Medical Examiners. Forensic Autopsy Performance Standards, September 12, 2016. https://netforum.avectra.com/public/temp/ClientImages/NAME/684b2442-ae68-4e64-9ecc-015f8d0f849e.pdf

59. Lunetta P, Lounamaa A, Sihvonen S. Surveillance of injury-related deaths: medicolegal autopsy rates and trends in Finland. Inj Prev 2007; 13(4): 282–284. pmid:17686941

60. Ylijoki-Sørensen S, Boldsen JL, Boel LW, Bøggild H, Lalu K, Sajantila A. Autopsy rate in suicide is low among elderly in Denmark compared with Finland. Forensic Sci Int 2014; 244: 158–165. pmid:25244292

61. Ylijoki-Sørensen S, Sajantila A, Lalu K, Bøggild H, Boldsen JL, Boel LW. Coding ill-defined and unknown cause of death is 13 times more frequent in Denmark than in Finland. Forensic Sci Int 2014; 244: 289–294. pmid:25300069

62. Kapusta ND, Tran US, Rockett IRH, De Leo D, Naylor CP, Niederkrotenthaler T, et al. Declining autopsy rates and suicide misclassification: a cross-national analysis of 35 countries. Arch Gen Psychiatry 2011; 68(10): 1050–1057. http://archpsyc.jamanetwork.com/article.aspx?articleid=1107298 pmid:21646567

63. Shojania KG, Burton EC, McDonald KM, Goldman L. Changes in rates of autopsy-detected diagnostic errors over time: a systematic review. JAMA 2003; 289(21): 2849–2856. pmid:12783916

64. Centers for Disease Control and Prevention. National Center for Injury Prevention and Control. National Violent Death Reporting System. www.cdc.gov/violenceprevention/nvdrs/.

65. Logan JR, Karch DL, Crosby AE. Reducing “unknown” data in violent death surveillance: study of death certificates, coroner/medical examiner and police reports from the National Violent Death Reporting System, 2003–2004. Homicide Stud 2009; 13(4): 385–397.

66. Parks SE, Johnson LL, McDaniel DD, Gladden M; Centers for Disease Control and Prevention (CDC). Surveillance for violent deaths—National Violent Death Reporting System, 16 states, 2010. MMWR Surveill Summ 2014; 63(1): 1–33. pmid:24430165

67. Kaplan MS, Caetano R, Giesbrecht N, Huguet N, Kerr WC, McFarland BH, et al. The National Violent Death Reporting System: Use of the Restricted Access Database and recommendations for the system’s improvement. Am J Prev Med 2017; 52(4): 130–133.