Improving birth certificate data accuracy in Alabama

Abstract

Objective

Accurate vital statistics data are critical for monitoring population health and strategizing public health interventions. Previous analyses of statewide birth data have identified several factors that may reduce birth certificate accuracy including systematic errors and limited data review by clinicians. The aim of this initiative was to increase the proportion of hospitals in Alabama reporting accurate birth certificate data from 67% to 87% within 1 year.

Methods

The Alabama Perinatal Quality Collaborative led this statewide collaborative effort. Process measures included monthly monitoring of 11 variables across 5–10 patient birth certificates per month per hospital. Accuracy determination, defined as ≥95% accuracy of the variables analyzed, was performed by health care specialists at each hospital by comparing birth certificate variables from vital statistics with data obtained from original hospital source materials. Three months of retrospective, baseline accuracy data were collected before project initiation from which actionable drivers and change ideas were identified at individual hospitals. Data were analyzed using statistical process control measures.

Results

Thirty-one hospitals entered data throughout the course of the initiative, accounting for 850 chart analyses and 9350 variable assessments. The least accurately reported variables included birth weight, maternal hypertension, and antenatal corticosteroid exposure. At baseline, 67% of hospitals reported birth certificate accuracy rates ≥ 95%, which increased to 90% of hospitals within 2 months and was sustained for the remainder of the initiative.

Conclusion

Statewide, multidisciplinary quality improvement efforts increased birth certificate accuracy vital to public health surveillance.

INTRODUCTION

Vital statistics collected from birth certificates provide critical data for monitoring population health. As data are universally collected from all births, variables can help determine the quality of care at the local, state, and federal levels. In conjunction with aggregate reporting, these data can identify racial, ethnic, and socioeconomic health disparities¹ in addition to variables associated with poor health outcomes.² From these data and analyses, targeted quality improvement interventions can be developed. Before efforts by the National Center for Health Statistics to develop standards for data collection in 2003,³ multiple states reported inaccuracies between birth certificates and medical records.^4–6 Although using birth certificate data enables the analysis of larger data sets beyond the institutional level, inaccuracies may reduce the sensitivity of detecting specific outcomes such as prior preterm birth,⁶ a variable inaccurately reported on up to 35.5% of birth certificates.⁷

Birthing people and babies residing or born in Alabama face a number of health challenges including both infant and pregnancy-related morbidity and mortality. Since 1970, the infant mortality rate in the state has exceeded the national average. Moreover, the infant mortality rate has ranked as one of the 10 worst in the country since 2010.⁸ As prematurity and low birth weight substantially contribute to infant mortality in Alabama,⁹ it remains critical that variables that may reduce mortality (e.g., antenatal corticosteroids¹⁰), are accurately reported so as to develop meaningful interventions for mortality reduction. Maternal mortality in Alabama has also steadily increased since 2014 with up to 70% of pregnancy-associated and pregnancy-related cases determined to be preventable.¹¹ Similarly, accurate reporting of morbidities contributory to maternal mortality is critical to best inform strategic interventions.

The Alabama Perinatal Quality Collaborative (ALPQC) was formed in 2017 with the mission to promote optimal health for Alabama birthing people and babies by connecting key health and community stakeholders, sharing opportunities for education and training, and advancing the quality and safety of care through collaborative cooperation, evidence-based practices, and equitable approaches to care. Key stakeholders within the ALPQC chose birth certificate accuracy as the collaborative's first initiative. Prior birth certificate accuracy quality improvement initiatives have targeted an accuracy rate of 95% across variables recorded within the certificate. Our baseline monitoring period revealed that less than 70% of hospitals were reporting this level of accuracy. After identifying low accuracy variables and hospital-specific drivers for inaccurate reporting, hospitals implemented relevant change ideas with an initiative aim to increase the number of hospitals reporting accurate birth certificates by 20% over a 12-month period ending in January of 2020.

METHODS Initiative setting

This initiative was a statewide effort led by the ALPQC and made available to 46 delivery hospitals of which 31 hospitals participated. Our global aim was to collect timely, high-quality birth registry data for health surveillance and quality improvement. Included hospitals completed a data use agreement before participation. Project stakeholders included representation from the University of Alabama at Birmingham (Departments of Pediatrics, Obstetrics and Gynecology, and School of Public Health), the Alabama Hospital Association, the Alabama Department of Public Health, Alabama Center for Health Statistics, March of Dimes Alabama Chapter, and Alabama Medicaid. These representatives chose 11 key variables (Table 1) from the birth certificate from which accuracy was determined, a Key Driver Diagram was constructed (Figure 1), and improvement strategies were later implemented. As we anticipated that some key variables were already being accurately reported, we chose those with the lowest accuracy as process measures for the initiative. Guiding principles for variable selection included importance to population health surveillance, contributors to adverse outcomes in either infants or birthing people, and variables previously reported in the literature to be less accurate than other initiatives.

Table 1 Key variables for birth certificate accuracy determination

Key variables	Baseline accuracy (%)
Method of delivery	100
Maternal transfusion	99
NICU admission	97
Main source of payment for this delivery	97
Assisted ventilation >6 h	96
Obstetric estimate of gestational age at delivery	96
Previous preterm birth <37 weeks	95
Maternal diabetes	95
Maternal hypertension	92
Steroids for fetal lung maturation	86
Birth weight	82

Figure 1. Key Driver Diagram illustrating primary and secondary drivers for inaccurate birth certificate variables.

Interventions

Each enrolled hospital was required to have at least two team members consisting of a quality representative, clinical lead (either MD or RN), and birth registrar/birth certificate abstractor. Hospital teams were sent randomly selected birth certificates reported from their center to audit monthly. Hospitals with less than 500 annual births received five charts and hospitals with more than 500 annual births received 10 charts. Clinical representatives at each hospital familiar with each reported variable determined accuracy by comparing the variable on the reported certificates to original source materials (e.g., medical records). During data entry, representatives proposed drivers for variable inaccuracy (e.g., improper data transformation from customary to metric units) from which hospital-specific interventions could be implemented. Monthly run charts were provided to each participating hospital detailing changes in overall birth certificate accuracy and accuracy for each birth certificate variable for both their own facility's accuracy and initiative-wide accuracy.

At the initiative level, the first learning session held in November 2018 provided education regarding birth certificate accuracy assessments and REDCap data entry. Bimonthly webinars attended by enrolled hospitals were then conducted between January and October of 2019 during which data from the prior monitoring period were reviewed. These facilitated discussions highlighted low accuracy birth certificate variables and their relevant drivers, and change ideas were implemented at the hospital level in an iterative manner to continually improve strategies in context to reported accuracy measures. Additionally, site visits were offered to hospitals to review site procedures for data abstraction and data entry.

Birth certificate variables were stratified by accuracy following the baseline monitoring period. Initial interventions targeted those variables with the lowest accuracy including birth weight, antenatal corticosteroid exposure, and maternal hypertension. Birth weight inaccuracy most frequently resulted from improper data transformation from customary to metric units prompting modifications to the statewide Electronic Vital Events Registration System to standardize data entry. Interventions for inaccurate antenatal corticosteroid reporting included registrar education regarding commonly used corticosteroid nomenclature and hospital-specific approaches to more accurately abstract exposure from the electronic medical record. Regarding maternal hypertension and related diagnoses, systematic education targeted diagnoses with overlapping features or nomenclature such as pre-eclampsia and eclampsia.

Measurement strategy

To determine baseline birth certificate accuracy, 3 months of retrospective birth certificates previously entered at each hospital between July and September 2018 were provided to each hospital team by the Alabama Department of Public Health. By comparing certificates to source materials, accuracy for the 11 key variables selected (Table 1) was determined. Low accuracy variables served as process measures for the duration of the initiative and were monitored at the initiative and, for specific variables, at the state level. Each variable was defined as accurately reported if the variable from the birth certificate was consistent upon audit with the patient's medical record. For each specific variable, hospitals' monthly reporting was considered accurate if ≥90% of birth certificates recorded the variable accurately (e.g., a hospital accurately recorded maternal hypertension if ≥9 out of 10 or 5 out of 5 monthly certificates were accurate for that variable). P-charts were then used for statistical process control reporting the monthly proportion of hospitals accurately reporting each variable. A center's monthly overall birth certificate reporting was considered accurate if ≥95% of monthly variables were accurately reported (e.g., if among 10 charts each with 11 variable audits ≥104 out of 110 variables were accurately reported). The outcome measure for the initiative was defined as the proportion of hospitals reporting accurate birth certificates and was monitored using statistical process control via a p-chart. Control charts created at Cincinnati Children's were utilized for data analyses with standard control chart rules to identify special-cause variation.¹² Upon review of baseline data, a Pareto chart of lower accuracy variables was constructed to prioritize interventions impacting these variables.

RESULTS

Forty-three of 46 birthing hospitals in the state attended the initial learning session and 31 hospitals participated in the initiative. Collectively, 79% of annual births in Alabama occur at these hospitals. Three hundred seventy charts were assessed during the baseline period and 850 charts were assessed during the initiative reflecting ~2% of annual births in Alabama. From the key variables chosen, birth weight, antenatal corticosteroid exposure, and maternal hypertension accounted for 80% of inaccurate variable reporting as demonstrated in the Pareto chart in Figure 2. Maternal transfusion and delivery methods were accurately reported by >90% of hospitals throughout the initiative. The proportion of hospitals with accurate birth certificates increased from 67% to 90% after the initiative launch and remained at 90% for the duration of the initiative (Figure 3). The percentage of annual births with antenatal corticosteroid exposure in the state increased from 1.9% to 4.6% with special-cause variation occurring at the onset of the initiative (Figure S1A). The proportion of hospitals accurately reporting antenatal corticosteroid exposure increased from 60% to 86% (Figure S1B). Regarding other process measures, the proportion of hospitals reporting birth weight accurately increased from 70% to 94% (Figure S2B) and hypertension accuracy increased from 72% to 94% (Figure S2A). All identified special-cause variations resulted from eight or more points on one side of the centerline.

Figure 2. Pareto chart of the least accurate variables from the baseline reporting period.

Figure 3. P-chart of the percentage of participating hospitals entering accurate monthly birth certificates defined as ≥95% accuracy across the 11 selected variables from hospitals' 5–10 monthly chart assessments.

DISCUSSION

In this statewide initiative, the number of hospitals reporting accurate birth certificate data increased from 67% to 90% shortly after initiative launch. This initiative identified several birth certificate variables with lower accuracy, which may be critical in considering the sensitivity of reports utilizing these variables for conclusions regarding population health. These lower accuracy variables of birth weight, antenatal corticosteroids, and maternal hypertension were utilized as process measures—all of which increased in accuracy during the initiative.

Other perinatal quality collaboratives have reported similar successes in increasing birth certificate accuracy. However, they have only reported within their organization. The Illinois Perinatal Quality Collaborative launched an initiative targeting birth certificate accuracy¹³ which included 82 out of 93 delivery hospitals at the time of the initiative. Through monitoring the accuracy of 17 variables, birth certificate accuracy increased from 87% to 97%. Variables with lower accuracy during the baseline period included prenatal care, enrollment in the Women, Infant, and Children Program, and last menstrual period. The Florida Perinatal Quality Collaborative's birth certificate accuracy initiative included 17 hospitals, monitored 23 variables, and increased accuracy to 95%.¹⁴ In contrast to the data from the present study's baseline period, maternal hypertension and antenatal corticosteroid exposure were accurately reported in Illinois' initiative. This suggests that the accuracy of specific birth certificate variables likely varies by state. This variation highlights the need for baseline accuracy assessments before planning hospital- and state-specific interventions to improve birth certificate accuracy.

Other assessments of birth certificate reliability have reported lower accuracy with specific variables. In a study of birth certificate accuracy in Indiana, lower frequency events were less reliable. Additionally, complications during pregnancy and variables of maternal illness were reported with low accuracy.⁴ Analyses of birth certificate accuracy in other states including New York and Vermont have also reported a low rate of reliability for pregnancy complications including hypertension, previous preterm birth, and maternal transfusion.^15,16 Conversely, other variables were reported with higher accuracy including method of delivery and Medicaid coverage.¹⁶ In the present study, the only variable related to pregnancy complications with low accuracy was maternal hypertension. Similar to findings from other states, certain variables including the source of payment and delivery method were accurately reported.

As antenatal corticosteroid exposure reduces perinatal mortality and promotes fetal lung maturity,¹⁷ epidemiologic studies have used birth certificate data to both characterize variations in antenatal corticosteroid exposure¹⁸ as well as relate exposure rates to postnatal outcomes such as intraventricular hemorrhage.¹⁹ Such investigations have also concluded that antenatal corticosteroid exposure may result in a reduction in birth size for infants,²⁰ demonstrate the efficacy of antenatal corticosteroid exposure in reducing mortality,²¹ and to characterize racial disparities in antenatal corticosteroid exposure in preterm infants.¹ Given the frequency with which population level data are utilized to inform policy and quality improvement, it remains critical to continuously appraise data accuracy. In the present study, the state-level frequency of antenatal corticosteroid exposure increased following this quality improvement initiative.

Strengths of this quality improvement initiative include a multidisciplinary approach, inclusion of 31 hospitals across the state of Alabama, and detectable impact upon data at the state level. Limitations include the absence of standardization of some data accuracy definitions, primarily birth weight, which was the variable with the lowest reported accuracy. Additionally, while 15 hospitals did not participate in this initiative, nearly 80% of annual births in Alabama occur at hospitals that participated. Lastly, while improvement was noted during the reporting period, there was limited data supporting the sustainability of these efforts.

CONCLUSION

In summary, this statewide quality improvement initiative increased the number of hospitals in the state submitting accurate birth certificate data. Given both state- and hospital-level variation in variables impacting birth certificate data accuracy and the utility for this data to inform population health policy, systematic efforts to monitor data accuracy remain critical. Such efforts to ensure birth certificate accuracy allow identification and conceptualization of future improvement work at the state level so as to optimize health outcomes for infants and birthing people.

AUTHOR CONTRIBUTIONS

Samuel Gentle: conceptualization; data curation; formal analysis; methodology; project administration; supervision; writing – original draft; writing – review & editing. Matthew Moore: conceptualization; methodology; project administration; validation; writing – review & editing. Rosemary Blackmon: conceptualization; funding acquisition; investigation; project administration; writing – review & editing. Brenda Brugh: conceptualization; investigation; methodology; project administration; supervision; writing – review & editing. Allison Todd: conceptualization; formal analysis; investigation; methodology; project administration; writing – review & editing. Martha Wingate: data curation; funding acquisition; methodology; project administration; supervision; writing – review & editing. Sara Mazzoni: conceptualization; data curation; formal analysis; investigation; methodology; project administration; supervision; validation; writing – review & editing.

ACKNOWLEDGMENTS

The authors would like to acknowledge all the participating hospitals in this initiative for their participation in this quality improvement work.

CONFLICTS OF INTEREST

The authors declare no conflicts of interest.

TRANSPARENCY STATEMENT

This study and analysis plan was not formally registered. Materials and analytic code from this study are not available in a public archive.

ETHICS STATEMENT

IRB determined this initiative not human subject research. No individually identifiable information was used for birth certificate accuracy assessments.