Uncertainty in Health Impact Assessments of Smoke From a Wildfire Event

Abstract

Wildfires cause elevated air pollution that can be detrimental to human health. However, health impact assessments associated with emissions from wildfire events are subject to uncertainty arising from different sources. Here, we quantify and compare major uncertainties in mortality and morbidity outcomes of exposure to fine particulate matter (PM_2.5) pollution estimated for a series of wildfires in the Southeastern U.S. We present an approach to compare uncertainty in estimated health impacts specifically due to two driving factors, wildfire‐related smoke PM_2.5 fields and variability in concentration‐response parameters from epidemiologic studies of ambient and smoke PM_2.5. This analysis, focused on the 2016 Southeastern wildfires, suggests that emissions from these fires had public health consequences in North Carolina. Using several methods based on publicly available monitor data and atmospheric models to represent wildfire‐attributable PM_2.5, we estimate impacts on several health outcomes and quantify associated uncertainty. Multiple concentration‐response parameters derived from studies of ambient and wildfire‐specific PM_2.5 are used to assess health‐related uncertainty. Results show large variability and uncertainty in wildfire impact estimates, with comparable uncertainties due to the smoke pollution fields and health response parameters for some outcomes, but substantially larger health‐related uncertainty for several outcomes. Consideration of these uncertainties can support efforts to improve estimates of wildfire impacts and inform fire‐related decision‐making.

Introduction

Wildfires are one of the largest sources of fine particulate matter (PM_2.5) emissions in the U.S. (U.S. EPA, 2016), which is the primary pollutant of concern for human health (Cohen et al., 2017). As emissions from major anthropogenic sources continue to decrease and wildfire frequency and magnitude intensify under climate change, their influence on U.S. air quality is expected to grow (Kaulfus et al., 2017; Liu et al., 2016; McClure & Jaffe, 2018; U.S. EPA, 2019b). Exposure to wildfire smoke has been strongly associated with negative health impacts including premature mortality and respiratory morbidities, such as asthma and chronic obstructive pulmonary disease (COPD) (Cascio, 2018; Liu et al., 2015; Reid et al., 2016). Assessing the health risks posed by wildfire smoke is important to aid in the protection of the public as a whole and at-risk populations. Health impact assessments are used in cost-benefit analyses to provide economic context and, when assessing wildfire smoke exposure, can justify land management practices that mitigate wildfire risk. However, these health impacts have been quantified using various approaches that may lead to differing results. Identifying and weighing major sources of uncertainty is thus a key step to improve estimates of public health impacts associated with wildfire-related PM_2.5 and develop effective mitigation strategies.

Health impact analyses focused on wildfire smoke have been carried out across a range of temporal and spatial scales. Globally, exposure to wildfire PM_2.5 has been estimated to cause 260,000–600,000 annual deaths (Johnston et al., 2012). In the continental U.S., central estimates of annual premature mortalities due to long-term exposure to wildland fire smoke range from 8,700 to 32,000 (Fann et al., 2018). Smoke impact analyses have previously used health response information from epidemiological studies of both ambient and wildfire-attributable PM_2.5 (e.g., Fann et al., 2018, 2013; Ford et al., 2018; Hänninen et al., 2009; Johnston et al., 2012). Epidemiological studies of smoke exposure often observe increases in health outcomes in areas affected by smoke from specific wildfire events within the state or region (Liu et al., 2015) and by considering short-term (i.e., daily) exposure to smoke. An analysis of smoke exposure and emergency department visits during a peat bog wildfire in rural Eastern North Carolina found an increased risk for asthma, COPD, pneumonia, acute bronchitis, and heart failure during the event (Rappold et al., 2011). Higher respiratory and cardiovascular hospital admissions were similarly observed during a Southern California wildfire (Delfino et al., 2009), though evidence for the association between smoke exposure and increased cardiovascular outcomes is variable (Cascio, 2018; Reid et al., 2016). An examination of premature mortality during a 14-day smoke episode in Finland showed an increase in line with the risks reported by epidemiological studies based on general ambient PM_2.5 (Hänninen et al., 2009). During a summer wildfire season in British Columbia, respiratory health visits to physicians and hospitals were observed to increase with smoke exposure, although the risk ratio varied depending on the exposure metric used (Henderson et al., 2011). An analysis of COPD hospital admissions and wildfire smoke exposure in the state of Washington likewise found that different smoke estimation methods impact health outcome risk ratios to varying degrees (Gan et al., 2017). While health impacts have been observed in areas affected by smoke from wildfire events, the reported associations vary due to differences in several aspects of study design, including the exposure metric considered.

Various approaches have been used to quantify fire-attributable PM_2.5, including monitor observations and spatially interpolated monitor data (Delfino et al., 2009; Gan et al., 2017; Henderson et al., 2011; Johnston et al., 2021; Lassman et al., 2017; Liu et al., 2021), chemical transport models (Fann et al., 2018; Gan et al., 2017; Jiang & Yoo, 2019; Lassman et al., 2017; Rappold et al., 2017), dispersion modeling (Henderson et al., 2011), satellite imagery and retrievals of aerosol optical depth (AOD) (Geng et al., 2018; Henderson et al., 2011; Rappold et al., 2011; van Donkelaar et al., 2011), machine learning methods (Reid et al., 2015, 2019; Yao et al., 2018), and hybrid or blended data fields (Gan et al., 2017; Johnston et al., 2012; Lassman et al., 2017; O’Dell et al., 2021). While each method has particular strengths, each also has limitations that add uncertainty to health impact assessments and epidemiological studies of smoke exposure. Monitor data is spatially and temporally limited, and may not differentiate wildfire-related PM_2.5 from that emitted by other sources. Satellite data provide better spatial coverage but can be hindered by clouds, may be unable to reflect ground-level smoke concentrations, and is often temporally constrained, important drawbacks when analyzing health-relevant pollution from rapidly evolving wildfire events. Air quality models can address some of the limitations of monitors and satellites, but are dependent on accurate input data and subject to errors and biases, which can cause them to significantly over- or underestimate smoke concentrations (Baker et al., 2016; Goodrick et al., 2013). Modeled smoke is subject to additional uncertainties due to the variety of representations of fire-related processes such as heat release and fire spread, atmospheric chemical mechanisms, spatial and temporal resolution of emissions, and representations of plume rise and structure (Garcia-Menendez et al., 2014; Goodrick et al., 2013; Liu et al., 2019; Mallia et al., 2020).

Estimates of wildfire-driven concentration changes are used either in epidemiological studies (i.e., to examine associations of exposures and health outcomes) (e.g., Delfino et al., 2009; Gan et al., 2017; Henderson et al., 2011; Rappold et al., 2011), or in health impact assessments which use previously found exposure relationships (e.g., Fann et al., 2018; Hänninen et al., 2009; Johnston et al., 2012, 2021; van Donkelaar et al., 2011). Uncertainty is introduced at each stage of the modeling approach typically used for health impacts assessment of wildfire smoke. Estimates of wildfire smoke exposures carry uncertainties associated with measurement error, model parameters, and spatial interpolation or aggregation (Nethery & Dominici, 2019). Exposure-response relationships are subject to uncertainty arising from measurement error in pollution and cofounder data, sampling error, uncertainty in the functional form of the relationship, and heterogeneity among epidemiological studies (Nethery & Dominici, 2019). When wildfire smoke exposure estimates are used with exposure response relationships to assess regional health impacts, these uncertainties in wildfire PM_2.5 fields and the relationships between exposure and health outcomes propagate to estimates of smoke-attributable outcomes. Additional uncertainty is added to such estimates due to errors in population data and the generalization of exposure-response relationships to populations that may differ from those used to determine the relationships (Nethery & Dominici, 2019).

While some studies have contrasted methods of wildfire PM_2.5 exposure and others have examined variability in estimates resulting from the use of parameters from different epidemiological studies, no analysis has systematically assessed and compared these major drivers of uncertainty along the pathway to estimate the consequences of wildfire smoke in a health impact assessment. This study bridges the gap by evaluating multiple approaches to represent spatially resolved smoke fields, health response relationships from various epidemiological studies, and the resulting uncertainties that arise from these sources in the health impacts estimated for a wildfire event. Here, we investigate a large-scale wildfire complex in the Southeastern U.S. and calculate fire-attributable changes in PM_2.5 and associated mortality and morbidity in the state of North Carolina due to smoke exposure. We then examine and compare air-quality- and health-related uncertainties in these estimates. The analysis highlights the effects of two key sources of uncertainty in air pollution health impact assessments focused on wildfire smoke, and suggests areas for continued research toward better informed land and air quality management.

Methods 2016 Southern Appalachian Wildfires

In late 2016, a series of over 20 wildfires occurred in the Southeastern U.S. These fires, present across Georgia, South Carolina, North Carolina, Virginia, and Tennessee, burned in excess of 150,000 acres between October 2016 and January 2017, largely fueled by extreme drought conditions (James et al., 2020). Because this event had a substantial effect on air quality across the region, it offers a unique opportunity to explore the impacts of wildfire smoke on human health. Figure 1 shows the locations of wildfires reported to the U.S. Forest Service's interagency all-risk incident information management system (InciWeb; https://inciweb.nwcg.gov). During the wildfires, the largest increases in PM_2.5 concentrations in North Carolina were observed in the month of November, as highlighted in Figure 1b. Concentrations exceeding the 24-hr PM_2.5 National Ambient Air Quality Standard (NAAQS) were recorded on 14 days in November at 12 regulatory North Carolina PM_2.5 monitors (Table S1). Using the 2016 Southern Appalachian wildfires event as a case study, we estimated fire-attributable changes in 24-hr average PM_2.5 concentrations and associated health impacts on the population of North Carolina for the 30-day period of November 2016. By applying multiple methods to estimate fire-related PM_2.5 and corresponding health impacts, we evaluated and compared the uncertainties that can arise due to differences in spatially resolved PM_2.5 fields, response parameters from different epidemiological studies, and response parameters grouped by health outcome.

Figure 1. 2016 Southern Appalachian wildfires and air pollution. (a) Moderate Resolution Imaging Spectroradiometer (MODIS) visible imagery of the Southeastern U.S. on 14 November 2016 (https://modis.gsfc.nasa.gov/). Red markers denote locations of all fires active during the month of November, as reported by InciWeb. (b) 24-hr average PM2.5 concentrations observed by air quality monitor in three North Carolina cities in 2016. The 24-hr PM2.5 National Ambient Air Quality Standard (NAAQS) is indicated by the dashed red lines. Observations during November are highlighted by yellow shading.

Spatial Fields of Wildfire PM_2.5

Based on monitor observations and model-based products, we created six different spatial PM_2.5 fields to estimate wildfire-attributable air quality impacts in North Carolina. The fields represent wildfire-attributable daily PM_2.5 concentration during November 2016, the month with the largest observed air quality impacts in the state. All are based on publicly available data sources of daily ground-level air pollution. The data types used are available at a range of spatial resolutions; however, we normalized the horizontal resolution of the continuous spatial fields generated to a 12-km grid. On days in which data is missing, wildfire PM_2.5 was assumed to be zero.

Observed 24-hr PM_2.5 concentrations were obtained from the U.S. Environmental Protection Agency's (U.S. EPA) Air Quality System (https://www.epa.gov/aqs) for 172 monitors in North Carolina, South Carolina, Virginia, Tennessee, Kentucky, Georgia, and Alabama. For the November period, daily PM_2.5 attributable to the 2016 Southern Appalachian wildfires was approximated at each monitor by subtracting a baseline value from the daily concentrations observed. The baseline value was calculated for each monitor as the November-average concentration recorded at the site from 2012 to 2015, years with minor wildfire impacts relative to 2016. To reflect the variety of approaches in which station data is used, we created different fields of wildfire-related 24-hr average PM_2.5 concentrations in North Carolina. The “Closest Monitor” field uses the value from the nearest monitor to represent the concentration of each county. The “Central Monitor” field applies observed PM_2.5 values to the 12-km grid cell in which the monitor is located and grid cells within a 20 km radius. For grid cells within 20 km of more than one monitor, PM_2.5 concentration was calculated using inverse distance weighting (IDW). For grid cells with no monitors within 20 km, a fire-related concentration of zero was assigned. In addition, two fields were created by applying commonly used spatial interpolation methods across the entire state, inverse distance weighting (IDW) interpolation and spherical Kriging. Both interpolation fields are based on the 12 nearest monitors at each location.

Two wildfire PM_2.5 fields were created using model-based smoke products. The first was generated using the National Oceanic and Atmospheric Administration's (NOAA) Smoke Forecasting System (Rolph et al., 2009). This system uses the Hybrid Single-Particle Lagrangian Integrated Trajectory (HYSPLIT) model to simulate 48-hr smoke transport and concentrations over the continental U.S., based on satellite-derived wildfire locations, meteorological fields from the North American Mesoscale Forecast System, and U.S. Forest Service estimates of vegetation and emissions (Rolph et al., 2009). Daily-average smoke concentrations at 12-km resolution were calculated from the hourly forecast outputs. A second model-based field of wildfire impacts was created using concentrations predicted by NOAA's experimental High-Resolution Rapid Refresh Smoke (HRRR-Smoke) forecasting model (https://rapidrefresh.noaa.gov/hrrr/HRRRsmoke). The HRRR-Smoke model relies on the state-of-the-science Weather Research and Forecasting model coupled with Chemistry (WRF-Chem), data from National Weather Service models and observations, and satellite-derived fire radiative power measurements to predict hourly near-surface smoke PM_2.5 concentrations over North America at 3-km resolution. The forecast outputs were also processed into 12-km fields of wildfire-attributable changes in 24-hr PM_2.5 concentration.

Concentration Response Functions and Health Outcomes

An extensive body of epidemiologic research provides evidence of associations between various health effects and exposure to ambient PM_2.5 (U.S. EPA, 2019a). These relationships can be formulated as health-response functions and follow the general form $${\Delta}Y={y}_{0}(1-{e}^{-\beta {\Delta}P{M}_{2.5}})\ast P$$, where $${\Delta}Y$$ is the predicted change in a health outcome incidence, $${y}_{0}$$ is the baseline incidence rate of the health outcome, $${\Delta}P{M}_{2.5}$$ is the change in concentration, $$P$$ is the exposed population, and β is the study risk coefficient. We used U.S. EPA's Benefits Mapping and Analysis Program (BenMAP-CE version 1.5) (Sacks et al., 2018) to predict changes in health outcomes associated with wildfire PM_2.5 pollution. BenMAP-CE compiles census data, baseline health incidence rates, and concentration-response functions (CRFs) reported by the epidemiological research. We used 2010 U.S. Census population at the county level and reported county or state incidence rates, to estimate smoke-related health impacts (Sacks et al., 2018; U.S. EPA, 2021).

In prior analyses quantifying the health burden of PM_2.5 from different sources, CRFs derived from studies of general ambient PM_2.5 have been used for regional-scale estimates of health impacts from wildfire smoke (e.g., Fann et al., 2018, 2013; Ford et al., 2018; Hänninen et al., 2009; Johnston et al., 2012). A growing number of epidemiologic studies have reported relationships specifically between wildfire smoke exposure and health outcomes, which could plausibly be used in health impact analyses of other wildfire events. However, a limited number of health endpoints have been evaluated by these wildfire studies, many were conducted outside of the U.S., and the associations commonly use smoke exposure metrics different to PM_2.5 concentration (Henderson et al., 2011; Johnston et al., 2007; Liu et al., 2015; Morgan et al., 2010), which can complicate the identification of studies to be used in a health impact analysis of wildfire smoke exposure. To estimate wildfire-attributable impacts on mortality and morbidity, we used CRFs relating both general ambient and wildfire-related PM_2.5 concentrations to several health outcomes. We focused on health outcomes that have strong or very strong association with exposure to smoke (Cascio, 2018; Liu et al., 2015; Reid et al., 2016; Youssouf et al., 2014) and epidemiological studies conducted in the U.S., with the exception of Hänninen et al., (2009). We used parameters from at least one wildfire-specific CRF for each health outcome group, as well as general ambient PM_2.5 CRFs that have been used in other wildfire smoke health assessments. Estimates for the same outcome and PM_2.5 exposure source were pooled using the fixed- or random-effects weighting models in BenMAP (U.S. EPA, 2021). This method creates a weighted average impact by weighting individual study estimates according to their level of uncertainty and the variability between studies. Information about the epidemiological studies considered (Aguilera et al., 2021; Babin et al., 2007; Delfino et al., 2009; Gan et al., 2017; Glad et al., 2012; Hänninen et al., 2009; Kloog et al., 2012; Krewski et al., 2009; Laden et al., 2006; Lepeule et al., 2012; Mar et al., 2010; Norris et al., 1999; Ostro, 1987; Pope et al., 2002; Rappold et al., 2012; Reid et al., 2019; Sheppard, 2003; Slaughter et al., 2005; Zanobetti & Schwartz, 2009; Zanobetti et al., 2009) is included in Table S2.

Uncertainty Evaluation

We evaluated uncertainty in estimates of health impacts associated with wildfire smoke due to two driving factors, representation of fire-related air pollution and health responses to air quality. Uncertainty is weighed as 95% confidence intervals (CIs) on mean estimates of health impacts (point estimates), consistent with U.S. EPA benefits analyses (Fraas, 2011), and examined based on different air quality fields, epidemiological studies, and health endpoint groups. To compare uncertainty consistently across sources and health endpoints, CIs were represented as a fraction of the point estimate. Uncertainty due to fire-attributable PM_2.5 concentrations was calculated using a bootstrap simulation in AuvTool 1.0 (Zheng & Frey, 2002), which assumes that the health impact estimates resulting from each of the six air pollution fields and each CRF are a random sample from the population distribution. Bootstrap samples were generated with Monte Carlo simulations and used to calculate a 95% CI on the mean health impact estimate for each CRF. The uncertainty associated with individual health response relationships was calculated in BenMAP using a 20-point Latin Hypercube method to represent the underlying distribution of β in the CRFs associated with each epidemiological study considered. This sampling method enhances processing efficiency by dividing a probability distribution into intervals of equal probability, and is more precise than conventional Monte Carlo sampling (U.S. EPA, 2021). This uncertainty reflects a range of study-specific factors, including study duration, exposure estimates, and confounder adjustment, among others. When pooled impact estimates were made for a health outcome, a pooled distribution of the incidence changes was derived and was used to characterize the uncertainty in the pooled estimate. Outcome group uncertainty was also examined by calculating an unweighted average of the study uncertainties (as a percent of the mean point estimate) for each group.

Results and Discussion Air Quality and Public Health Impacts of the 2016 Appalachian Wildfires

The 2016 Southern U.S. wildfires had a substantial impact on North Carolina air quality, as reflected in monitor observations and air quality forecasts. During the month of November, heavy smoke was observed most frequently over the western region of the state where the wildfires were concentrated. The highest daily PM_2.5 concentrations were recorded at monitors in western North Carolina, with the maximum, 99 µg/m³, recorded in Bryson City on 23 November. During November, Bryson City experienced 12 days of air pollution exceeding the 24-hr PM_2.5 NAAQS, the largest number among state monitors. The maximum observed 24-hr concentrations in the cities of Asheville, Charlotte, and Raleigh were 87 µg/m³, 61 µg/m³, and 49 µg/m³, respectively. Asheville experienced multiple days of elevated PM_2.5 during the month of November, while Charlotte and Raleigh, the largest metropolitan areas in North Carolina, recorded one and two exceedances of the NAAQS, respectively. The large number of fires burning throughout the study period makes it difficult to pinpoint which might be directly responsible for specific impacts. However, on the days with the highest PM_2.5 concentrations observed in Raleigh and Charlotte, smoke transport to these areas was likely from the Party Rock and Chestnut Knob fires in North Carolina, the Pinnacle Mountain fire in South Carolina, and the Rock Mountain fire in Georgia (shown in Figure 1).

Different approaches to represent the air pollution caused by the wildfires led to a wide range of predicted air quality impacts. Figure 2 shows average wildfire-attributable PM_2.5 concentrations over North Carolina for each data source and spatial method considered. In general, monitor-based approaches resulted in a large spread of PM_2.5 impacts across the state, while the model-based approaches yielded both the highest and the lowest estimates. November-average wildfire PM_2.5 concentration for the state ranged from 0.6 µg/m³ using the Central Monitor method to 4.6 µg/m³ using the HYSPLIT Smoke Forecast. The model-based approaches did not capture the full magnitude of concentration spikes observed at monitors in Charlotte and Raleigh, although the HYSPLIT Smoke Forecast reflected the high concentrations observed in the western region of the state.

Figure 2. Average wildfire-attributable PM2.5 concentration during November 2016 for each spatial method considered. The statewide average concentration for each method is listed above each map with the population-weighted average value in parentheses. On observation-based fields, dots show North Carolina monitor locations.

Across spatial fields, the largest air quality impacts were estimated over western North Carolina, with smaller, varying effects throughout the rest of the state. While the magnitude and distribution of fire-attributable PM_2.5 varies among methods, the highest impacts were consistently assigned to western counties. The largest November-average estimate of wildfire PM_2.5, 29.4 µg/m³, was estimated in Polk County by the HYSPLIT Smoke Forecast. Considering the ratio of county population to state population, this corresponds to a population-weighted impact of 0.06 µg/m³. In contrast, the wildfire PM_2.5 impact estimated by the same method in Wake County, the state's most populous, was 2.62 µg/m³ but equivalent to a population-weighted impact of 0.25 µg/m³, showing how although smoke pollution may have been lower in large urban areas outside of western North Carolina, the exposed population at these locations was substantially larger. State-wide population-weighted concentrations of wildfire PM_2.5 in November range from 0.8 to 5.1 µg/m³, depending on the spatial method used. The uncertainty in these fire-related pollution fields propagates to the estimates of associated health impacts, where the uncertainty due to health response modeling may be larger still.

The air quality impacts from this event potentially translate to tens of premature mortalities, tens to hundreds of asthma-related emergency room visits, and thousands of work loss days. Depending on the spatial PM_2.5 pollution fields and health-response functions selected, point estimates of smoke-related premature mortalities range from 5 to 34, work loss days from 3,700 to 24,900, asthma-related emergency room visits from 12 to 70, and respiratory-related hospital admissions (ages 65+) from 9 to 58 in North Carolina during the month of November (Table 1). The largest number of health impacts were estimated to occur in both western North Carolina and large urban areas (Figure S1). While estimates vary by exposure estimation, the highest smoke-related health incidences were estimated in Buncombe, Mecklenburg, and Wake Counties—containing the metro areas of Asheville, Charlotte, and Raleigh. For example, work loss days (WLD) estimates for Buncombe county range from 212 (179, 243; 95% CI) to 1,831 (1575, 2079), depending on the exposure estimation. Further, estimated impacts based on wildfire-specific CRFs vary from those based on CRFs derived from general PM_2.5 (Table S3). Respiratory and asthma hospital admissions estimates from wildfire-specific CRFs tend to be larger than those from general ambient PM_2.5 CRF estimates. This is likely due to the magnitude of PM_2.5 concentrations during wildfire events relative to typical ambient concentrations. Mortality estimates produced by CRFs based on short-term exposure to general and wildfire-specific PM_2.5 are comparable, but an order of magnitude smaller than those that would be predicted by long-term ambient PM_2.5 mortality CRFs. While these types of estimates shed light on the potential gravity of wildfire impacts, they are also susceptible to large uncertainties that arise from the air quality and health response methods used.

Table 1 Health Impacts of Wildfire Smoke in North Carolina During November 2016 Estimated With Each Air Pollution Field Considered

Air quality field	Health outcome a
	Mortalities b	WLDs	ER Visits (asthma) c	HA Resp. 65+ d
Closest Monitor	19	14,300	46	30
	(14, 23)	(12,284, 16,324)	(23, 67)	(10, 60)
Central Monitor	9	7,800	25	17
	(7, 11)	(6652, 8896)	(12, 36)	(5, 33)
IDW	30	20,400	67	52
	(22, 37)	(17,328, 23,280)	(32, 98)	(16, 108)
Kriging	27	18,200	60	47
	(20, 33)	(15,455, 20,786)	(29, 88)	(15, 98)
HYSPLIT Smoke Forecast	34	24,900	70	58
	(26, 42)	(21,111, 28,749)	(35, 100)	(19, 120)
HRRR Smoke Forecast	5	3,700	12	9
	(4, 7)	(3111, 4234)	(6, 18)	(3, 21)

^a95% confidence intervals shown in parentheses.

^bMortality estimates based on Zanobetti and Schwartz (2009) (short-term ambient PM_2.5 exposure).

^cER visits estimated by Rappold et al. (2017) (wildfire smoke exposure study).

^dHA (Resp., 65+) indicates all respiratory hospital admissions for population aged 65 and over, estimated by pooling Delfino et al., (2009), and Gan et al., (2017) (wildfire smoke exposure studies).

Uncertainty in Estimates of Smoke-Related Health Impacts

Uncertainty in wildfire-related PM_2.5 strongly propagates to estimates of public health impacts. Statewide health impacts for four outcomes are shown in Table 1. Across spatial methods, impact estimates can span an order of magnitude, with premature mortalities ranging from 5 to 34 for the 30-day period in November 2016. Projected impacts on WLDs, asthma-related emergency room (ER) visits, and respiratory-related hospital admissions (HAs) also vary significantly across methods. The HYSPLIT Smoke Forecast field and the Closest Monitor field yielded the largest health impact estimates for this wildfire event. The HRRR Smoke Forecast and the Central Monitor fields yielded the smallest impacts. The range in health impact estimates across spatial fields based on a single CRF relating premature mortality to short-term PM_2.5 exposure is shown in Figure 3.

Figure 3. Wildfire smoke-related mortality in North Carolina during November 2016 based on Zanobetti and Schwartz (2009), estimated with each spatial air quality field considered. Center lines of boxes indicate point estimates. Whiskers and box edges indicate 95% and 50% confidence intervals, respectively, based on the epidemiological study uncertainty.

To evaluate the uncertainty in health impact estimates due to the differences in the smoke pollution fields, we used a bootstrapping method to calculate 95% CIs for the mean estimate of each health response function, as described in Methods. Uncertainty in the health impact estimates due to the variation in smoke pollution fields ranges from ±36 to ±55% of the spatial mean impact estimate, as listed in Table 2. The uncertainty due to spatial fields varies slightly between CRFs due to differences in the spatial resolution of baseline incidence and prevalence reporting (e.g., county- or state-level), as well as the magnitude of the impact. When averaged across all CRFs, the uncertainty due to the smoke fields is ±47%.

Table 2 Uncertainty in Smoke Impact Estimates due to PM_2.5 Concentration Fields and Health Response Relationships

[Table omitted. See PDF]

^aAverage uncertainties estimated for groups of multiple wildfire-specific or general PM_2.5 morbidity studies.

^bUncertainty values are averages (+/−) of all smoke fields considered. Some CIs are not equal in positive and negative directions.

^cAverage outcome uncertainty calculated from uncertainties in the positive and negative directions for each study in the group.

^dWildfire-specific study.

^eWRF-Chem and GWR refer to chemical transport modeling and geographically weighted regression approaches in study, respectively.

^fImputation and Interaction refer to two regression methods used to isolate wildfire PM_2.5 in study.

^gShort-term PM_2.5 exposure study.

Uncertainty in the impact estimates from the CRF parameters varies among studies and outcomes. Expressed as a percentage of the point estimate, the uncertainty is smallest (±14%) for WLDs and largest (±548%) for mortality due to short-term exposure to wildfire PM_2.5 based on Hänninen et al. (2009). The magnitudes of uncertainty, expressed as the 95% CIs relative to the point estimate, among all epidemiological studies considered are compared in Figure 4 based on the Kriging smoke pollution field. Uncertainty stemming from individual epidemiological studies differs significantly (due to a variety of factors of the study design) and can be considerably larger than the uncertainty introduced by the smoke fields. Figure 4 also includes 95% CIs for some pooled estimates to compare uncertainty magnitude by health outcome. Within outcome groups, health response uncertainty can vary widely, but the largest uncertainties in this analysis are associated with hospital admissions estimates. Table 2 lists the uncertainty associated with each epidemiological study and average uncertainty for each health outcome, along with the uncertainty due to air pollution fields. Average health outcome uncertainties are estimated separately for those based on general or wildfire-specific PM_2.5 studies. Among the epidemiological studies used in this work, the largest average outcome uncertainty accompanies asthma hospital admissions estimates, ranging from ±59% to ±426% for general ambient PM_2.5 studies and ±35% to ±68% for wildfire smoke studies. The smallest average outcome uncertainty (for outcomes with more than one study considered) is ±52% for long-term exposure mortality estimates (Table S3). Fewer wildfire-specific epidemiological studies were analyzed—however, these generally carry smaller uncertainty than the ambient PM_2.5 studies considered here.

Figure 4. Health response uncertainty in North Carolina smoke-related incidences during November 2016 expressed as a percentage of the point estimate for pooled and individual study health impacts. Darker boxes show pooled estimate uncertainty while lighter boxes indicate individual study uncertainty. Box edges and whiskers indicate 50% and 95% confidence intervals, respectively. Values here reflect uncertainty in impact estimates based on the smoke pollution field generated with the Kriging method. Studies and pooled estimates marked with ⁺ are based on ambient exposure parameters and those marked with ‡ are based on wildfire smoke exposure parameters.

Large uncertainty may exist in wildfire impact estimates due to both representation of fire-related air pollution and the health responses to smoke. The magnitude of uncertainty due to spatial air pollution fields (±47% on average) is comparable to the health response uncertainty for some outcomes, such as asthma hospital admissions specifically associated with wildfire smoke (±35–68%). However, the uncertainty due to the smoke fields is smaller than the health response uncertainty for many of the epidemiological studies considered. This indicates that estimates of health impacts attributable to wildfire smoke would benefit from improvements to either spatial representations of smoke concentrations or epidemiological associations, as the uncertainties stemming from each of these can be large. The lower uncertainties accompanying many of the wildfire-specific studies suggest that source-specific epidemiological studies may reduce uncertainty. However, because this large-scale wildfire event occurred in a densely populated region of the country, the impacts and health-related uncertainties discussed here may be larger than for other wildfires in the U.S. This region also has a denser network of PM_2.5 monitors compared to other wildfire-prone areas, which may have reduced uncertainty in the smoke fields estimates derived from monitor observations.

Implications for Smoke Impact Assessments and Fire Management

Wildfires are a complex problem that require unique management strategies. Mitigating smoke pollution is an important component of this challenge. To arrive at effective solutions, decisions must be informed by complete wildfire impacts analyses, including the consequences of air pollution on public health. However, current estimates of smoke-related health effects remain largely uncertain. Reducing these uncertainties is a critical step to improve fire- and air quality-related decision-making.

Here, a unique approach is used to compare major sources of uncertainty in wildfire smoke health impact analyses. With the exception of one study examining bias in modeled smoke and a single epidemiological function (Jiang & Yoo, 2019), prior research has not compared and evaluated the possible uncertainties that arise in health impact analyses due to the variety of smoke estimation methods and available epidemiological risk-ratios. Our analysis of a large wildfire complex shows the important influence of uncertainties due to representation of fire-related air pollution and the relations between smoke exposure and health risks. Estimates of regional wildfire smoke health impacts resulting from different air pollution fields can span an order of magnitude. Still, for several outcomes expected to be associated with smoke pollution, the health-related uncertainty is substantially larger.

While there is no established methodology to evaluate wildfire smoke-related health impacts, the air quality data types and methods considered here are regularly used for this purpose. Despite these inherent uncertainties, it is apparent that the 2016 Southern Appalachian wildfires led to elevated PM_2.5 concentrations across the Southeastern U.S. and very likely had consequences on public health. In North Carolina alone, this event potentially was associated with tens of premature mortalities, several thousand lost days of work, and dozens of respiratory- and asthma-related hospital admissions. No estimates of public health impacts associated with smoke from the historic 2016 Southern Appalachian wildfires have been previously reported.

Large uncertainties in estimates of wildfire smoke impacts highlight several research needs. Analyses of air pollution and health effects methods used to quantify the impacts of additional wildfires in different locations would show whether major drivers of uncertainty identified in this study carry similar weights in estimates for other events. Data fusion or blended smoke products (Diao et al., 2019; O'Dell et al., 2019b, 2020, 2021; Yuchi et al., 2016) can combine the strengths and minimize the limitations of multiple data sources to improve representations of wildfire air pollution. Increased air quality monitoring, in particular further from large urban centers and near fire-prone lands, can improve spatial coverage provided by observational networks. Temporary and low-cost sensors have been proposed for this application (AirNow, 2020; Gupta et al., 2018; Kelleher et al., 2018; Morawska et al., 2018). New satellite retrievals may be better able to capture ground-level smoke concentrations with improved temporal resolution and algorithms (Griffin et al., 2020; Lyapustin et al., 2020; Veefkind et al., 2012). Regional-scale air quality modeling of smoke can be improved in several ways, including emissions data, smoke-relevant chemical mechanisms, and resolution of concentrated smoke plumes (Goodrick et al., 2013; Jaffe et al., 2020). The spatial resolution of simulated smoke should be evaluated for use in health impact assessments (Jiang & Yoo, 2018).

Some of the largest uncertainties in this analysis correspond to the relationships between air pollution concentrations and health responses reported by several epidemiological studies. The nature of these studies, particularly for certain morbidity outcomes and wildfire-specific pollution, can make it challenging to reduce this uncertainty. Specific elements of this health-related uncertainty, such as the linearity and shape of CRFs, thresholds, confounding, and effects modifications, should be further investigated in the context of wildfire smoke. Most epidemiological studies of air pollution exposure, including those considered here, use a log-linear association, which is beneficial for studying large cohorts but may not provide a true representation of the shape of the CRF, especially when considering exposure thresholds or elevated pollution concentrations (Nasari et al., 2016). Other nonlinear models (e.g., Integrated Exposure-Response (IER) and Global Exposure Mortality Model (GEMM)) have been used to examine responses globally and at higher exposure concentrations. However, to represent more realistic responses at extreme concentrations, these models may use information on exposure to non-ambient sources of PM_2.5, thus assuming that exposure to all sources of PM_2.5 results in the same health risk, or the model shape is restricted such that exposures at higher concentrations result in declining risk (Burnett & Cohen, 2020). For wildfire events in particular, health impacts are likely driven by the acute effects of short-term smoke exposure. Studies of the relationships between short-term exposure events like wildfire and health responses can be more uncertain relative to those between long-term exposure to PM_2.5 pollution or multi-year studies of short-term exposure responses which are often the main focus of air pollution impact assessments. Although prior estimates of the public health impacts of wildfire smoke have relied on CRFs derived from studies of general ambient PM_2.5 and some analyses have found the magnitude of impacts to be similar to those obtained with wildfire-specific health response relationships (Hänninen et al., 2009; Johnston et al., 2012), other recent studies have reported larger health effects associated with wildfire-specific PM_2.5 (Aguilera et al., 2021; DeFlorio-Barker et al., 2019). While the health-related uncertainty discussed here is specific to the studies considered and is dependent on several aspects of study design, the overall lower uncertainty associated with the wildfire-specific studies considered in our analysis further suggests that additional source-specific epidemiological research investigating the acute effects wildfire air pollution may reduce uncertainty in estimated smoke health impacts.

The findings of this study can provide a helpful assessment of the magnitude and major sources of uncertainty in health impact estimates due to wildfire smoke. However, the constraints of the analysis may limit its applicability to U.S. wildfire broadly. We apply a range of methods, from simple, conservative estimates of smoke (e.g., 20-km radii from regulatory site measurements) to advanced, comprehensive tools (e.g., NOAA's HRRR model), to reflect the variety of approaches previously used in quantifying the health impacts of wildfire air pollution. This is not an exhaustive compilation of the methods available. Rather, we evaluate methods and data types that can shed light on the uncertainty present in the existing collective body of research. Further, we do not quantify the uncertainties contributed to health impact assessments by population data, including population counts and demographics, incidence rates, and generalization of concentration-response functions to populations that may be non-representative of those used to determine the relationships (Nethery & Dominici, 2019). Here, we explore a specific wildfire event in the Southeastern U.S. Uncertainty in smoke impact estimates will vary for different wildfires, regions, and ecosystems. The uncertainties may also differ at higher or lower smoke levels, as well as for pollutants other than PM_2.5. Still, we expect that for most events with significant public health impacts, smoke- and health-related uncertainties will similarly be large. Whether uncertainty in health responses to smoke averts the need to improve representations of pollutant concentrations, or the opposite, may depend on the wildfire event being considered.

Although air pollution has recently decreased over much of the U.S., smoke is limiting improvements in fire-prone regions (McClure & Jaffe, 2018; O’Dell et al., 2019a). In contrast to other major pollution sources, however, eliminating natural fire processes, and their emissions, from ecosystems is not a viable strategy. As land managers work to reach sustainable fire regimes, decisions about fuel treatments and interventions, including prescribed burning, mechanical fuel reduction, and adaptation measures, must be informed by comprehensive analyses that consider fire damages, suppression and treatment costs, ecological services, climate benefits, smoke impacts, and other aspects of wildland fire (Altshuler et al., 2020). Given its potential effects on public health, smoke pollution can be a major component of wildfire cost-benefit assessments. Uncertainty in smoke impacts will propagate to valuations of wildfire costs (Fann et al., 2018; Kochi et al., 2012; Rappold et al., 2014; Rittmaster et al., 2006), which carry additional uncertainties. Recognizing the uncertainty in estimates of the health impacts of smoke, and working to improve them, is needed to aid in developing effective policies addressing wildfire challenges in the U.S.

Acknowledgments

This material is based upon work supported by the National Science Foundation under Grant No. 1751601. Research reported in this publication was also supported by the National Institute of Environmental Health Sciences of the National Institutes of Health under Award Number P30ES025128. The authors thank and acknowledge Ravan Ahmadov and Eric James of NOAA for sharing the 2016 HRRR data, and the EPA BenMAP-CE team and the NCSU Libraries Data and Visualization Services for their assistance.

Conflict of Interest

The authors declare no conflicts of interest relevant to this study.

Data Availability Statement

The data and details on the software used for this study are available at https://doi.org/10.5061/dryad.79cnp5hwq.