Quantifying habitat losses and gains made by U.S. Species Conservation Banks to improve compensation policies and avoid perverse outcomes

Abstract

Compensation policies seek to counterbalance biodiversity losses caused by development; however, their effectiveness is rarely tested. We examined U.S. Species Conservation Banks (SCBs) in California, a compensation program initiated 30 years ago. We quantified the effect of 59 SCBs (15,350 ha) on habitat extent using statistical matching methods. SCBs averted a small, yet significant, amount of habitat loss (62 ha) between 2001 and 2011. However, unexpectedly, SCBs also averted significant habitat gains (1,424 ha). It is not possible to determine if losses averted by SCBs equaled losses caused by development for which credits were sold (because records of the latter do not exist), but estimated averted gains were 35 times greater than averted losses. To improve practice, SCBs must be designed to achieve outcomes that are additional and avoid crowding out other programs incentivizing statewide conservation goals.

1 INTRODUCTION

Compensation policies aim to counterbalance residual biodiversity losses caused by development, following efforts to first avoid and minimize biodiversity losses. An estimated 115 countries now have compensation policies (IUCN, ) and other individual compensatory programs and projects exist (Madsen, Carroll, & Moore Brands, ). Unsurprisingly, this has also corresponded with a huge research effort to understand the effectiveness of compensation. Studies have scrutinized differences in policy goals (Maron et al., ) and project implementation (Sonter et al., ), predicted future effects of compensation on biodiversity (Gordon et al., ; Sonter, Barrett, & Soares‐Filho, ), and demonstrated outcomes of individual projects (Lindenmayer et al., ). However, with the exception of U.S. wetland compensation activities (Matthews & Endress, ), ex‐post assessments of policy‐wide outcomes are rare, and so policy effectiveness remains unclear and debate continues over whether compensation has positive or negative outcomes.

Quasi‐experimental approaches, such as statistical matching, provide an opportunity to fill this gap (Morgan & Winship, ). Although matching methods have not yet been applied in a compensation context, they have been used to evaluate effects of protected areas on environmental outcomes (Andam, Ferraro, Pfaff, Sanchez‐Azofeifa, & Robalino, ; Pfaff et al., ). The approach compares treatment sites (e.g., areas protected for compensation) with matched controls (e.g., unprotected areas, with similar biophysical and socioeconomic characteristics). Matched controls represent a counterfactual scenario against which differences in an outcome variable (e.g., biodiversity loss) can be determined.

Estimating the effect of compensation is particularly challenging because counterfactual scenarios are complicated (Peterson, Maron, Moillanen, Bekessy, & Gordon, ). A treatment site might differ from a matched control site in terms of biodiversity losses—but if losses at the treatment site trigger compensatory requirements themselves, they should be excluded from counterfactual scenarios (Maron et al., ); failing to do so would overestimate the benefits of compensation (Maron, Bull, Evans, & Gordon, ). Further, compensation often occurs in a context of other environmental interventions, and might even reduce or crowd out their benefits (Gordon, Bull, Wilcox, & Maron, ). A carefully calibrated statistical matching approach accounts for this complexity to estimate the true effect of compensatory policies.

We use statistical matching to examine the effect of the U.S. Species Conservation Banking program on the extent of natural vegetation. Species Conservation Banks (SCBs) seek to counterbalance negative impacts of development (typically mining, infrastructure, and urban expansion) on threatened and endangered species and habitats (US FWS, ). Conservation bankers, using easements or other mechanisms, establish SCBs. Management plans are developed to benefit target species (Fox & Nino‐Murcia, ) and sometimes to improve ecosystem services and achieve no net loss of biodiversity (Gamarra & Toombs, ). Bank credits are generated through habitat protection or, in some cases, restoration. Methods used to determine credits generated by SCBs differ among SCBs; many use habitat extent alone, some consider the conservation value of the site, none assess the additionality of gains made by compensation (Gamarra & Toombs, ). Credits are then sold to developers who are required to compensate for their impacts on listed species, within the same “service area” (i.e., a nearby area with similar physical and ecological attributes).

A comprehensive database of SCBs exists (RIBITS, ) and several studies have reviewed their implementation (DOI, ; Fox & Nino‐Murcia, ; Gamarra & Toombs, ). However, none has examined the effect of SCBs on habitat extent, with the purpose of evaluating policy effectiveness and consequences for statewide conservation goals. Here, we examine SCBs in California, home to 90% of SCBs, to ask three questions: (1) Which species and habitats do SCBs conserve? (2) What explains the distribution of SCBs across California? and (3) To what extent do SCBs affect habitat losses and gains? Although we were unable to determine whether SCBs counterbalance biodiversity losses due to development (due to data limitations), results highlight potential perverse outcomes, and reveal challenges in comprehensive assessments of policy effectiveness, particularly when considered in the context of other, potentially competing, environmental policies.

2 METHODS

2.1 Data summary

We linked spatial data (point locations) on California's 98 SCBs (RIBITS, ) to information on project size, approval date, targeted species and habitats, and management goals (Gamarra & Toombs, ). We approximated SCBs’ spatial footprints, using their area (available for 72 SCBs) and the map buffer tool in ArcGIS (Figure ). We obtained modeled distributions of species and habitats targeted by SCBs when available (Comer et al., ; USGS, ) and overlaid them with our map of SCBs to determine spatial coincidence. We also obtained and overlaid data on land cover in 2001 and land cover change between 2001 and 2011 (Homer et al., ). Land cover maps were reclassified as natural (forest, shrubland, and herbaceous vegetation), agricultural (grazing and cultivated areas), and developed (artificial land cover) land. Land cover changes were reclassified as gains in natural land cover, losses in natural cover to agriculture, and losses in natural cover to developed land. We performed analyses at 1 ha resolution.

Distribution of SCBs and land cover in 2001 across California. Insets show the area around Sacramento. (a) Treatments (SCBs established pre‐2011), land cover in 2001, and land cover change between 2001 and 2011. Size of SCB symbol is proportional to its area (ha). (b) Treatments, single covariate used in matching algorithm (i.e. elevation [m] – see Methods) and SCBs that target Swainson's hawk (Buteo swainsoni; Table S2). (c) Treatments, matched controls (see Methods) and modelled distribution of Swainson's hawk (USGS, )

2.2 Quantifying effects of SCBs on biodiversity

We used statistical matching to quantify the effect of SCBs established between 2001 and 2011 on biodiversity. The outcome variable used to evaluate conservation should relate to the policy objective, which here was to compensate for development impacts on threatened species and habitats (US FWS, ). It was not possible to measure this outcome directly, because statewide data were unavailable for most species. Instead, we examined the effect of SCBs on natural land cover, as a proxy for habitat extent. Our analysis examined habitat extent alone, rather than also changes in habitat quality. Within SCBs, we assumed natural cover indicated species’ habitat, given that (when data were available) most banks coincided with modeled distributions of target species and that most (79%) SCBs calculate credits using habitat area alone (Gamarra & Toombs, ). Outside SCBs, we assumed natural cover with similar landscape characteristics would provide similar habitat and, when these areas were near SCBs, also contained appropriate habitat for targeted species. To make these assumptions, we chose matched controls proximate to SCBs and used covariates that represented species distributions (see sections below).

We examined the effect of SCBs on habitat losses and gains separately. We expected SCBs to negatively affect habitat loss (indicating SCBs avert losses) and positively affect habitat gains (indicating SCBs undertake restoration). Losses were indicated by transitions from natural cover to agricultural or developed land; gains were indicated by transitions from agricultural or developed to natural, which could represent natural recovery or active restoration.

Treatments were any observation (1 ha grid cells) within SCBs approved prior to 2011 (59 SCBs). To quantity effects on habitat losses, we examined treatment observations classified as natural cover in 2001 (10,475 observations). We also separately examined treatment observations located within SCBs approved prior to 2001 (24 SCBs) to strictly align with our analysis period (2001–2011); and observations more than 100 m from SCB edge boundaries (3,408 observations) to exclude observations misclassified as SCBs when delimiting spatial footprints. To quantify effects of SCBs on habitat gains, we examined treatments classified as developed or agricultural land in 2001 (192 observations). For each set of treatments, we randomly selected 1,000 observations except when examining habitat gains, where we used all observations.

We used 11 covariates hypothesized to explain SCB allocation across California and drivers of habitat losses and gains. These included seven datasets from USGS (): aspect, human disturbance (dummy variable for high/medium vs. low/no human alteration), elevation, ecotone type, forest edge width, canopy cover, and slope. These covariates were used to parameterize threatened species distribution models (US FWS, ) and thus improve chances that treatments and matched controls were of similar habitat type. They also explained the distribution of U.S. conservation easements (Baldwin & Leonard ). We also included four indicators for demand for ecosystem services (carbon storage, sediment export from agriculture, recreational bird watching, and crop pollination; US EPA, ) because some SCBs aimed to conserve them (Gamarra & Toombs, ).

Potential control observations were located outside protected areas (IUCN GAP status 1 or 2; GreenInfo Network, ). Control observations for examining effects on habitat loss were natural cover in 2001, whereas controls for examining habitat gains were developed or agricultural. We randomly sampled 100,000 observations for each analysis in R version 3.4 with the package “Matching” (Ho, Imai, King, & Stuart, ; Sekhon, ). We calculated covariate means for treatment and control observations prematching and used the Nearest Neighbor method to select matched controls that minimized differences. We matched each treatment observation to a unique control observation, but also assessed three alternative matching methods: (1) controls matched to multiple treatments; (2) five controls matched to each treatment; and (3) using calipers to constrain matches within 0.25 standard deviations of propensity scores (Rosenbaum & Rubin, ; Stuart, ).

Matching removed all significant differences in covariate values between treatments and controls (Table ). All covariates had lower absolute mean differences postmatching, except for agricultural sediment export, which slightly increased (–0.07 to –0.14 tons; Table ). Alternative matching methods did not improve match statistics (Supporting Information Table S3); results were not sensitive to hidden bias (Supporting Information Table S1). To determine effects of SCBs, we compared outcome variables between treatments and matched controls, using Chi square tests. For habitat losses, we analyzed two levels of loss: all losses and unregulated losses. Unregulated losses were transitions from natural cover to agricultural land, and thus did not completely remove all potential habitat (i.e., some species can utilize agricultural landscapes).

Covariate balance, comparing treatments to potential (U), and matched (M) controls. Lower mean differences postmatching indicate good performance for each covariate. Table also shows mean (eQQ Mean) and max (eQQ Max) difference from empirical quantile–quantile plots. Treatments were SCBs established pre‐2011 (59 SCB); matching method was nearest Neighbor, 1‐to‐1, without caliper. Supporting Information Table S4 shows covariate balance for SCBs established pre‐2001 and when excluding SCB edges

		Natural land cover (for analysis of loss)	Developed or agricultural land (for analysis of gain)
Variable		Mean Treat	Mean Control	Diff in mean	eQQ Mean	eQQ Max	Percent improve mean diff	Mean Treat	Mean control	Diff in mean	eQQ Mean	eQQ Max	Percent improve mean diff
Aspect	U	236.48	183.65	52.83	57.42	115	93.71	234.59	200.93	33.67	34.61	102.00	69.25
	M		241.48	−5.00	18.14	30			213.84	20.76	14.44	60
Human	U	0.22	0.15	0.07	0.08	1	90.85	0.57	0.44	0.13	0.13	1.00	91.89
impact	M		0.23	−0.01	0.02	1			0.57	0.00	0.01	1
Elevation	U	126.39	934.15	−807.76	805.92	3,438	99.81	112.76	606.53	−493.3	506.01	3,861.00	98.24
	M		125.65	0.74	25.70	255			109.14	3.62	19.61	124
Forest edge	U	0.21	3.15	−2.94	2.89	11	97.85	0.07	1.46	−1.38	1.42	11	97.74
	M		0.27	−0.06	0.13	6			0.11	−0.04	0.03	1
Canopy	U	2.50	18.69	−16.18	16.07	81	95.89	0.33	7.13	−6.81	7.06	75	99.46
	M		2.74	−0.24	1.60	3.5			0.34	−0.01	0.05	4
Slope	U	4.41	12.32	−7.91	7.52	22	98.9	2.62	6.59	−3.97	4.18	37	87.87
	M		4.06	0.35	1.07	6			2.42	0.20	0.87	6
Ecotone	U	5.00	10.50	−5.50	5.45	8	97.7	4.51	7.61	−3.10	3.18	5	91.12
	M		4.92	0.08	0.66	2			4.99	−0.48	0.59	2
Biomass	U	0.58	3.82	−3.24	3.26	24	94.5	0.27	1.78	−1.51	1.65	37	92.89
	M		0.66	−0.08	0.34	14			0.23	0.03	0.12	1
Birdwatching	U	9,741.90	5,574.00	4,167.90	5,050.19	105,490	95.2	5,804.85	8,906.44	−3,101	4,288.49	139,083	77.16
	M		9,551.06	190.84	2,596.47	13,735			4,922.25	882.60	1,131.62	4,904
Erosion	U	0.52	0.60	−0.07	0.71	69	36.96	0.47	0.54	−0.07	0.78	69	−9.32
	M		0.46	0.07	0.31	5			0.61	−0.14	0.14	9
Pollination	U	0.06	51.36	−51.30	68.30	19,837	99.98	0.01	104.01	−104.00	143.53	19,838	99.99
	M		0.05	0.01	0.05	197			0.00	0.01	0.02	2
Prematching	χ2 = 3,028***; df = 11; P < 0.001; n controls = 100,000	χ2 = 269***; df = 11; P < 0.001; n controls = 100,000
Postmatching	χ2 = 7.83; df = 11; P = 0.728; n controls = 1,000	χ2 = 12.7; df = 11; P = 0.313; n controls = 192

^***P < 0.001.

3 RESULTS

3.1 Which species and habitats do SCBs conserve?

California's 59 SCBs approved prior to 2011 differed in size (2–2,450 ha; mean: 240 ha; total: 16,350 ha) and land cover (natural: 12,490 ha; agricultural: 2,000 ha; developed: 1,010 ha; Figure ). SCBs lost 37 ha, but gained 17 ha, of natural cover between 2001 and 2011. All SCBs listed at least one species or habitat target (Supporting Information Table S2). Sixty‐six SCBs (95%) listed species targets. Species distribution models were available for seven (all vertebrates) of the 44 listed species, which were targeted by 29 SCBs (42%). All but one of these SCBs coincided with their species’ modeled distribution. The SCBs that did not coincide targeted the blunt‐nosed leopard lizard (Gambelia sila), which has a restricted range. Additionally, 46 SCBs (67%) listed habitat targets (34 habitat types in total). Most targeted vernal pools (18 SCBs) or wetlands (15 SCBs); 11 SCBs (24%) with listed habitat targets did not coincide with known distributions of these habitats.

Land cover in 2001 within California's SCBs, ranked by their proportion of vegetation. Top panel shows land cover area, bottom panel shows percent of land cover within each SCB. Figure shows 69 SCBs; but we analyse those approved prior to 2011 (59 SCBs) and 2001 (24 SCBs)

3.2 What explains the distribution of SCBs across California?

SCBs were nonrandomly distributed across the landscape, according to variables in Table , all of which were controlled for by matching (see next paragraph). SCBs were established in areas of high human disturbance, low elevation, near narrow forest edges, low canopy cover, and on flat slopes. SCBs were also located in areas that stored less carbon, retained less sediment, contained less pollinator habitat, and had higher demand for bird watching. Differences held for SCBs established pre‐2001 (χ² = 3,530; P < 0.001; Supporting Information Table S3A) and when SCB edges were excluded (χ² = 2,948; P < 0.001; Supporting Information Table S3A).

Characteristics of SCBs established pre‐2011 were statistically similar to matched controls (losses: χ² = 7.83; P = 0.728; gains: χ² = 12.7; P = 0.313; Table ) and this result held when examining SCBs established pre‐2001 (χ² = 16.4; P = 0.125; Supporting Information Table S3A). Some differences remained when excluding SCB edges (χ² = 59.1; P = 1.35 × 10^–08; Supporting Information Table S3A) and thus we do not report results for this analysis. Results held when using alternative matching methods (Supporting Information Table S3B–D).

3.3 To what extent do SCBs affect habitat losses and gains?

SCBs had a significant effect on habitat loss (Table ; P < 0.05). The rate of habitat loss within SCBs established pre‐2011 was significantly less than the rate observed in matched controls. In total, SCBs averted 6.3% of their natural land cover from being lost between 2001 and 2011. However, excluding regulated losses (i.e., to developed land) from calculations reduced their apparent effects. In this case, SCBs averted only 0.5% of their natural land cover (62.5 ha; 21–228 ha) from being lost between 2001 and 2011 (Figure ). SCBs established pre‐2001 averted 0.1% more unregulated loss than all SCBs established pre‐2011 (Figure ; Supporting Information Table S3). SCBs also had a significant effect on habitat gains; the rate of habitat gain was significantly less than that observed in matched controls (Table ); 45% of developed or agricultural land within SCBs would have otherwise transitioned to natural cover, amounting to 1,424 ha (743–2,075 ha) between 2001 and 2011 (Figure ). Results were similar for alternative matching methods (Supporting Information Table S3).

Impact of Species Conservation Banks’ (SCBs) on habitat losses and gains (Diff in mean), reported as the proportion of natural cover that would have otherwise been lost or gained, respectively. For natural cover losses, we show subset results to all losses and unregulated loss

	Natural cover losses	Natural cover gains
Treatment	Clearing component	Diff in mean	CI lower	CI upper	t‐statistic	Diff in mean	CI lower	CI upper	t‐Statistic
SCBs established pre‐2011 (59 SCBs)	All	−0.063	−0.093	−0.032	−4.0728***	−0.469	−0.690	−0.247	−4.177***
	Unreg.	−0.005	−0.018	−0.002	−2.3624*
SCBs established pre‐2001 (24 SCBs)	All	−0.096	−0.128	−0.063	−5.7467***
	Unreg.	−0.006	−0.011	−0.001	−2.4556*

Significance codes: ^***P < 0.001; ^*P < 0.05.

Effect of SCBs on natural land cover, shown as the area (ha) of change averted between 2001 and 2011. Error bars show upper and lower confidence intervals for mean differences (Table ) multiplied by area natural cover/agricultural/developed land within SCBs in 2001

4 DISCUSSION

The 59 SCBs established pre‐2011 in California conserved 15,350 ha of land and a range of species and habitats. We found SCBs averted some habitat loss, but also apparently negatively affected rates of habitat gain. We discuss consequences for achieving compensation objectives, identify factors limiting comprehensive assessments, and recommend ways to strengthen policy outcomes.

SCBs coincide with modeled distributions of target species (Figure c; Supporting Information Table S2); however, spatial coincidence does not demonstrate positive effects on biodiversity. Such assessments require data on biodiversity trajectories—that is, species losses and gains across California—which were unsurprisingly unavailable. Instead, we assessed effects on habitat extent, indicated by natural land cover, assuming habitat within SCBs was appropriate for targeted species and that treatments and matched controls contained similar habitat. These assumptions were reasonable given that our covariates were used to model species distributions (USGS, ) and that most SCBs calculate credits from habitat area alone (Gamarra & Toombs, ). If these assumptions were untrue, our results may be biased, particularly if small habitat gains yielded large benefits for species. However, even in these cases, examining effects on natural cover still illustrates consequences of SCBs for other statewide conservation priorities, such as carbon storage (Anderson, Field, & Mach, ).

Statistical matching controlled for the nonrandom allocation of SCBs across California. Failing to account for this bias would have suggested SCBs cause (rather than avert) losses, because SCBs were established in areas of development pressure (Table ) with high background rates of loss. This finding is similar to other privately protected areas (Baldwin and Leonard ), but differs from publicly funded protected areas, which often have low agricultural or development value (Andam et al., ).

Most SCBs conserved existing habitat (Figure ), averting habitat losses equivalent to 0.4% of natural land cover within SCBs and less than 0.01% of regulated losses across California (881,428 ha) during 2001–2011 (Table ; Figure ). However, these estimates were influenced by how counterfactual scenarios considered regulated losses. Considering all losses suggested SCBs averted 788 ha of loss; excluding regulated losses reduced averted losses to 62.5 ha. These estimates illustrate the huge influence of counterfactual assumptions on perceived policy effectiveness.

We expected SCBs to elevate rates of habitat gains because some had restoration goals (Figure ), but we found the opposite, suggesting SCBs also avert 35 times more habitat gains than losses (Table ; Figure ). Several factors may explain this counterintuitive finding. First, within SCBs, preserving agricultural land may benefit some target species (e.g., Swainson's Hawk, Giant Garter Snake; Supporting Information Table S2), or be less expensive than returning it to a natural state (Merenlender, Huntsinger, Guthey, & Fairfax, ; Rissman et al., ). Second, outside SCBs, habitat gains may be in response to other habitat restoration programs (NRCS, ), and because SCBs must create additional gains, they would not qualify for them. Whether SCBs truly avert the gains that these programs would have made depends on whether programs generate a capped amount of restoration (e.g., a fixed budget) or not (e.g., tax incentives). If the former dominates, our results may reflect a redistribution of restoration among properties. If the latter, averted gains may be quite real, revealing a perverse outcome. Further research should examine factors explaining averted gains and how they relate to species targeted by SCBs, and whether results hold when using information on species gains achieved by restoration.

Regulators have two options to mitigate impacts of development on biodiversity: deny development that cannot remove impacts through avoidance and minimization measures or improve compensation so that SCBs counterbalance impacts without causing additional losses. To improve practice, SCBs must have an explicit goal against which to assess effectiveness. Goals currently differ among SCBs and are sometimes absent entirely (Gamarra & Toombs, ). A consistent metric is required to calculate additional credits generated by SCBs, rather than using habitat area alone. Generating gains through habitat creation rather than preservation (such as the UK's proposed Net Gain policy; DEFRA, ) would help; however, these policies must avoid displacing other restoration programs that incentivize restoration. Finally, detailed management and monitoring plans, with feedback to a central authority, are required so that management can respond to project failure.

At the program scale, compensation can be evaluated using statistical matching methods, so long as sufficient information exists. In California, data gaps limited our conclusions. To overcome these limitations, we recommend regulators: (1) Map compensation sites. We mapped SCBs using point locations and the map buffer tool in ArcGIS, and removed some uncertainties by excluding edges; however, an accurate map of SCBs would eliminate uncertainty entirely. (2) Record biodiversity transactions. Data are available on SCBs, but information on regulated losses is missing. (3) Monitor threatened species inside and outside SCBs, to obtain baseline trajectories. Gaps in data and policy goals limit policy evaluations and prevent adaptive management. Compensation programs must be designed with clear, testable objectives to permit monitoring, evaluation, and adaptive management.

ACKNOWLEDGMENTS

Discussions with Carla Archibald, Becky Chaplin‐Kramer, Megan Evans, Maria Jose Carreras Gamarra, Alex Pfaff, Taylor Ricketts, and Ted Toombs greatly improved our manuscript. All data are publicly available, as referenced throughout.