Is habitat restoration targeting relevant ecological needs for endangered species? Using Pacific Salmon as a case study

Abstract

Conservation and recovery plans for endangered species around the world, including the US Endangered Species Act (ESA), rely on habitat assessments for data, conclusions and planning of short and long‐term management strategies. In the Pacific Northwest of the United States, hundreds of millions of dollars ($US) per year are spent on thousands of restoration projects across the extent of ESA‐listed Pacific salmon—often without clearly connecting restoration actions to ecosystem and population needs. Numerous decentralized administrative units select and fund projects based on agency/organization needs or availability of funds with little or no centralized planning nor post‐project monitoring. The need therefore arises for metrics to identify whether ecosystem and species level restoration needs are being met by the assemblage of implemented projects. We reviewed habitat assessments and recovery plans to identify ecological needs and statistically compared these to the distribution of co‐located restoration projects. We deployed two metrics at scales ranging from the sub‐watershed to ESA listing units; one describes the unit scale match/mismatch between projects and ecological concerns, the other correlates ecological need with need treated by projects across units. Populations with more identified ecological concerns contained more restoration effort, but the frequency of ecological concerns in recovery plans did not correlate with their frequency as restoration targets. Instead, restoration projects were strongly biased towards less expensive types. Many ESA‐listed salmon populations (78%) had a good match between need and action noted in their recovery plan, but fewer (31%) matched at the smaller sub‐watershed scale. Further, a majority of sub‐watersheds contained a suite of projects that matched ecological concerns no better, and often worse, than a random pick of all project types. These results suggest considerable room for gains in restoration funding and placement even in the absence of centralized planning. This analytical approach can be applied to any species for which habitat management is a principle tactic, and in particular can help improve efficiencies in matching identified needs with explicit management actions.

Introduction

Around the world, restoration and protection of habitat form the foundation of long‐term conservation strategies for threatened species recovery. In many countries, protected species laws require recovery and management plans to incorporate habitat restoration (Bottrill et al. 2011). These plans, in turn, provide the scientific framework to inform restoration project design and prioritization (Bernhardt et al. 2007, Miller and Hobbs 2007, Beechie et al. 2008). Moreover, aquatic ecosystems and species worldwide generally face more threats than terrestrial ecosystems (Richter et al. 1997, Magurran 2009), thus restoration actions more commonly target freshwater and estuarine habitats (Bernhardt et al. 2005, Roni et al. 2008). However, species or watershed plans do not always guide aquatic restoration. A survey of river restoration practitioners in the United States reported that although one third of habitat restoration projects could be considered part of a larger plan, only 16% of projects were initiated within the specific context of a watershed management plan (Bernhardt et al. 2007). Such fragmentation of restoration effort, in which individual projects proceed independent of a centralized plan, may not effectively recover species with ranges larger than that of a single project.

This observed fragmentation, or decentralized decision making with respect to restoration effort, is rationalized as having lower transaction costs, increased equity for local agents and better matching local knowledge with decision making (e.g., Berkes et al. 1989, Sewell 1996, Sharma 2003, Berkes 2007). However, fragmented restoration decision making has also been criticized (e.g., Prud'Homme 1995, Mody 2004) for re‐enforcing existing dis‐equity and inefficiencies (Lebel et al. 2004, Lane and Corbett 2005). In a striking case, resource management in the U.S.'s Columbia River Basin has achieved a sufficient level of complexity that its water management policy has been seen to move to both greater and lesser centralization simultaneously (Wandschneider 1984). For example, the overwhelming majority of management funding for habitat restoration and protection in the Columbia River Basin comes from a small number of Federal sources, while over 65 local, regional and tribal agents coordinate, direct and prioritize that funding (G.A.O. 2002). In this study we will look at the efficiency of habitat restoration decisions in efforts to recover endangered Pacific salmon in the Northwest United States given the complex management structure of this region.

Since 1991, 18 Evolutionarily Significant Units (ESU; Waples 1991) and Distinct Population Segments (DPS; O. mykiss) of anadromous Pacific salmonids (Oncorhynchus spp.) have been listed under the Endangered Species Act (ESA) in Washington, Oregon and Idaho covering a footprint of over 290,000 km², an area larger than the state of Oregon. As with most endangered species, the ESA listings (16 U.S.C. §§ 1531–1544) for Pacific salmon identified habitat degradation as a cause of decline. Thus habitat restoration has been widely applied to improve spawning and rearing habitat in hopes of increasing salmon numbers to meet recovery goals (National Research Council et al. 1996). The Pacific Northwest now contains one of the highest densities of freshwater restoration projects in the U.S. (Bernhardt et al. 2005), and is essentially the largest freshwater restoration effort ever undertaken on behalf of an endangered species with billions of dollars spent to date (G.A.O. 2002, NMF. 2013). Despite this extraordinary financial investment, projects to restore freshwater and estuarine habitat are assessed for effectiveness at the individual project level rather than in the context of the greater recovery efforts, if post‐project monitoring is funded at all (Bash and Ryan 2002). Many projects are funded piece‐meal by various administrative units based on the priorities of the funding agency, availability of funds, ease of implementation and buy‐in by landowners and stakeholders rather than needs outlined in management plans (Miller and Hobbs 2007, Kondolf et al. 2008, Katz 2009). Indeed, in 2002 the U.S. Government Accountability office estimated that 66% of non‐mainstem and non‐hatchery federal expenditures (i.e., the fraction available for habitat restoration and protection) were distributed directly to local, non‐Federal entities for salmon recovery management (US $537.2M from 1997 to 2002; G.A.O. 2002).

In light of the lack of coordination across the restoration enterprise (Bernhardt et al. 2007, Kondolf et al. 2008), there is a critical need to evaluate how well habitat restoration actions match the impaired habitat conditions, especially when targeting a threatened or endangered species like Pacific Salmon. Such evaluations establish accountability for the use of public and private funds and establish reasonable performance expectations for the considerable efforts and resources applied to restoration (G.A.O. 2002, Katz et al. 2007). In recent years databases have been developed to inventory aquatic restoration actions (Bernhardt et al. 2005, Katz et al. 2007); unfortunately, monitoring‐derived data to characterize the impairment of aquatic habitat conditions is not similarly forthcoming (Rumps et al. 2007, O'Donnell and Galat 2008, Hamm 2012). Thus, assessing the match between actions needed across the landscape and actions completed must resort to more creative approaches.

Acknowledging that recovery planning could be improved by a better linkage between basic ecology and management actions (Clark et al. 2002, Palmer 2009, Dickens and Suding 2013), and that restoration success at species relevant spatial scales requires better coordinated approaches (Paulsen and Fisher 2005, Kondolf et al. 2008), here we develop methods to assess how restoration expenditures for Pacific Salmon align with ecological needs. We have adopted two approaches: relating expressed habitat concerns to completed restoration with a presence/absence metric on a unit scale, and correlations of need and project frequencies across units. We assay expressed ecological need by surveying management plans (i.e., data sets) encompassing various spatial scales relevant to species management, combine this information with data on the restoration actions in those same spatial units, and ask (1) does restoration address ecological concerns at the sub‐watershed scale within a salmon population, and (2) does restoration address ecological concerns at the scale of a salmon population within an Endangered Species Act listing unit (ESU/ DPS)? This method provides an objective way to retrospectively assess types of restoration projects and their placement on landscapes, and evaluate the appropriateness of proposed projects for a given species or population based on documented ecological concerns.

Methods

Identifying ecological concerns

We assembled ecological concern data by surveying documents from two hierarchical programs addressing watershed management and recovery planning for statements of ecological need. While these programs have large conceptual overlap, they use different semantics, cover different geographic footprints, and use different source data. As a consequence, we had to standardize the datasets before synthesizing; each data set is described in turn.

We reviewed subbasin plans for the Columbia Cascade Ecoprovince (http://www.nwcouncil.org/fw/subbasinplanning), located in North Central Washington State (Fig. 1). The 2005 subbasin plans prioritize habitat restoration actions and provide guidance for funding restoration as a part of the Northwest Power and Conservation Council's Columbia River Basin Fish and Wildlife Program, enacted to mitigate the impacts of hydropower and to fulfill legal obligations under the Endangered Species Act and Clean Water Act (Northwest Power and Conservation Council 2005). The Columbia Cascade Ecoprovince encompasses six subbasins: Chelan, Entiat, Methow, Okanogan, Upper Middle Columbia and Wenatchee and some smaller watersheds that drain directly to the Columbia River (Fig. 1). The subbasin plans divide the Columbia Cascade into 86 smaller spatial units, what we refer to as sub‐watershed assessment units. This region historically produced healthy runs of anadromous salmonids (e.g., Mullan et al. 1992) but the Upper Columbia Chinook ESU and Upper Columbia Steelhead DPS now are listed as Endangered.

ESA listed ESU/DPSs of Pacific Salmon cover large portions of the states of Washington, Oregon, and Idaho, outlined in black. Populations with completed recovery plans are shown gray. The Columbia Cascade Ecoprovince (patterned) is within the state of Washington and overlaps with the Upper Columbia ESU/DPSs.

As the subbasin plans are only available for the Columbia River Basin, which excludes a number of ESA listed salmon ESU/DPSs, we also evaluated NOAA Fisheries final or public draft recovery plans for Pacific salmon in Washington, Oregon, and Idaho (http://www.nmfs.noaa.gov/pr/recovery/plans.htm). The recovery plans cover 219 populations nested within 16 ESU/DPSs and were developed from 2000 to the present (Fig. 1). The Oregon Coast Coho ESU and Puget Sound Steelhead DPS recovery plans are currently in development so could not be used in this analysis. Four additional populations were not evaluated in the recovery plan for a given ESU/DPS, and so were excluded from our analyses (Upper Willamette Steelhead–West Side Tributaries, Mid‐Columbia Steelhead–Willow Creek, Upper Columbia Steelhead–Crab Creek, Puget Sound Chinook–Skykomish). We also excluded extirpated populations if they were not covered in their respective recovery plans (Snake Spring Summer Chinook–Clearwater populations and Snake River Steelhead–Clearwater and Hells Canyon populations) (Fig. 1).

For each spatial unit described in the plans we polled projects from the Pacific Northwest Salmon Habitat Project Database (Katz et al. 2007, NMFS 2014). All restoration projects in the PHSHP database are attributed with a project type and subtype as well as geo‐referenced which enabled sampling at any scale using GIS. We defined a project as a unique location and project subtype combination. We confined our queries to projects completed in the 20 years since the first Pacific Salmon ESA listing (1992–2011, 36,895 projects) and to actions involving physical changes to freshwater or estuarine habitat. We evaluated completed projects recognizing that in total, it can take up to 10 years from the application for funding and permits through project implementation, completion and final reporting before inclusion in a database (Katz et al. 2007). While the PNSHP restoration database contains the vast majority of known projects it is not considered a complete census (Barnas and Katz 2010).

Semantic synthesis

Among the challenges to the current work, the subbasin and recovery plans lack not only a prioritization of habitat impairment, but lack the common language, definitions and spatial units to characterize such impairments (Hamm 2012). Prior work to address the lack of common semantics produced a standardized dictionary of ecological concerns, which classifies degraded salmon habitat in the Pacific Northwest (Hamm 2012) and has been incorporated into the most recent salmon recovery plans. An ecological concern was defined as: changes to the ecological conditions essential for maintaining the long‐term viability of a given salmonid population, causing mortality, injury, reduced health or reduced reproduction (Hamm 2012) and consists of 10 types and 34 nested subtypes. Each of the source documents in this study were reviewed for all references to ecological concerns as defined by Hamm (2012) (e.g., habitat limiting factors, impaired habitat etc.) within each spatial unit.

Sub‐watersheds generally followed the watershed boundaries of approximated fifth or sixth field United States Geological Survey (USGS) Hydrologic Cataloging Unit Codes (HUCs), or in some cases an ad hoc mixture of the two. In general the sub‐watersheds did not map one‐to‐one with the population units described in Pacific salmon recovery plans. The spatial footprint of salmon populations within an ESU/DPS range in size from 40 to 12,000 km², or approximately a seventh field HUC to a third field HUC.

Crosswalk development

To relate restoration project types to the ecological concerns they treat, we developed crosswalks defined as “a table that maps the relationships and equivalencies between two or more metadata schemes”(Dublin Core Metadata Initiative: glossary; http://dublincore.org/documents/usageguide/glossary.shtml), using the ecological concerns (Hamm 2012) and PNSHP data dictionaries (Katz et al. 2007). Both the ecological concerns data dictionary and the PNSHP database are structured with general types and nested subtypes allowing us to create detailed crosswalks. Ideally each project subtype would map to a single ecological concern subtype; in reality, a restoration project type or subtype may address many ecological needs, e.g., a riparian planting project may treat bank stabilization, sedimentation, provide riparian cover, or all three. In response to this potential ambiguity, we created separate crosswalks to obtain upper (Broad) and lower (Narrow) bounds to relate restoration projects to ecological need. The bounding crosswalks then allowed us to create a third “Intermediate” crosswalk.

In the “Broad” crosswalk (Appendix A: Tables A1–A4), we attributed projects to ecological concerns even if the likelihood of success was highly optimistic, contingent on large‐scale implementation or on long‐term maintenance for project effectiveness. For example, “riparian planting” addresses riparian condition as well as decreases stream temperatures due to shading, buffers sediment runoff, enhances nutrient subsidy, reduces bank erosion, augments the recruitment of large woody debris, and increases stream complexity due to mature trees falling into the river. For the “Narrow” crosswalk (Appendix B: Tables B1–B4), we attributed project types with addressing only the most probable goal. In this case, a “riparian planting” project is credited with addressing only “riparian conditions”.

The “Intermediate” crosswalk attributed projects with all concerns suggested by the literature unless cause and effect are separated by many steps, require coordinated implementation across large spatial scales or require time scales of decades for success (Appendix C: Tables C1–C4). For example in the “Intermediate” crosswalk, riparian planting addresses water temperature (through future shading), decreases sediment runoff (by stabilizing banks and slowing water runoff) but does not receive credit for altering channel form or increasing instream habitat complexity. These channel changes would result only if riparian plantings matured and fell into the river in sufficient quantities, a process that takes decades (Beechie et al. 2010). We used the “Intermediate” crosswalk to assign ecological concerns to restoration projects for our analyses.

Analysis

We initially surveyed subbasin and restoration plans for prioritization of ecological needs screening for terms such as ‘primary' or ‘major' within a sub‐watershed or salmon population. This would have provided the data to correlate the highest priority needs with the most numerous or intensive restoration type(s). This level of relative prioritization was not available across all documents. Secondarily, we looked for spatially explicit information on ecological need (e.g., lat/long coordinates for needed barrier removals). This too was lacking. In response, we developed a metric of match/mismatch based simply on presence or absence. For each spatial assessment unit, using Geographic Information Systems (GIS) we compared restoration projects from PNSHP to ecological concerns gathered from subbasin plans (sub‐watershed) or recovery plans (population). When an ecological concern was treated by one or more projects based on the intermediate crosswalk it was considered a match.

We calculated the percentage of projects matching the ecological concerns (Appendix D: Table D1), and the percentage of ecological concerns matched by at least one project for each unit resulting in the Salmon Habitat Assessment and Project Evaluator metric (SHAPE): $$$$

The metric ranges from −1, if projects failed to match any ecological concerns, to 1, if all projects were appropriate and all ecological concerns were addressed by at least one project. A zero is reported if there are no projects, whether or not ecological concerns are present. If no ecological concerns are identified but there are projects, the SHAPE score is −1. We considered a “good” SHAPE score anything above the average of all (sub‐watershed and population combined) SHAPE scores (0.56) and a “poor” score anything below the lower quartile for all scores (0.40).

To evaluate SHAPE metric performance, we re‐sampled with replacement the projects found in each sub‐watershed from all possible project types, compared these project types to the ecological concerns using the crosswalk, and re‐calculated the SHAPE. We ran this resampling 5000 times to test if projects completed in each spatial unit better matched ecological concerns than project types selected at random. We then compared the mean permuted SHAPE score to the measured SHAPE score.

The metric values are continuous, but the ratios are bounded and based on presence/absence data. Therefore all statistical comparisons used non‐parametric techniques such as Kendall's rank correlations. All statistics were estimated using the R statistical computing environment (R Core Team 2013).

Results

ESA listing units and populations within listing units

Ecological concern types were not equally likely to be documented in recovery plans across the salmon recovery domain; there were six relatively common and three relatively rare types (Fig. 2A). The six most common types made up almost 85% of the total, and were problems with: Peripheral and Transitional Habitats, Channel Structure and Form, Water Quantity, Riparian Condition, Water Quality and Sediment Conditions (see Hamm, 2012 for ecological concerns definitions). The aggregate distribution of restoration projects was distinct from ecological need in being dominated by a couple project types with the rest being relatively rare (Fig. 2A). Projects that targeted Water Quality and Sediment Conditions together represented over half of all projects (52%). Many restoration types address Water Quality and Sediment Condition (Appendix C: Tables C1–C4) so the allocation of projects to those categories may in part be a product of the crosswalk. The least common projects targeted Food Limitation and Injury and Mortality (5% of total), these categories are also the least commonly expressed ecological needs (Fig. 2A). Over the first two decades of salmon recovery we found no change in the project types utilized with the same types making up the majority of projects completed (1990s vs 2000s, Kendall's τ = 0.88, p = 0.00001).

The frequency of expressed ecological concern (gray) and restoration to address that ecological concern (black) for (A) the 219 salmonid populations with ecological concern data from recovery plans and (B) the 86 sub‐watersheds within the Columbia Cascade Ecoprovince subbasin plans.

The number of ecological concern types and subtypes in a salmon population varied from 0 to 17 with the majority of populations having 9–12 concerns. We found a wide range in the number of projects within a given population, 0–2577, though the majority of populations had project numbers in the hundreds rather than thousands (median = 138). We see a positive relationship between the number of ecological concerns and the number of restoration projects across salmon populations (r² = 0.20, p = 2.8·10⁻¹²; Fig. 3). While this result supports the inference that more ecological concerns result in more restoration effort, it does not speak to the appropriateness of the restoration or other potential factors. We found a number of stated ecological concern subtypes untreated by restoration in 10 or more populations: Predation (Injury and Mortality), Instream Structure and Bed and Channel Form (Channel Structure and Form), Side Channel, Floodplain, and Nearshore (Peripheral and Transitional Habitats), Decreased Water Quantity (Water Quantity), and Large Woody Debris Recruitment (Riparian). When we evaluate appropriateness at the individual population level using the SHAPE metric, scores covered the range from −1.0 to 1.0. The distribution skewed toward higher values (mean = 0.75, median = 0.87) with 79% of populations (174/219) scoring in the “good” range (>0.56), while 27 fell into the “poor” range (<0.40) (Fig. 4A).

Plot of the log of total number of restoration projects as a function of number of ecological concerns within each salmon population (Fs = 54.92, df = 1,217, p = 2.8E−12).

SHAPE score by (A) assessment unit type binned as the percentage of all assessment units: Columbia Cascade sub‐watersheds (light grey; n = 86), salmon populations (dark grey; n = 219); ESU/DPSs (black; n = 16) and SHAPE score (B) as a function of assessment unit size (km2) for sub‐watersheds and populations. The inset includes scores for ESU/DPSs. We considered a “good” SHAPE score anything above the average, 0.56, and a “poor” score anything below the lower quartile for all scores, 0.40.

The Columbia Cascade and sub‐watershed units

In the Columbia Cascade Ecoprovince, ‘Channel Structure and Form' was the most frequently documented ecological concern category (Fig. 2B) although the top six concern categories were similarly frequent (12.5–17.5% of concerns): Habitat Quantity, Water Quantity, Water Quality, Peripheral and Transitional Habitats, and Sediment Conditions (Fig. 2B). The Columbia Cascade Ecoprovince contains a few relatively frequent project types and a larger number of rarer types similar to the distribution of restoration projects across the salmon recovery domain (Fig. 2). Projects addressing Water Quality were the most numerous and constituted 24% of all projects in the Columbia Cascade. Water Quality along with Riparian Condition, Sediment Condition and Water Quantity, represented almost 75% of all projects (Fig. 2B).

Over both the entire salmon recovery domain and the Columbia Cascade Ecoprovince, we found a poor rank‐order correlation between ecological concerns and restoration projects (Kendall's τ = 0.22, p = 0.46 salmon populations; τ = 0.25, p = 0.40 Columbia Cascade sub‐watersheds). Interestingly, we also found a poor rank‐order correlation between the expressed ecological concerns in the Columbia Cascade Ecoprovince compared to the larger salmon recovery domain (τ = 0.30, p = 0.29). Despite this, the frequencies of restoration project types were highly correlated between the Columbia Cascade and the salmon recovery domain (τ = 0.88, p = 0.001).

The 86 sub‐watersheds within the Columbia Cascade Ecoprovince contained between 0 and 12 ecological concerns (median = 3) and 0 to 188 restoration projects (median = 6). Of the Columbia Cascade sub‐watersheds, 7% had ecological concerns but no projects, and an additional 47% of sub‐watersheds had one or more ecological concerns not matched by a project. Of sub‐watershed units, 31% (27/86) scored in the “good range” (SHAPE > 0.56), compared to 79% of salmon populations (Fig. 4A). Over 86% of populations and 43% of sub‐watersheds had a SHAPE score that was above “poor” (SHAPE < 0.40; Fig. 4A). Unlike the salmon population SHAPE scores however, the Columbia Cascade distribution has two distinct modes; 14 sub‐watersheds scored 0.0, 13 of which had no restoration projects, and 24 sub‐watersheds scored a −1.0, with 67.0% (56/86) scoring “poorly” (mean = 0.06, median = 0.08).

When ecological concerns and project numbers were held constant, but project type chosen at random, 10 sub‐watersheds (14%) scored the same between the measured SHAPE score and the re‐sampled SHAPE score, 53% of sub‐watersheds scored higher and 33% lower. The mean difference was 0.19 higher for re‐sampled SHAPE scores over measured SHAPE scores.

Effect of scale

The assessment units in this study varied three orders of magnitude, yet are all management units directly applicable to restoration planning for Pacific Salmon recovery. Smaller units averaged more ecological concerns and restoration projects per unit area (<500 km² unit: 0.9 EC/km², 2.1 projects/km²; >5000 km² unit: 0.002 EC/km², 0.06projects/km²), but larger units on average had more total restoration. Thus, we evaluated how the match and mismatch of restoration effort and ecological concern interacted with size of unit.

Plotting the SHAPE scores for all assessment units reveals a “dust bunny” distribution (McCune et al. 2002), where data distribute along the axes (Fig. 4B). We found the greatest range of SHAPE scores in the smallest units and the greatest range in assessment unit size at the highest SHAPE scores. The ESU/DPS's make up almost all of the values from 12,000 to 86,000 km² (Fig. 4B inset), so we removed the ESU/DPSs to evaluate the remaining data from smaller units (Fig. 4B). The Columbia Cascade Ecoprovince SHAPE score modes of −1.0 and 0.0 (Fig. 4B) are almost entirely from very small sub‐watersheds (<500 km²), although numerous small sub‐watersheds also received higher SHAPE scores. However, no large assessment units (>5000 km², n = 22) scored poorly and all but one scored in the ‘good' range.

We further evaluated the manner in which the SHAPE score responds to scale by looking at how the components of the SHAPE score are affected by the size of the assessment unit. When salmon populations are binned into five size classes, SHAPE scores start above 0 and increase when going from small to large units (Fig. 5A). Over this range of unit size, the percent of ecological concerns (arcsin transformed) addressed increases with unit size and shows a decreasing trend in the variance across unit sizes (Fig. 5B). While the percent of restoration actions not addressing a need (arcsin transformed) is about the same for all population sizes, the variance decreases markedly for larger populations (Fig. 5C). This supports the idea that the SHAPE score variance has a scale‐dependence that may bias toward higher values in larger units.

Box and whisker plots as a function of five assessment unit size categories for (A) SHAPE Score, (B) the ratio of expressed ecological concerns addressed by restoration (arcsin transformed) to the total number of expressed ecological concerns within each assessment unit (i.e., the left side of Eq. 1) and (C) the ratio of restoration projects that do not match an ecological need (arcsin transformed) to the total number of restoration projects within each assessment unit (i.e., 1 minus the right side of Eq. 1). In all panels of the figure, the median values are plotted as dots.

Where salmon species and assessment units overlap, unit size and species specific ecological concerns can lead to differing SHAPE scores (Fig. 6A–E). For example, much of central Idaho has a “good” SHAPE score (>0.56) with respect to steelhead (O. mykiss), but “poor” scores (<0.40) for Chinook populations (O. tshawytscha). All species except sockeye salmon (O. nerka) had an average SHAPE score above 0.56. Despite this, 27/219 populations still fell between −1 and 0.04, indicating a poor relationship between actions completed and expressed habitat need (Fig. 6A–D).

Maps of SHAPE score for each population of ESA‐listed Pacific salmon (A) Chinook salmon populations (B) Chum and Sockeye salmon populations (C) Coho salmon populations (D) Steelhead trout populations and for the (E) Columbia Cascade Ecoprovince sub‐watersheds. In each case, poor SHAPE scores (<0.40) are colored in shades of red, good SHAPE scores (>0.56) are green and blue, and intermediate scores are yellow or orange.

Effect of cost

Water Quality projects were the most expensive, averaging $2.3M while Sediment Reduction and Upland Management projects were cheapest, averaging $55,000 and $85,000 respectively. Projects that matched an ecological concern had similar costs to those of unmatched projects with one exception; Water Quality projects averaged almost $200K more per project in assessment units with a water quality concern (Fig. 7). Water Quality and Estuary/Nearshore projects were both the most expensive and the least numerous, while the inverse was true of Riparian and Sediment Reduction projects. Thus, we found a negative association between cost and abundance of project types.

Average cost for Pacific Northwest Salmon Habitat Restoration Database projects by type found within ESU/DPSs. The cost for restoration projects that matched an ecological need within a population (black) is distinguished from the cost for those projects of the same type that did not address an expressed need (gray). Small squares show total project number for each type.

Effect of crosswalk

The sensitivity of our results depends on the choice of crosswalk. Both the percentage of projects treating a concern (τ = 0.67, p < 0.001) and the percent of concerns addressed (τ = 0.62, p < 0.001) differed significantly between the two bounding crosswalks. Predictably, the Intermediate crosswalk results fell in the middle with an assessment unit median of 74% (39% Narrow, 80% Broad) of Columbia Cascade projects matching an expressed ecological concern, and 50% (35% Narrow, 78% Broad) of sub‐watersheds with ecological concerns treated by restoration. The median SHAPE score for the Columbia Cascade sub‐watersheds varied between 0.0 (Narrow), .07 (Intermediate) and .28 (Broad).

Discussion

Do habitat restoration projects address the ecological needs identified within sub‐watersheds or salmon populations for ESA‐listed Pacific salmon? Our results show variability in restoration decision making across the extent of salmon recovery and suggest room for tangible gains in restoration efficiency. Given the increasing availability of restoration project spatial data, the lack of consistently reported and prioritized ecological needs is the principle limit in our ability to evaluate project placement. That these data were not forthcoming is perhaps surprising as the Endangered Species Act offers guidance in the development of recovery plans that includes prioritizing actions taken in recovery (Stanford Environmental Law Society 2001). Importantly however, the ESA does not require explicit prioritization of ecological concerns, nor an explicit connection between need and action. We were left with using two approaches to the question, operating at different scales, with different abilities to evaluate decision making, and supporting different inferences about the appropriateness of restoration actions. Being a retrospective look at the question, both approaches are ultimately constrained by the properties of the available data and represent an attempt to extract as much information as possible from a limited resource.

Aggregating the data from all assessment units, we found a weaker than expected, and non‐significant correlation between project type and frequency of ecological concern. This suggests a lack of connection between ecological need and the use of restoration across the spatial extent of either management process (Fig. 2). However, looking both at individual SHAPE scores and accumulating SHAPE scores across the region suggests that restoration types are often appropriately placed despite fragmentation of restoration efforts. The majority of populations and close to half of the sub‐watersheds had a SHAPE score that was above “poor” (Figs. 4A, 6).

Mismatches between stated need and action do exist. Almost half of the Columbia Cascade sub‐watersheds had one or more ecological concerns not matched by a project. When we randomly resampled project types for Columbia Cascade sub‐watersheds, SHAPE scores for over half of sub‐watersheds were higher than when we calculated SHAPE using real project data. This suggests a poor connection between habitat assessment and restoration decision making. Further, across the extent of ESA listed salmon in the Pacific Northwest, we found over 7000 projects that did not match an expressed ecological concern for a salmon population in spite of generally positive SHAPE scores. Thus a good SHAPE score does not mean all projects have been appropriately placed, but a poor SHAPE score identifies the areas that could be prioritized for further investigation and restoration implementation improvements.

Spatial scale and patterns

Whether aggregating SHAPE scores from assessment units or looking at single unit comparisons, scale drives some of the observed patterns in this analysis. Fig. 5 suggests the SHAPE metric is best suited to the HUC 4 to HUC 6 scale (i.e., <~2,000 km²), as this is where the dynamic range of the SHAPE score is the greatest and this scale aligns well with other conservation analyses. The HUC 5–6 is the “local” spatial scale used by other groups to determine limiting factors for Pacific Salmon in order to “review and prioritize restoration activities and guide future funding decisions” (Oregon Coast Coho Conservation Plan for the State of Oregon, March 16, 2007) and to calculate population‐status metrics such as the Conservation Success Index (Williams et al. 2007). Other studies have used the larger USGS HUC 3 and 4 units to assess watershed conservation value (Pinsky et al. 2009).

Populations with more ecological concerns also have more total restoration effort (Fig. 3). This suggests that net restoration effort is appropriately distributed based on ecological need, although the specific project types may not match the stated needs. This pattern may reflect the common view of restoration funders that the cumulative impact of multiple restoration projects will result in enhanced ecosystem function, and in turn, improve salmon population survival and abundance (NMFS 2000). Restoration effort alone may indeed have other benefits for salmon (Allen et al. 1997). For example, Paulsen and Fisher (2005) found that watersheds with more restoration effort correlated with higher juvenile Chinook parr to smolt survival. These findings do not discriminate between more effort being more effective in aggregate (efficiency neutral), and more effort generating more appropriate, and thus more effective project types by chance (efficiency decreasing).

Restoration projects often don't identify a target species, thus where species spatially overlap a project may address an ecological concern for one species, but not address the ecological needs of others. However, we found this had little impact on SHAPE scores. Where identical population boundaries cover more than one species or multiple run‐timings of a single species (71 populations, 27 comparisons), SHAPE scores varied among species/run‐timing just over half time, but by small amounts (<0.05) in most cases. The observed low impact in part motivated our consideration of multiple, overlapping populations. By conducting an analysis at the scales used to evaluate salmon recovery, the ESU/DPS and populations within, we have done a triage, which will allow more localized analyses targeted at the specific areas where they are most needed (e.g., areas with few or no projects matching one or more ecological concerns).

Other patterns in restoration usage

A suite of common project types appear to be implemented throughout the Western U.S. for Pacific Salmon regardless of habitat need, likely a result of decentralized decision making regarding restoration. We found no change in the project types utilized when comparing the first decade of salmon recovery (1990s) to the more data rich 2000s. The most common project types in this study mirror those identified by Katz et al. (2007), who found that sediment reduction, riparian planting and instream structures are both the least expensive project types and the most common based on the PNSHP database (Figs. 2, 7). Sediment reduction (via road repair), riparian stabilization and instream structures were also the most common project types in an analysis of the Russian River basin, CA (Christian‐Smith and Merenlender 2010). Further, the high correlation between frequency of project types in the Columbia Cascade sub‐watersheds and the salmon populations on the one hand, and the poor correlations between frequency of project type and ecological need on the other, reinforces the idea that there is a default suite of restoration actions. The differences in size, diversity and character of the spatial units covered in these planning processes and the fact that the ecological need frequencies were poorly correlated between the two planning efforts suggest that while the assessment authors were likely acting independently, the ultimate restoration types implemented were still similar.

Whatever the underlying process, the resulting pattern of commonly used restoration actions leaves some ecological concerns less likely to be treated and others perhaps over treated. Sediment reduction, riparian planting and fish passage were both the most common project types and the project types least likely to match a known concern, with 73%, 75%, 72% of projects matching stated needs respectively (Fig. 7). Fish passage was the third most common project type, and although a majority of fish passage projects did match a known habitat quantity concern, that still left 1492 completed fish passage projects in 74 populations without a stated fish passage issue. In these cases, project sponsors may have been relying on guidance that in the absence of detailed information, restoration types with a high success rate and quick response time, like barrier removal, should be employed first (Beechie et al. 2008).

In previous work (Katz et al. 2007) and the present study, project cost most strongly predicts the use of restoration types, suggesting cost drives decision making by funders and project sponsors. Ideally, funding agencies would direct restoration efforts at identified ecological concerns, given that most funders have stated goals of restoring ecosystem function, maintaining populations and adhering to relevant laws (Clean Water Act, ESA). That cost is a significant driver of decision making is not surprising however. Social constraints such as landownership, public acceptance, and funder priorities influence project type and placement (Halle 2007, Miller and Hobbs 2007, Kondolf et al. 2008, Christian‐Smith and Merenlender 2010).

Some restoration project types have functional connections with land use that constrain their utility across a diverse landscape. For example, fish screens keep fish out of surface water diversions (e.g., hydroelectric facilities, irrigation, municipal and industrial water withdrawal projects). Fish screens are also relatively expensive and custom fabricated so are unlikely to be utilized where not essential. We found that 99% of fish screening projects matched an ecological concern and only two populations that needed fish screening projects did not have at least one. Thus, at the salmon population scale fish screening projects are, for the most part, efficiently funded and placed.

Implications for species recovery

Pacific salmon cover an extremely large area, matched only by other highly‐migratory animals, or plants with large ranges. Pacific Salmon ranges overlap management jurisdictions that span municipal to international scales. The scale of restoration effort is equally as large, as the total number of completed projects in PNSHP doubled from around 7000 in the 1990s to over 14000 in the 2000s as a result of the Pacific Salmon ESA listings (NMFS 2014). This considerable restoration effort took place at the same time the subbasin and recovery plans were being drafted. Though the management documents lag behind the rush to implement restoration to conserve a species, our findings provide a baseline to inform adaptive management and suggest where restoration types are underutilized based on need (Runge 2011).

Widely distributed species, or those with large migration corridors present a number of unique recovery challenges including decentralized and overlapping multi‐agency management, increased diversity of restoration funders, and increasingly complex ecological threats—each of these factors demand increased cost, longer‐term management, and consequently greater accountability (Boyd et al. 2014, Carroll et al. 2014). The tools developed here address some of these challenges by examining large amounts of project data from multiple sources, over large spatial extents in a fast, consistent and transparent fashion. To do so however, required the development of a new project appropriateness measure since there were no existing mechanisms in use across the scale of salmon recovery that linked restoration actions to local ecosystem needs. If restoration deployment in the future is to be more efficient in addressing ecological need, common metrics will need to be incorporated into the regional management decision making frameworks. The approach developed here makes restoration appropriateness transparent to all decision makers from the landowner proposing a restoration project to the federal, state, local, tribal, and private entities involved in restoration funding, habitat assessment, and conservation planning.

While the SHAPE score evaluates restoration activity appropriateness, we have not assessed if the restoration was ultimately successful or if enough restoration has been done to alleviate an ecological concern. Ultimately, even appropriately placed projects cannot be deemed successful without proper monitoring (O'Donnell and Galat 2008, Dickens and Suding 2013, Palmer et al. 2007). In addition, even when projects are identified and population response metrics are available, high variability limits statistical power to inferences about effectiveness at only the largest spatial scales if at all (Paulsen and Fisher 2003, 2005). To obtain reliable inferences of management action effectiveness on the scale of a salmon population would require either data that does not currently exist over that scale, including restoration project success criteria, habitat monitoring, and spatially explicit habitat assessments, or application to a species with smaller spatial scales. For threatened species with smaller spatial footprints however, it is likely that the complexity of fragmented and decentralized management is less of an issue, ultimately making explicit effectiveness assessments easier and metrics such as a SHAPE score, less useful. The scale at which that complexity horizon exists is likely hard to predict, highly variable, and species specific.

Numerous recent papers have called for improved data standardization, reporting and monitoring of restoration, and approaches to restoration that facilitate adaptive management leading to improved understanding of species and ecosystem responses to restoration (Bernhardt et al. 2005, Palmer et al. 2005, Beechie et al. 2008, Runge 2011). Indeed, increased data standardization and sharing will continue to improve the scientific study of restoration (Palmer et al. 2007, Palmer 2009, Dickens and Suding 2013), and importantly empower higher resolution analysis than is possible with a presence/absence metric such as the SHAPE score. In the absence of standardization, different assessment methods can lead to biases in habitat evaluation (Al‐Chokhachy and Roper 2010). Spatially referenced assessments with consistent methods would greatly aid in analyses and future restoration planning for ESA listed species; generating such a data system would entail significant up front cost, but those costs would be offset to some degree by increased management efficiencies. With the present low‐availability and high‐cost restoration effectiveness data, our metrics address the near‐term need for an accountability mechanism in decentralized endangered species habitat management. While designed with salmon in mind, these methods are generally applicable to any imperiled species with a habitat assessment or recovery plan that identifies habitat concerns and can to improve information accessibility for project planning and placement across the diversity of stakeholders involved in habitat restoration and conservation planning.

Acknowledgments

We would like to thank George Pess, Jeff Jorgensen, Rodney Sayler, Matthew Carroll and Stephanie Hampton for thoughtful insights on earlier drafts.

Supplemental Material

Appendix A

Table A1. Broad crosswalk table relating the Pacific Northwest Salmon Habitat Project Database project types and subtypes (see https://www.webapps.nwfsc.noaa.gov/pnshp/ for definitions) to Ecological Concern types 1–3 (see Hamm 2012 for definitions). In the Broad crosswalk we attributed projects to ecological concerns even if the likelihood of success was highly optimistic, contingent on large‐scale implementation or on long‐term maintenance for project effectiveness. Ecological Concern types are: 1, Habitat quantity; 1.1, Anthropogenic barriers; 1.2, Natural barriers; 2, Direct mortality; 2.1, Predation; 2.2, Pathogens; 3, Toxic contaminants; 3.1, Water; 3.2, Biota.

Table A2. Broad crosswalk table relating the Pacific Northwest Salmon Habitat Project Database project types and subtypes (see https://www.webapps.nwfsc.noaa.gov/pnshp/ for definitions) to Ecological Concern types 4–5 (see Hamm 2012 for definitions). In the Broad crosswalk we attributed projects to ecological concerns even if the likelihood of success was highly optimistic, contingent on large‐scale implementation or on long‐term maintenance for project effectiveness. Ecological Concern types are: 4, Food; 4.1, Altered primary productivity; 4.2, Competition; 4.3 Altered prey species composition and diversity; 5, Riparian; 5.1, Riparian condition; 5.2, LWD recruitment.

Table A3. Broad crosswalk table relating the Pacific Northwest Salmon Habitat Project Database project types and subtypes (see https://www.webapps.nwfsc.noaa.gov/pnshp/ for definitions) to Ecological Concern types 6–8 (see Hamm 2012 for definitions). In the Broad crosswalk we attributed projects to ecological concerns even if the likelihood of success was highly optimistic, contingent on large‐scale implementation or on long‐term maintenance for project effectiveness. Ecological Concern types are: 6, Peripheral habitat; 6.1, Side channel and wetland conditions; 6.2, Floodplain condition; 7, Channel structure and form; 7.1, Bed and channel form; 7.2, Instream structural complexity; 8, Sediment conditions; 8.1, Decreased sediment quantity; 8.2, Increased sediment quantity.

Table A4. Broad crosswalk table relating the Pacific Northwest Salmon Habitat Project Database project types and subtypes (see https://www.webapps.nwfsc.noaa.gov/pnshp/ for definitions) to Ecological Concern types 9–10 (see Hamm 2012 for definitions). In the Broad crosswalk we attributed projects to ecological concerns even if the likelihood of success was highly optimistic, contingent on large‐scale implementation or on long‐term maintenance for project effectiveness. Ecological Concern types are: 9, Water quality; 9.1, Temperature; 9.2, Oxygen; 9.3, Turbidity; 9.4, pH; 9.5, Salinity; 10, Water quantity; 10.1, Increased water quantity; 10.2, Decreased water quantity; 10.3, Altered flow timing.

Appendix B

Table B1. Narrow crosswalk table relating the Pacific Northwest Salmon Habitat Project Database project types and subtypes (see https://www.webapps.nwfsc.noaa.gov/pnshp/ for definitions) to Ecological Concern types 1–3 (see Hamm 2012 for definitions). In the Narrow crosswalk we attributed project types with addressing only the most probable goal. Ecological Concern types are: 1, Habitat quantity; 1.1, Anthropogenic barriers; 1.2, Natural barriers; 2, Direct mortality; 2.1, Predation; 2.2, Pathogens; 3, Toxic contaminants; 3.1, Water; 3.2, Biota.

Table B2. Narrow crosswalk table relating the Pacific Northwest Salmon Habitat Project Database project types and subtypes (see https://www.webapps.nwfsc.noaa.gov/pnshp/ for definitions) to Ecological Concern types 4–5 (see Hamm 2012 for definitions). In the Narrow crosswalk we attributed project types with addressing only the most probable goal. Ecological Concern types are: 4, Food; 4.1, Altered primary productivity; 4.2, Competition; 4.3 Altered prey species composition and diversity; 5, Riparian; 5.1, Riparian condition; 5.2, LWD recruitment.

Table B3. Narrow crosswalk table relating the Pacific Northwest Salmon Habitat Project Database project types and subtypes (see https://www.webapps.nwfsc.noaa.gov/pnshp/ for definitions) to Ecological Concern types 6–8 (see Hamm 2012 for definitions). In the Narrow crosswalk we attributed project types with addressing only the most probable goal. Ecological Concern types are: 6, Peripheral habitat; 6.1, Side channel and wetland conditions; 6.2, Floodplain condition; 7, Channel structure and form; 7.1, Bed and channel form; 7.2, Instream structural complexity; 8, Sediment conditions; 8.1, Decreased sediment quantity; 8.2, Increased sediment quantity.

Table B4. Narrow crosswalk table relating the Pacific Northwest Salmon Habitat Project Database project types and subtypes (see https://www.webapps.nwfsc.noaa.gov/pnshp/ for definitions) to Ecological Concern types 9–10 (see Hamm 2012 for definitions). In the Narrow crosswalk we attributed project types with addressing only the most probable goal. Ecological Concern types are: 9, Water quality; 9.1, Temperature; 9.2, Oxygen; 9.3, Turbidity; 9.4, pH; 9.5, Salinity; 10, Water quantity; 10.1, Increased water quantity; 10.2, Decreased water quantity; 10.3, Altered flow timing.

Appendix C

Table C1. The Intermediate crosswalk table relating the Pacific Northwest Salmon Habitat Project Database project types and subtypes (see https://www.webapps.nwfsc.noaa.gov/pnshp/ for definitions) to Ecological Concern types 1–3 (see Hamm 2012 for definitions). The Intermediate crosswalk attributed projects with all concerns suggested by the literature unless cause and effect are separated by many steps, require coordinated implementation across large spatial scales or require time scales of decades for success. Ecological Concern types are: 1, Habitat quantity; 1.1, Anthropogenic barriers; 1.2, Natural barriers; 2, Direct mortality; 2.1, Predation; 2.2, Pathogens; 3, Toxic contaminants; 3.1, Water; 3.2, Biota.

Table C2. The Intermediate crosswalk table relating the Pacific Northwest Salmon Habitat Project Database project types and subtypes (see https://www.webapps.nwfsc.noaa.gov/pnshp/ for definitions) to Ecological Concern types 4–5 (see Hamm 2012 for definitions). The Intermediate crosswalk attributed projects with all concerns suggested by the literature unless cause and effect are separated by many steps, require coordinated implementation across large spatial scales or require time scales of decades for success. Ecological Concern types are: 4, Food; 4.1, Altered primary productivity; 4.2, Competition; 4.3 Altered prey species composition and diversity; 5, Riparian; 5.1, Riparian condition; 5.2, LWD recruitment.

Table C3. The Intermediate crosswalk table relating the Pacific Northwest Salmon Habitat Project Database project types and subtypes (see https://www.webapps.nwfsc.noaa.gov/pnshp/ for definitions) to Ecological Concern types 6–8 (see Hamm 2012 for definitions). The Intermediate crosswalk attributed projects with all concerns suggested by the literature unless cause and effect are separated by many steps, require coordinated implementation across large spatial scales or require time scales of decades for success. Ecological Concern types are: 6, Peripheral habitat; 6.1, Side channel and wetland conditions; 6.2, Floodplain condition; 7, Channel structure and form; 7.1, Bed and channel form; 7.2, Instream structural complexity; 8, Sediment conditions; 8.1, Decreased sediment quantity; 8.2, Increased sediment quantity.

Table C4. The Intermediate crosswalk table relating the Pacific Northwest Salmon Habitat Project Database project types and subtypes (see https://www.webapps.nwfsc.noaa.gov/pnshp/ for definitions) to Ecological Concern types 9–10 (see Hamm 2012 for definitions). The Intermediate crosswalk attributed projects with all concerns suggested by the literature unless cause and effect are separated by many steps, require coordinated implementation across large spatial scales or require time scales of decades for success. Ecological Concern types are: 9, Water quality; 9.1, Temperature; 9.2, Oxygen; 9.3, Turbidity; 9.4, pH; 9.5, Salinity; 10, Water quantity; 10.1, Increased water quantity; 10.2, Decreased water quantity; 10.3, Altered flow timing.

Appendix D

Table D1. Example table of how we applied the crosswalk comparison rules.