Decision-Making in Forestry: A Review of the Hybridisation of Multiple Criteria and Group Decision-Making Methods

Abstract

The degree of complexity in forest management has increased in the last few decades, not only due to the inclusion of specific new issues (e.g., climate change, social protection, etc.), but also because these new, as well as classic, issues have to be dealt with in a context characterised by multiple conflicting criteria that are evaluated by different stakeholders. Nowadays, the multicriteria issue enjoys a relatively sound tradition in forest management. However, the consideration of several stakeholders, which requires the formulation of management models within a collective decision-making setting, is not that advanced. This paper aims to provide a critical overview of forestry case studies that have been published in primary journals and that deal with multiple criteria and several stakeholders. Based on this overview, some highlights of the most promising methods were obtained, and recommendations for the fruitful use of these combined methodologies for dealing with numerous types of forest management problems are provided.

1. Introduction

In many current forest management scenarios, there is an unquestionable need to incorporate criteria of very different kinds, i.e., economic, environmental, social, etc., into the decision-making process. This is mainly due to the fact that modern societies demand various types of goods and services from forest systems. Thus, economic profitability, carbon uptake and biodiversity conservation make up a small sample of those aspects requested, despite their being, to some extent, contradictory. For instance, the strategies optimising economic profitability clash with the maximisation of the carbon uptake, with biodiversity conservation and so on [1,2]. The complications arising from the existence of several conflicting criteria have increased in the past few years due to the considerations or viewpoints on these criteria of the individuals or social groups potentially affected by the decision-making process implications (i.e., the so-called stakeholders).

Against this background, there is need for a precise integration of multicriteria methods with those belonging to disciplines such as group decision-making (GDM) and social choice. It is inevitable that a confident merging of methods belonging to these two disciplines is required to successfully deal with many current and relevant forest management problems. This necessity is especially crucial within a sustainability scenario. After all, the very conceptualisation of sustainability implies the contemplation of several criteria within a well-defined social context [3].

Indeed, the incorporation of multiple criteria into forest management can be considered to be a well-established field, with many theoretical and successful applications [4,5]. However, the use of GDM methods in forest management or, lesser still, the simultaneous use of both methods (i.e., modelling a forest management problem with multiple criteria within a collective decision-making context), is not very well ensconced [6].

In this context, the purpose of this paper is to provide a critical overview of forest management case studies published in primary journals using simultaneously, in one way or another, multiple criteria and GDM methods. The main features of these case studies will be commented on and critically evaluated. In this way, the pros and cons of the different approaches most widely used will be highlighted. Based on this analysis, some recommendations for the rational and simultaneous use of multicriteria optimisation and GDM models in forest management will be obtained. Hopefully, these results will encourage the sensible use of this new and promising method for realistically dealing with forest management problems formulated at any hierarchical decision-making level.

2. Theoretical Framework

2.1. Related Studies

Some works appearing prior to our research should be mentioned here [5,7,8]. However, these papers focus on the use of multiple criteria decision-making (MCDM) techniques in forestry, only indicating possible linkages with GDM techniques. Other authors also integrate decision support systems (DSSs) into their analysis [6,9,10,11]. It should be noted that the latter paper analyses MCDM and DSSs within a participatory context.

Following with significant precedents, Table 1 shows several reviews published in the past 10 years, indicating for each one the MCDM technique most frequently used as well as the percentage of papers focusing on GDM approaches. This table was set up to enable a comparison of the MCDM methods most widely used in works jointly employing GDM approaches.

Neither in the reviews previously analysed nor in those classified in Table 1 have authors analysed the main features of the respective case studies, where the method used is illustrated with the help of a practical example. It is important to point out that, generally, the reviews in the forestry sphere published up to now do not evaluate the characteristics of the case study in which these techniques have been applied. Based on the above, there are two main objectives of this work: first, to analyse, within the forestry environment, the studies in which, based on GDM and MCDM techniques, there is an interaction with more than one decision-maker and, second, to analyse in depth the main aspects of the decision-making process, taking into account aspects such as the number of interactions with decision-makers (DMs), how the criteria are chosen, and the main features of the DMs.

In the following sections, the methodology by which articles were selected is explained; subsequently, the results obtained are shown and classified into three groups according to the techniques employed, the characteristics of the participatory process conducted in each article, and the main features of the decision-makers involved in the process.

2.2. Participatory Decision-Making Methods

To classify the methodologies, in this section, the basic ideas of the techniques most used in the articles reviewed are described. These participatory decision-making techniques are aimed at by reaching a consensus among all the stakeholders involved in the underlying forest management problem. Such consensus can be obtained by resorting to any method for aggregating the preferences of each stakeholder implicated in the decision-making process.

With the intention of classifying the methodological tools most applied in the articles included in this review, the first discriminatory element lies in the cardinal or ordinal character of the preferences provided by the stakeholders. After that, the MCDM techniques used are described and a brief explanation of the interaction techniques most widely used is provided.

2.3. Group Decision-Making—Classification Based on the Nature of the Information of Preferences

The classification proposed in this block is based on the nature of the information of preferences from each decision-maker. This will be classified into ordinal information (normally expressed in a ranking of alternatives) and cardinal information (typically collected in a value or utility function or in a priority vector defining a weights system). In the case of ordinal information, the stakeholders only have to express the different alternatives in order from best to worst, without specifying the intensity of preferences that they would assign to each of them; this is something that is done in cardinal information.

Exceptionally, the Delphi method [13] does not include any of the following classifications since it is a method for achieving consensus on a particular topic through the use of rounds of questioning of experts in the field [14], and it is not classified as the previous one, either as ordinal or cardinal. Starting from three crucial features (anonymous response, iteration and controlled feedback and statistical group response), the first step in the Delphi method is having a group of experts/stakeholders answer questionnaires on a certain subject. After tabulating their responses, feedback is given anonymously to the entire group. In the second round of the Delphi method, the experts are requested to revise their answers and to comment on the group’s responses. After new feedback is given to the group, the process continues until a convergence of opinion is reached. Ultimately, a final group response is statistically compiled using the results of the process [15].

2.3.1. Ordinal Information

Ordinal information regarding a finite set of alternatives is usually expressed in the form of a ranking. We shall distinguish between two scenarios:

• Only part of the information contained in the ranking is used in the aggregation process—for instance, the information relating to the best alternative. This scenario is the case of voting methods.

• The entire ranking is used in the aggregation process. This is the case of social choice rules.

Voting Methods

Voting is a simple, well-known, conventional way to obtain ordinal information on individual preferences with which, subsequently, large-scale or small-group decisions are made. Although in all voting methods, each stakeholder is implicitly using a ranking of the set of alternatives, the only information finally employed corresponds to the alternative occupying the first position.

Voting methods are common in forestry participatory decision-making; by these methods, the stakeholders’ preferences are obtained and included in the forest planning process [16]. Their practical use in forestry does not technically differ from their use in other fields [17]. As a particular voting method, approval voting is one in which each member can vote for all the alternatives that he/she wishes and can use more information than merely that of the best alternative [18]. It does not require any classification of options [19] or any ranking of the alternatives [20]. The voters reveal their options by saying yes or no, casting a vote on each alternative [17], the winning alternative being the one with the highest number of votes received [21]. This introduces an alternate conception of voting as veto.

Social Choice Rules

When the full ranking of alternatives is used, we enter into the so-called field of social choice [18]. The decision based on social choice rules requires the voters to declare their preferences in the whole set of alternatives. According to Kangas et al. [21], the primary objective of social choice is to define the collective ranking from individual ones.

Borda’s rule of social choice is perhaps the best known. It starts from scoring the alternatives for each decision-maker, giving the highest score to the best alternative and the lowest to the worst: to the best alternative, n points are usually assigned, to the second one n − 1 and so on. The final score of an alternative is obtained by adding up the scores given by the decision-makers, with the winning alternative being the one obtaining the maximum final score [17].

Another social choice rule is based on the Condorcet functions, choosing the best alternative by examining all the pairwise comparisons from each individual stakeholder [22]. The score of each criterion is given by the number of times that it is classified as being higher than another [23]. The alternative selected according to the Condorcet criterion would be the one that exceeded any other candidate by a simple majority [18].

2.3.2. Cardinal Information

Some more sophisticated GDM models require stakeholders to supply information on the intensity of their preferences. Providing information of that nature is rather difficult and demands certain rationality traits in the stakeholder [24].

Cardinal information is usually gathered in the form of a utility function or of a vector of priority weights. Let us focus on the latter case, emphasising the rating method and pairwise comparison method, where a finite set of alternatives is considered. The rating method is similar to the ordinal ranking method in the sense that all indicators are judged by their relative degrees of importance, indicated by “scores” instead of a cardinal rank [25].

Additionally, the pairwise comparison method is commonly used to quantify priorities among the alternatives considered. In its more extended version, it is known as the “analytic hierarchy process” (AHP), a multiple criteria decision-making methodology widely used [26]. The latter consists of classifying alternatives/criteria [27], comparing them two by two and employing a scale introduced by Saaty [28], which indicates how many times one alternative/criterion is more important or dominant than another [29].

2.4. Multicriteria Decision-Making Methods Hybridised with GDM Techniques

Due to the proliferation of other reviews in the forestry sphere, in which MCDM techniques hybridised with GDM techniques have been explained in depth [3,6,13,30], here, no detailed descriptions are provided. However, in some cases, a bibliographic reference directly linked to the forestry environment is provided.

2.4.1. Discrete Methods

The discrete methods used in multiple criteria problems are characterised by the presence of a finite number of alternatives, in accordance with the level of fulfilment of the criteria considered. In this work, we employed the same classification of discrete methods used in Diaz-Balteiro et al. [3]: outranking methods (preference ranking organisation method for enrichment evaluation (PROMETHEE), novel approach to imprecise assessment and decision environments (NAIADE) and elimination and choice-expressing reality (ELECTRE)); hierarchical methods (analytic hierarchy process (AHP) and analytic network process (ANP), in relation to which the authors in Schmoldt et al. [31] compiled multiple applications of these techniques in forestry); and average optimisation (weighted arithmetic mean (WAM), weighted geometric aggregation (WGA), multiattribute utility theory (MAUT), multiattribute value theory (MAVT), heuristic optimisation method (HERO), simple multiattribute rating (SMART) and stochastic multicriteria acceptability analysis (SMAA)).

2.4.2. Continuous Methods

Continuous methods in multicriteria problems refer to situations where the feasible set is implicitly defined by a set of constraints (rigid or flexible) [32,33]. The continuous methods detected in this work are goal programming (GP), compromise programming (CP) and multiobjective programming (MOP).

2.5. Interactive Methods

To enable information on the DMs’ preferences to be obtained, and even to explore a priori the existence of a consensus, different interactive techniques are carried out. By means of an interactive method, more than one point of view is shared, and the DMs examine them with the intention of broadening their knowledge by a dialogue [34]. Normally, out of the DMs’ discussions, a consensus is achieved through which the objectives to be reached are common to all of them. Sometimes, through brainstorming, a face-to-face discussion is triggered between individuals with the aim of generating ideas [35]. Constant meetings are held to provide information to the participants before they have to make decisions. As many studies have reported, face-to-face contact is perhaps the most potent means of communication [36].

Another frequently used interaction technique is a survey, which can be performed in person, via e-mail or via a posted letter. The survey’s aim is to obtain individual information (quantitative or qualitative) from a specific group. Thus, all the individuals are asked the same questions [37]. The questions included can be open or closed questions or both, and can be direct or indirect. For the same purpose, but in a different interactive mode, interviews are conducted personally. On the other hand, the group work and field trips are meetings between people, in which the objective is to solve a problem through dialogue. In the case of fieldwork, these meetings are held in the area in which the problem has arisen, so that the decisions made are based on concrete, real facts that can be handled and dealt with personally.

3. Materials and Methods

Before explaining how the articles included in this work were found, it should be mentioned that the primary hypothesis tested in this work is that of demonstrating that the hybridisation of MCDM and GDM techniques is strongly increasing in the forestry sphere.

First, to be able to analyse the different study cases, an extensive review of the literature, published up to 31 December 2017, was carried out. This exhaustive task was done by making use of the Web of Science Database. In this database, two searches were conducted using the keywords indicated in Appendix A. In these searches, only scientific articles present in that database were taken into account, excluding other articles, book chapters or books. At the end of the searches, 1226 articles were obtained.

Taking into account these 1226 articles, there was a selection process (Table 2), of which the objective was to obtain articles in which one (or several) study cases were analysed by simultaneously using GDM and MCDM methods. In this first phase, we excluded 893 articles because 559 of them did not apply MCDM techniques, 40 articles (fundamentally reviews) did not incorporate a case study, and, as a result of both searches, 294 were identified as duplicates and were only counted once.

After this first phase, 333 articles were obtained and analysed in detail. Twelve did not refer to any case study, and 169 did not use MCDM or GDM methods, so these were excluded. Moreover, there was no access to 10 of them, so these could therefore not be analysed (they were not available online, and, despite requesting them from their authors, no reply was received); three articles were written in a language other than Spanish, English or Portuguese, and five of them were outside the forestry context.

After compilation of all articles, we proceeded to define the aspects that would be taken into account. Thus, the published articles were classified in terms of the year of publication and the sphere of the problems analysed. In the same direction, the GDM technique and the interactive method most widely used were identified. In addition, DSSs (a computerised combination of methods, models and database components) were also considered to find out whether there was a relationship between GDM-MCDM methods and DSSs. For some authors, e.g., those of Borges et al. [38], an active involvement of stakeholders in a DSS is a positive condition. Finally, works in which computer programmes were applied in the decision process were identified.

With regard to the stakeholders’ participation, how criteria can be fixed and whether a consensus process should be set up to establish them were determined. Likewise, the number of criteria used was analysed, and the origin of the preferences was classified into two categories: individual, if the preferences of each DM were obtained, or group, when the total amount of DMs was found. As for the decision-makers, the terms most frequently employed to refer to them were determined. Similarly, the characterisation of the main stakeholders, their numbers, and the types of interactions implemented were established. Although, rigorously, decision-makers and stakeholders are different entities, and several taxonomies can be provided to distinguish them, our criterion was to maintain how the stakeholders have been mentioned and characterised in each paper reviewed.

4. Results

Results were obtained by taking into account what the authors stated in the papers analysed. Thus, criteria used to determine the topic of the problem or the type of stakeholder, for example, were chosen so as to maintain what the authors provided in each paper.

4.1. Techniques Employed

Following the results shown in Table 2, 134 articles were obtained (Appendix B), and these articles constitute the base of the work in this review and are analysed next. Articles were published between 1992 and 2017, and publications significantly increased in 1998, as shown in Figure 1.

In relation to the problems most frequently addressed (see Table 3), 78 of them (i.e., 58% of the total) were forest management issues. The second most frequent type of problem was related to sustainability (22% of the total). Ten percent of the articles referred to industrial forest plantations, and 9% to cases concerning forest conservation issues. Various themes appear in other articles (biodiversity, climate changes, restoration, etc.), but with a lower number of cases. By way of example, works, such as Mikkilä et al. [39], simultaneously refer to forest management and forest plantations and were therefore computed in both fields. One exception produced in that analysis, and which occurred repeatedly, was that a higher summation was obtained than the total articles analysed, as most of the case studies comprised more than one characteristic or attribute. For that reason, in the following tables, the totals may exceed the number of articles considered.

Table 4 shows the breakdown of works, hybridising GDM techniques with MCDM techniques and the interaction methods applied in each study case. What drew our attention was the frequent use of pairwise comparisons in the GDM techniques, since the latter was employed in 89 of the 134 study cases analysed (66% of the total). Of these 89 cases, 78 employed AHP, which is thus the multicriteria technique most commonly used in GDM problems.

Another characteristic taken into account was the use of DSSs in the problems in which GDM techniques and MCDM were employed. Of the 134 articles analysed, their authors refer to DSSs in 22 of them (16% of the total). Similarly, in 30 cases, a computer programme was used in the decision-making process. In 15 of those 30 cases, Geographic Information System (GIS) tools were used.

4.2. Participatory Process

Regarding the selection of criteria (Figure 2) employed in each case study, they were previously predefined in 33 of them and were not chosen by the DMs involved. However, in 59 studies, the criteria were determined through interactions with the DMs, and a consensus was finally reached in the election of those criteria. The method used for criteria selection was not specified in 42 of the studies.

A discussion was the approach most frequently used for determining the relevant criteria (24% of the total). Additionally, in 27 articles (20% of the total), a consensus was obtained through those discussions. These consensuses were obtained based on the information supplied by the different authors, who expressly mentioned this fact. In short, there may be cases in which a consensus was reached but not reported by the authors, and, for this reason, the consensus was not counted as such. In the studies analysed, other methods for determining criteria were detected: they were not employed in more than five articles or they were annotated as “others”. With regard to the number of criteria used (see Figure 3) in the different works, in 66% of them, between 2 and 10 criteria were applied, whereas in the remaining 34%, there were more than 11 criteria.

With respect to the elicitation of preferences (Table 5), this was usually done on an individual level and then aggregated (90% of the total) against the 10% corresponding to collective ones. It should be noted that, in several articles, both preferences were employed when performing more than one interaction.

4.3. Results Related to the Decision-Makers

One peculiarity included in this review was the use of specific vocabulary (Table 6), quantifying the terms most used. Thus, of the 134 articles examined, in 73% of them, the term “decision-maker” (DM) was employed most. This percentage was very similar to the term “stakeholder”, mentioned in 64% of study cases. Another term, “expert”, was referred to in 46% of the papers.

Another aspect examined was the type of stakeholder selected to conduct the analysis (see Table 7). Through this analysis, it was ascertained that in 51 of the 134 articles analysed, government or public administration officials were selected as a source of information, thus becoming the most frequently consulted stakeholders. The number of stakeholders belonging to each type is quite variable among them. Likewise, the opinion of private property owners was frequently taken into account, since, in 43 articles, their opinion was recorded. In contrast, in 37 of the articles, ecologists took part, and in 34, the local population was considered. In 22 articles where the term “experts” referred to DMs, the type of expert was not specified. The opinions of scientists, technicians and foresters were used in a smaller proportion than those mentioned previously.

In the same way, evaluating the number of DMs interacting in each process was considered essential (Figure 4). In 39% of the cases in which a single interaction occurred, the number of DMs oscillated between 2 and 9. On the other hand, of the cases with a second interaction, 43% contained between 10 and 50 DMs. Finally, there were only four cases where three or more interactions carried out. It should be noted that the summation of the number of DMs used in each interaction is less than 134 because the exact number of DMs participating in the analysis was not specified in all the articles.

It is essential not only to take into account the number of decision-makers involved in each interaction but also to compute the number of groups involved in the different decision-making processes (Figure 5). This figure clearly shows how the most common method is to resort to a single group of stakeholders. It is interesting to note how the number of study cases diminishes as the number of groups increases.

Finally, a crucial question pertains to the nature of the articles analysed. Thus, the distinction between demonstrations and real-world applications could be useful to contextualise the type of problems to which these hybrid techniques are applied. Our results show (Table 8) that the number of papers involving real-world problems is substantially lower compared to those involving illustrative examples, research projects and pilot studies.

5. Discussion

First, it is important to highlight the most frequent results obtained in each section. Generally, it can be said that the context of the problem most widely addressed is forest management, mostly applying pairwise comparisons within an AHP format to obtain the most satisfactory results. The number of criteria ranges between two and 10 and is obtained interactively from a number of DMs of between two and nine, and these DMs are united in a group and are interacted with once or twice.

This study demonstrates an increase in works published in the past few years in which GDM and MCDM methods are used together. Thus, according to the results given in the above section, the number of articles published between the years 2013 and 2017 is three times that of articles published from 1998 to 2002. This fact is further verified in Diaz-Balteiro et al. [3], a study related to sustainability issues, and in Velasquez and Hester [23], wherein the use of these techniques is shown to have significantly increased as new methods were developed. Similar to the works of Kabak and Ervural [32] and Bruña-García and Marey-Pérez [40], it can be seen how these approaches have been employed more consistently since the year 2000. In view of these results, and those obtained in the reviews revised, it can be considered that GDM and MCDM hybridisation is in the process of becoming a fruitful and consolidated methodology.

One aspect that needs to be kept in mind is the application of computer programmes to forestry participatory problems. Thus, the authors in Hujala et al. [41] investigated this and found that five of the 32 cases analysed used MCDM methods, i.e., 16%, only slightly lower than the 21% obtained in our work. These data show that there has not been any significant increase in the use of computer programmes, a counterproductive fact according to Martins and Borges [42]. In effect, those authors [41,42] emphasise the fact that setting up technological platforms could promote the effective integration of new methods and tools. On the other hand, some authors [11] consider that maps or diagrams of the case study are more favourable for helping stakeholders to better understand the conflict posed. As for possible relationships between GDM and MCDM methods with DSSs, no reliable conclusion has been reached, since DSSs have only been detected in 16% of the articles analysed. This is in disagreement with the opinions of some authors [10,43], given that in 84% of the articles analysed no DSSs were employed; such authors insist on the idea that hybrid methods (GDM + MCDM) ought to be integrated into the DSSs.

With regard to the participatory techniques used in forestry works, there is a striking difference in the results obtained in Khadka et al. [44]. In fact, in that study, surveys were made in 4% of the studies compared to 27% administered in this review. This is also the case for the results of Esmail and Geneletti [45], who confirm the use of interviews and questionnaires in 8% of the works analysed.

The multicriteria technique analysis most used, an aspect that has been extensively studied in other works [3,7,8,10,12], was AHP and has been applied in 84 reported cases. This result coincides with that obtained by Diaz-Balteiro et al. [3], in which it is shown that the two techniques most commonly employed are AHP and weighted arithmetic mean (AWM), in opposition to Esmail and Geneletti [45], the authors of which conclude that the multicriteria methodology most used in the past 20 years is weighted linear combination (WLC). The authors in de Castro and Urios [30] report that only 15% of the articles studied apply AHP or ANP. Coinciding with the results obtained in this review, the authors in Diaz-Balteiro et al. [46] confirm that goal programming was the method most frequently used to deal with continuous problems in forest management.

Regarding participatory approaches, the authors in de Castro and Urios [30] performed an in-depth analysis and pointed out an increase in the use of AHP or ANP between 2004 and 2007, albeit with a decrease from 2008 onwards, which, according to the authors, is the result of decision-making processes with a high degree of collaboration, which are laborious and costly in terms of time and resources. However, these data do not correspond to those obtained in our work, since starting from 2002 the use of participatory approaches in forestry issues was found to have significantly increased up to now. Concerning criteria election, Esmail and Geneletti [45] stands out. The authors found that 49% of the cases analysed use predefined criteria that had not been elected by members involved, whereas in our analysis, this figure was 25%. Finally, the authors in Martins and Borges [42] indicate that, if the decision-makers are examined, identifying the stakeholders is a necessary measure, as they are now beginning to actively participate in the planning process. In Khadka et al. [44], in 26 out of 32 articles analysed, information from a government authority was taken into account.

Finally, we cannot recommend any specific hybrid method because how an aggregation rule can be implemented for a set of individual preferences is still a wide open issue [47], as we have shown in Table 4. However, following Diaz-Balteiro et al. [48], the “pairwise” comparison format could be a clever option for aggregating the preferences of the stakeholders considered. This task is inexpensive from a computational perspective [49,50], and the AHP has been used in many GDM problems compiled in this document.

6. Conclusions

In this review, the joint use of GDM and MCDM methods in forestry was analysed, and it is clear that this type of methodology is significantly increasing. In fact, in the year 2000, publications of this sort started to become more abundant, leading to the conclusion that the simultaneous use of both techniques is in a process of consolidation, especially in works in which forest management cases are studied.

Based on the results obtained, it can be deduced that AHP is the technique most applied in these hybrid models, although this does not mean that it is the only one using a pairwise comparison format in GDM problems in the forestry fields, as has been shown above. It can be assumed that the number of decision-makers is related to the number of interactions, since the higher the number of interactions produced, the lower the number of decision-makers interacting with the decision-making process. As for the terminology used, the term preferred in referring to the members involved is “decision-maker” (DM). It is worth noting that the social groups most considered in the participatory process are the local or national governments as well as public administration officials and private owners.

Author Contributions

E.O.-U. carried out all the research and wrote the manuscript. J.G.-P. wrote the manuscript, focusing on GDM issues. L.D.-B. coordinated the study and wrote the manuscript, especially in terms of MCDM issues.

Funding

The work of Esther Ortiz-Urbina is part of her Doctoral Thesis, funded by a predoctoral support: Grant for Predoctoral Contracts for the Training of Doctors 2016 (BES-2016-079402) of the Ministry of Economy and Competitiveness of Spain. The work of Jacinto González-Pachón and Luis Diaz-Balteiro was funded by the Ministry of Economy and Competitiveness of Spain under project AGL2015-68657-R. Luis Diaz-Balteiro received funding from the SuFoRun Project (‘Models and Decision Support Tools for Integrated Forest Policy Development under Global Change and Associated Risk and Uncertainty’), which is funded by a Marie Curie International Research Staff Exchange Scheme Fellowship within the H2020 European Union Framework Program (H2020-MSCA-RISE-2015).

Acknowledgments

The authors deeply appreciate the comments and suggestions raised by Carlos Romero. Thanks are extended to Diana Badder for editing the English language. Additionally, this manuscript has been proofread by PRS and MDPI. Thanks are given to the Editor and the reviewers for their helpful comments and criticisms, which have greatly improved the presentation and accuracy of the paper.

Conflicts of Interest

The authors declare no conflicts of interests.

Appendix A. Key Words Used in the Literature Search

Keywords Used
Search 1	Topic: ‘Goal programming’ OR GP OR ‘Compromise programming’ OR CP OR ‘Multi objective programming’ OR MOP OR ‘Multiobjective programming’ OR ‘ELECTRE’ OR ‘elimination and choice expressing reality’ OR ‘Promethee’ OR ‘PROMETHEE’ OR ‘preference ranking organisation method for enrichment evaluations’ OR ‘MAUT’ OR ‘multiattribute utility theory’ OR ‘multi attribute utility theory’ OR ‘multiattribute utility theory’ OR ‘MAVT’ OR ‘multiattribute value theory’ OR ‘multi attribute value theory’ OR ‘SMART’ OR ‘simple multiattribute rating technique’ OR ‘simple multiattribute rating technique’ OR ‘SAW’ OR ‘simple additive weighting’ OR ‘TOPSIS’ OR ‘Technique for Order of Preference by Similarity to Ideal Solution’ OR ‘AHP’ OR ‘analytic hierarchy process’ OR ‘ANP’ OR ‘analytic network process’ OR ‘VIKOR’ OR ‘HERO’ OR ‘heuristic optimisation’ OR ‘heuristic optimisation’ OR ‘DEA’ OR ‘data envelopment analysis’ OR ‘CBR’ OR ‘case-based reasoning’ OR Fuzzy methods OR ‘NAI’ OR ‘negotiable alternative identifier’ OR ‘NAIADE’ OR ‘novel approach to imprecise assessment and decision environments’ OR ‘preference ratios in multiattribute evaluation’ OR ‘multicriteria approval’ OR ‘multicriteria approval’ OR ‘MA’ OR ‘SMAA’ OR ‘stochastic multicriteria acceptability analysis’ OR ‘stochastic multiobjective acceptability analysis’ OR ‘WPM’ OR ‘weighted product method’ OR ‘Weighted average’ OR ‘WAM’ OR ‘Weighted Arithmetic mean’ OR ‘weighted geometric mean’ OR ‘geometric mean’
	AND
	Topic: group decision-making OR decision criteria OR participatory OR participation OR collaborative OR social choice OR decision alternatives OR collaboration OR GDM OR GD OR voting OR stakeholder analysis OR decision theory OR decisions
	Refined by: Web of science, categories: (FORESTRY)
Search 2	Topic: Stakeholder OR participant OR expert OR professional OR technician OR voter OR group OR practitioner OR decision maker OR student OR respondent OR actors
	AND
	Topic: group decision-making OR group decision-making OR GDM OR GD OR participatory OR participation OR collaborative OR collaboration OR ‘social choice’ OR decisions OR ‘decision alternatives’ OR ‘decision criteria’ OR ‘decision theory’ OR ‘utility theory’ OR optimisation OR optimisation OR voting OR ‘participative modelling’ OR ‘multistakeholder assessment’ OR ‘stakeholder analysis’ OR ‘preferences’ OR ‘aggregation’
	AND
	Topic: multicriteria OR multicriteria OR MCDA OR MCDM OR multiple criteria OR multiobjective OR multiobjective
	Refined by: Web of science, categories: (FORESTRY)

Appendix B. Selected Articles

• Kangas, J. Multiple-use planning of forest resources by using the analytic hierarchy process. Scand. J. For. Res. 1992, 7, 259–268, doi:10.1080/02827589209382718.

• Kangas, J.; Kuusipalo, J. Integrating biodiversity into forest management planning and decision-making. For. Ecol. Manag. 1993, 61, 1–15, doi:10.1016/0378-1127(93)90186-Q.

• Kangas, J.; Laasonen, L.; Pukkala, T. A Method for Estimating Forest Landowner’s Landscape Preferences. Scand. J. For. Res. 1993, 8, 408–417, doi:10.1080/02827589309382787.

• Schmoldt, D.L.; Peterson, D.L.; Silsbee, D.G. Developing inventory and monitoring programs based on multiple objectives. Environ. Manag. 1994, 18, 707–727, doi:10.1007/Bf02394635.

• Reynolds, K.M.; Holsten, E.H. Relative importance of risk factors for spruce beetle outbreaks. Can. J. For. Res. 1994, 24, 2089–2095.

• Kangas, J. Incorporating risk attitude into comparison of reforestation alternatives. Scand. J. For. Res. 1994, 9, 297–304, doi:10.1080/02827589409382843.

• Kangas, J. An approach to public participation in strategic forest management planning. For. Ecol. Manag. 1994, 70, 75–88, doi:10.1016/0378-1127(94)90076-0.

• Alho, J.M.; Kangas, J. Analyzing Uncertainties in Experts’ Opinions of Forest Plan Performance. For. Sci. 1997, 43, 521–528.

• Bantayan, N.C.; Bishop, I.D. Linking objective and subjective modelling for landuse decision-making. Landsc. Urban Plan. 1998, 43, 35–48, doi:10.1016/S0169-2046(98)00101-7.

• Pukkala, T. Multiple risks in multi-objective forest planning: Integration and importance. For. Ecol. Manag. 1998, 111, 265–284, doi:10.1016/S0378-1127(98)00339-9.

• Tecle, A.; Shrestha, B.; Duckstein, L. A Multiobjective Decision Support System for Multiresource Forest Management. Group Decis. Negot. 1998, 7, 23–40, doi:10.1023/A.:1008671129325.

• Mendoza, G.A.; Prabhu, R. Development of a Methodology for Selecting Criteria and Indicators of Sustainable Forest Management: A Case Study on Participatory Assessment. Environ. Manag. 2000, 26, 659–673, doi:10.1007/s002670010123.

• Schmoldt, D.L.; Peterson, D.L. Analytical Group Decision Making in Natural Resources: Methodology and Application. For. Sci. 2000, 46, 62–75.

• Mendoza, G.A.; Prabhu, R. Multiple criteria decision making approaches to assessing forest sustainability using criteria and indicators: A case study. For. Ecol. Manag. 2000, 131, 107–126, doi:10.1016/S0378-1127(99)00204-2.

• Kangas, A.; Kangas, J.; Pykäläinen, J. Outranking Methods As Tools in Strategic Natural Resources Planning. Silva Fenn. 2001, 35, 215–227.

• Store, R.; Kangas, J. Integrating spatial multi-criteria evaluation and expert knowledge for GIS-based habitat suitability modelling. Landsc. Urban Plan. 2001, 55, 79–93, doi:10.1016/S0169-2046(01)00120-7.

• Pykäläinen, J.; Pukkala, T.; Kangas, J. Alternative priority models for forest planning on the landscape level involving multiple ownership. For. Policy Econ. 2001, 2, 293–306, doi:10.1016/S1389-9341(01)00035-1.

• Pesonen, M.; Kurttila, M.; Kangas, J.; Kajanus, M.; Heinonen, P. Assessing the Priorities Using A’WOT Among Resource Management Strategies at the Finnish Forest and Park Service. For. Sci. 2001, 47, 534–541.

• Vacik, H.; Lexer, M.J. Application of a spatial decision support system in managing the protection forests of Vienna for sustained yield of water resources. For. Ecol. Manag. 2001, 143, 65–76.

• Laukkanen, S.; Kangas, A.; Kangas, J. Applying voting theory in natural resource management: A case of multiple-criteria group decision support. J. Environ. Manag. 2002, 64, 127–137, doi:10.1006/jema.2001.0511.

• Hytönen, L.A.; Leskinen, P.; Store, R. A Spatial Approach to Participatory Planning in Forestry Decision Making. Scand. J. For. Res. 2002, 17, 62–71, doi:10.1080/028275802317221091.

• Bojórquez-Tapia, L.A.; Brower, L.P.; Castilleja, G.; Sánchez-Colón, S.; Hernández, M.; Calvert, W.; Díaz, S.; Gómez-Priego, P.; Alcantar, G.; Melgarejo, E.D.; et al. Mapping Expert Knowledge Redesigning the Monarch Butterfly Biosphere Reserve. Conserv. Biol. 2003, 17, 367–379, doi:10.1046/j.1523-1739.2003.01309.x.

• Ananda, J.; Herath, G. Incorporating stakeholder values into regional forest planning: A value function approach. Ecol. Econ. 2003, 45, 75–90, doi:10.1016/S0921-8009(03)00004-1.

• Ananda, J.; Herath, G. The use of Analytic Hierarchy Process to incorporate stakeholder preferences into regional forest planning. For. Policy Econ. 2003, 5, 13–26, doi:10.1016/S1389-9341(02)00043-6.

• Leskinen, P.; Kangas, J.; Pasanen, A.M. Assessing ecological values with dependent explanatory variables in multi-criteria forest ecosystem management. Ecol. Model. 2003, 170, 1–12, doi:10.1016/S0304-3800(03)00283-7.

• Purnomo, H.; Mendoza, G.A.; Prabhu, R. Model for Collaborative Planning of Community-Managed Resources Based on Qualitative Soft Systems Approach. J. Trop. For. Sci. 2004, 16, 106–131.

• Hjortsø, C.N. Enhancing public participation in natural resource management using Soft OR - an application of strategic option development and analysis in tactical forest planning. Eur. J. Oper. Res. 2004, 152, 667–683, doi:10.1016/S0377-2217(03)00065-1.

• Laukkanen, S.; Palander, T.; Kangas, J. Applying voting theory in participatory decision support for sustainable timber harvesting. Can. J. For. Res. 2004, 34, 1511–1524, doi:10.1139/x04-044.

• Mau-Crimmins, T.; De Steiguer, J.E.; Dennis, D. AHP as a means for improving public participation: A pre-post experiment with university students. For. Policy Econ. 2005, 7, 501–514, doi:10.1016/j.forpol.2003.08.001.

• Laukkanen, S.; Palander, T.; Kangas, J.; Kangas, A. Evaluation of the Multicriteria Approval Method for Timber-Harvesting Group Decision Support. Silva Fenn. 2005, 39, 249–264, doi:10.14214/sf.387.

• Mikkilä, M.; Kolehmainen, O.; Pukkala, T. Multi-attribute assessment of acceptability of operations in the pulp and paper industries. For. Policy Econ. 2005, 7, 227–243, doi:10.1016/S1389-9341(03)00062-5.

• Pasanen, K.; Kurttila, M.; Pykäiäinen, J.; Kangas, J.; Leskinen, P. MESTA- Non-industrial private forest owners’ decision-support environment for the evaluation of alternative forest plans over the internet. Int. J. Inf. Technol. Decis. Mak. 2005, 4, 601–620.

• Phua, M.-H.; Minowa, M. A GIS-based multi-criteria decision making approach to forest conservation planning at a landscape scale: A case study in the Kinabalu Area, Sabah, Malaysia. Landsc. Urban Plan. 2005, 71, 207–222, doi:10.1016/j.landurbplan.2004.03.004.

• Kangas, A.; Kangas, J.; Laukkanen, S. Fuzzy multicriteria approval method and its application to two forest planning problems. For. Sci. 2006, 52, 232–242.

• Leskinen, L.A.; Leskinen, P.; Kurttila, M.; Kangas, J.; Kajanus, M. Adapting modern strategic decision support tools in the participatory strategy process-a case study of a forest research station. For. Policy Econ. 2006, 8, 267–278, doi:10.1016/j.forpol.2004.06.007.

• Mendoza, G.A.; Prabhu, R. Participatory modeling and analysis for sustainable forest management: Overview of soft system dynamics models and applications. For. Policy Econ. 2006, 9, 179–196, doi:10.1016/j.forpol.2005.06.006.

• Linkov, I.; Satterstrom, F.K.; Kiker, G.; Batchelor, C.; Bridges, T.; Ferguson, E. From comparative risk assessment to multi-criteria decision analysis and adaptive management: Recent developments and applications. Environ. Int. 2006, 32, 1072–1093, doi:10.1016/j.envint.2006.06.013.

• Zadnik Stirn, L. Integrating the fuzzy analytic hierarchy process with dynamic programming approach for determining the optimal forest management decisions. Ecol. Model. 2006, 194, 296–305, doi:10.1016/j.ecolmodel.2005.10.023.

• Bowe, S.A.; Smith, R.L. Modeling Technology Adoption in the Hardwood Sawmill. Wood Fiber Sci. 2006, 38, 484–496.

• Leskinen, P.; Viitanen, J.; Kangas, A.; Kangas, J. Alternatives to Incorporate Uncertainty and Risk Attitude in Multicriteria Evaluation of Forest Plans. For. Sci. 2006, 52, 304–312.

• Tikkanen, J.; Isokääntä, T.; Pykäläinen, J.; Leskinen, P. Applying cognitive mapping approach to explore the objective-structure of forest owners in a Northern Finnish case area. For. Policy Econ. 2006, 9, 139–152, doi:10.1016/j.forpol.2005.04.001.

• Strager, M.P.; Rosenberger, R.S. Incorporating stakeholder preferences for land conservation: Weights and measures in spatial MCA. Ecol. Econ. 2006, 58, 79–92, doi:10.1016/j.ecolecon.2005.05.024.

• Pykäläinen, J.; Hiltunen, V.; Leskinen, P. Complementary use of voting methods and interactive utility analysis in participatory strategic forest planning: Experiences gained from western Finland. Can. J. For. Res. 2007, 37, 853–865, doi:10.1139/X06-241.

• Fürstenau, C.; Badeck, F.W.; Lasch, P.; Lexer, M.J.; Lindner, M.; Mohr, P.; Suckow, F. Multiple-use forest management in consideration of climate change and the interests of stakeholder groups. Eur. J. For. Res. 2007, 126, 225–239.

• Regan, H.M.; Davis, F.W.; Andelman, S.J.; Widyanata, A.; Freese, M. Comprehensive criteria for biodiversity evaluation in conservation planning. Biodivers. Conserv. 2007, 16, 2715–2728, doi:10.1007/s10531-006-9100-3.

• Oliver, I.; Jones, H.; Schmoldt, D.L. Expert panel assessment of attributes for natural variability benchmarks for biodiversity. Austral Ecol. 2007, 32, 453–475, doi:10.1111/j.1442-9993.2007.01718.x.

• Ananda, J. Implementing Participatory Decision Making in Forest Planning. Environ. Manag. 2007, 39, 534–544, doi:10.1007/s00267-006-0031-2.

• Vainikainen, N.; Kangas, A.; Kangas, J. Empirical study on voting power in participatory forest planning. J. Environ. Manag. 2008, 88, 173–180, doi:10.1016/j.jenvman.2007.02.004.

• Locatelli, B.; Rojas, V.; Salinas, Z. Impacts of payments for environmental services on local development in northern Costa Rica: A fuzzy multi-criteria analysis. For. Policy Econ. 2008, 10, 275–285, doi:10.1016/j.forpol.2007.11.007.

• Gomontean, B.; Gajaseni, J.; Edwards-Jones, G.; Gajaseni, N. The development of appropriate ecological criteria and indicators for community forest conservation using participatory methods: A case study in northeastern Thailand. Ecol. Indic. 2008, 8, 614–624, doi:10.1016/j.ecolind.2007.08.006.

• Sugimura, K.; Howard, T.E. Incorporating social factors to improve the Japanese forest zoning process. For. Policy Econ. 2008, 10, 161–173, doi:10.1016/j.forpol.2007.08.005.

• Dhar, A.; Ruprecht, H.; Vacik, H. Population viability risk management (PVRM) for in situ management of endangered tree species-A case study on a Taxus baccata L. population. For. Ecol. Manag. 2008, 255, 2835–2845, doi:10.1016/j.foreco.2008.01.059.

• Hiltunen, V.; Kangas, J.; Pykäläinen, J. Voting methods in strategic forest planning - Experiences from Metsähallitus. For. Policy Econ. 2008, 10, 117–127, doi:10.1016/j.forpol.2007.06.002.

• Kurttila, M.; Leskinen, P.; Pykäläinen, J.; Ruuskanen, T. Forest Owners’ Decision Support in Voluntary Biodiversity-Protection Projects. Silva Fenn. 2008, 42, 643–658, doi:10.14214/sf.238.

• de Oliveira AvernaValente Valente, R.; Vettorazzi, C.A. Definition of priority areas for forest conservation through the ordered weighted averaging method. For. Ecol. Manag. 2008, 256, 1408–1417, doi:10.1016/j.foreco.2008.07.006.

• Seidl, R.; Rammer, W.; Lexer, M.J. Climate change vulnerability of sustainable forest management in the Eastern Alps. Clim. Chang. 2011, 106, 225–254, doi:10.1007/s10584-010-9899-1.

• Lexer, M.J.; Seidl, R. Addressing biodiversity in a stakeholder-driven climate change vulnerability assessment of forest management. For. Ecol. Manag. 2009, 258S, S158–S167, doi:10.1016/j.foreco.2009.07.011.

• Pérez-Verdín, G.; Hernández-Díaz, J.C.; Márquez-Linares, M.A.; Tecle, A. Application of multi-criterion techniques on integral forest management in Durango, México. Madera Y Bosques 2009, 15, 27–46.

• Diaz-Balteiro, L.; Gonzalez-Pachon, J.; Romero, C. Forest management with multiple criteria and multiple stakeholders: An application to two public forests in Spain. Scand. J. For. Res. 2009, 24, 87–93, doi:10.1080/02827580802687440.

• Leskinen, P.; Pykäläinen, J.; Haara, A. Incorporation of preferential uncertainty into interval-scale priority functions—A case of multicriteria forestry decision making. Can. J. For. Res. 2009, 39, 1498–1506, doi:10.1139/X09-055.

• Nordström, E.-M.; Romero, C.; Eriksson, L.O.; Öhman, K. Aggregation of preferences in participatory forest planning with multiple criteria: An application to the urban forest in Lycksele, Sweden. Can. J. For. Res. 2009, 39, 1979–1992, doi:10.1139/X09-107.

• Mendoza, G.A.; Prabhu, R. Evaluating multi-stakeholder perceptions of project impacts: A participatory value-based multi-criteria approach. Int. J. Sustain. Dev. World Ecol. 2009, 16, 177–190, doi:10.1080/13504500902919672.

• Eyvindson, K.; Kangas, A.; Kurttila, M.; Hujala, T. Using preference information in developing alternative forest plans. Can. J. For. Res. 2010, 40, 2398–2410, doi:10.1139/X10-179.

• Eyvindson, K.; Kurttila, M.; Hujala, T.; Salminen, O. An Internet-Supported Planning Approach for Joint Ownership Forest Holdings. Small-scale For. 2011, 10, 1–17, doi:10.1007/s11842-010-9123-1.

• Greene, R.; Luther, J.E.; Devillers, R.; Eddy, B. An approach to GIS-based multiple criteria decision analysis that integrates exploration and evaluation phases: Case study in a forest-dominated landscape. For. Ecol. Manag. 2010, 260, 2102–2114, doi:10.1016/j.foreco.2010.08.052.

• Balana, B.B.; Mathijs, E.; Muys, B. Assessing the sustainability of forest management: An application of multi-criteria decision analysis to community forests in northern Ethiopia. J. Environ. Manag. 2010, 91, 1294–1304, doi:10.1016/j.jenvman.2010.02.005.

• Yousuf Hossain, S. Md.; (Ted) Robak, E.W. A Forest Management Process to Incorporate Multiple Objectives: A Framework for Systematic Public Input. Forests 2010, 1, 99–113, doi:10.3390/f1030099.

• Vadrevu, K.P.; Eaturu, A.; Badarinath, K.V.S. Fire risk evaluation using multicriteria analysis—A case study. Environ. Monit. Assess. 2010, 166, 223–239, doi:10.1007/s10661-009-0997-3.

• Nordström, E.-M.; Eriksson, L.O.; Öhman, K. Integrating multiple criteria decision analysis in participatory forest planning: Experience from a case study in northern Sweden. For. Policy Econ. 2010, 12, 562–574, doi:10.1016/j.forpol.2010.07.006.

• Duchelle, A.E.; Guariguata, M.R.; Less, G.; Albornoz, M.A.; Chavez, A.; Melo, T. Evaluating the opportunities and limitations to multiple use of Brazil nuts and timber in Western Amazonia. For. Ecol. Manag. 2012, 268, 39–48, doi:10.1016/j.foreco.2011.05.023.

• Nordström, E.-M.; Eriksson, L.O.; Öhman, K. Multiple Criteria Decision Analysis with Consideration to Place-specific Values in Participatory Forest Planning. Silva Fenn. 2011, 45, 253–265, doi:10.1016/S0378-1127(99)00204-2.

• Mustajoki, J.; Saarikoski, H.; Marttunen, M.; Ahtikoski, A.; Hallikainen, V.; Helle, T.; Hyppönen, M.; Jokinen, M.; Naskali, A.; Tuulentie, S.; et al. Use of decision analysis interviews to support the sustainable use of the forests in Finnish Upper Lapland. J. Environ. Manag. 2011, 92, 1550–1563, doi:10.1016/j.jenvman.2011.01.007.

• Diaz-Balteiro, L.; Voces, R.; Romero, C. Making Sustainability Rankings Using Compromise Programming. An Application to European Paper Industry. Silva Fenn. 2011, 45, 761–773.

• Kijazi, M.H.; Kant, S. Social acceptability of alternative forest regimes in Mount Kilimanjaro, Tanzania, using stakeholder attitudes as metrics of uncertainty. For. Policy Econ. 2011, 13, 242–257, doi:10.1016/j.forpol.2010.12.001.

• Kühmaier, M.; Stampfer, K. Development of a Multi-Criteria Decision Support Tool for Energy Wood Supply Management. Croat. J. For. Eng. 2012, 33, 181–198.

• Jalilova, G.; Khadka, C.; Vacik, H. Developing criteria and indicators for evaluating sustainable forest management: A case study in Kyrgyzstan. For. Policy Econ. 2012, 21, 32–43, doi:10.1016/j.forpol.2012.01.010.

• Garmendia, E.; Gamboa, G. Weighting social preferences in participatory multi-criteria evaluations: A case study on sustainable natural resource management. Ecol. Econ. 2012, 84, 110–120, doi:10.1016/j.ecolecon.2012.09.004.

• Khadka, C.; Vacik, H. Use of multi-criteria analysis (MCA) for supporting community forest management. iForest 2012, 5, 60–71, doi:10.3832/ifor0608-009.

• Malovrh, S.P.; Groselj, P.; Stirn, L.Z.; Krc, J. The Present State and Prospects of Slovenian Private Forest Owners’ Cooperation within Machinery Rings. Croat. J. For. Eng. 2012, 33, 105–114.

• Rantala, M.; Hujala, T.; Kurttila, M. Measuring and Monitoring Socio-Cultural Sustainability in the Action of Forest Biodiversity Cooperation Networks. Silva Fenn. 2012, 46, 441–459, doi:10.14214/sf.52.

• Myllyviita, T.; Holma, A.; Antikainen, R.; Lähtinen, K.; Leskinen, P. Assessing environmental impacts of biomass production chains—application of life cycle assessment (LCA) and multi-criteria decision analysis (MCDA). J. Clean. Prod. 2012, 29–30, 238–245, doi:10.1016/j.jclepro.2012.01.019.

• Nordstrom, E.-M.; Öhman, K.; Eriksson, L.O. Approaches for Aggregating Preferences in Participatory Forest Planning—An Experimental Study. Open For. Sci. J. 2012, 5, 23–32, doi:10.2174/1874398601205010023.

• Zhang, Z.; Sherman, R.; Yang, Z.; Wu, R.; Wang, W.; Yin, M.; Yang, G.; Ou, X. Integrating a participatory process with a GIS-based multi-criteria decision analysis for protected area zoning in China. J. Nat. Conserv. 2013, 21, 225–240.

• Grošelj, P.; Zadnik Stirn, L. Between compromise and consensus in group decisions in forest management. Sumar. List 2013, 7–8, 403–410.

• Maroto, C.; Segura, M.; Ginestar, C.; Uriol, J.; Segura, B. Sustainable Forest Management in a Mediterranean region: Social preferences. For. Syst. 2013, 22, 546–558, doi:10.5424/fs/2013223-04135.

• Kukrety, S.; Jose, S.; Alavalapati, J.R.R. Exploring Stakeholders’ Perceptions with Analytic Hierarchy Process- A Case Study of Red Sanders (Pterocarpus santalinus L.) Restoration in India. Restor. Ecol. 2013, 21, 777–784, doi:10.1111/j.1526-100X.2012.00938.x.

• Fontana, V.; Radtke, A.; Fedrigotti, V.B.; Tappeiner, U.; Tasser, E.; Zerbe, S.; Buchholz, T. Comparing land-use alternatives: Using the ecosystem services concept to define a multi-criteria decision analysis. Ecol. Econ. 2013, 93, 128–136, doi:10.1016/j.ecolecon.2013.05.007.

• Keča, L.; Keča, N.; Rekola, M. Value chains of Serbian non-wood forest products. Int. For. Rev. 2013, 15, 315–335, doi:10.1505/146554813807700164.

• Costa, R.; Menichini, T. A multidimensional approach for CSR assessment: The importance of the stakeholder perception. Expert Syst. Appl. 2013, 40, 150–161, doi:10.1016/j.eswa.2012.07.028.

• Nordström, E.-M.; Holmström, H.; Öhman, K. Evaluating continuous cover forestry based on the forest owner’s objectives by combining scenario analysis and multiple criteria decision analysis. Silva Fenn. 2013, 47, 1–22, doi:10.14214/sf.1046.

• Garcia-Gonzalo, J.; Palma, J.H.N.; Freire, J.P.A.; Tome, M.; Mateus, R.; Rodriguez, L.C.E.; Bushenkov, V.; Borges, J.G. A decision support system for a multi stakeholder’s decision process in a Portuguese National Forest. For. Syst. 2013, 22(2), 359–373, doi:10.5424/fs/2013222-03793.

• Enache, A.; Kühmaier, M.; Stampfer, K.; Ciobanu, V.D. An Integrative Decision Support Tool for Assessing Forest Road Options in a Mountainous Region in Romania. Croat. J. For. Eng. 2013, 34, 43–60.

• Srdjevic, Z.; Lakicevic, M.; Srdjevic, B. Approach of Decision Making Based on the Analytic Hierarchy Process for Urban Landscape Management. Environ. Manag. 2013, 51, 777–785, doi:10.1007/s00267-012-9990-7.

• Sedmák, R.; Fabrika, M.; Bahýľ, J.; Pôbiš, I.; Tuček, J. Application of simulation and optimization tools for developing forest management plans in the Slovak natural and management conditions. Implement Dss Tools Into For. Pract. 2013, 139–152.

• Kühmaier, M.; Kanzian, C.; Stampfer, K. Identification of potential energy wood terminal locations using a spatial multicriteria decision analysis. Biomass and Bioenergy 2014, 66, 337–347, doi:10.1016/j.biombioe.2014.03.048.

• Derak, M.; Cortina, J. Multi-criteria participative evaluation of Pinus halepensis plantations in a semiarid area of southeast Spain. Ecol. Indic. 2014, 43, 56–68, doi:10.1016/j.ecolind.2014.02.017.

• Porto, M.; Correia, O.; Beja, P. Optimization of Landscape Services under Uncoordinated Management by Multiple Landowners. PLoS ONE 2014, 9, 1–16, doi:10.1371/journal.pone.0086001.

• Aldea, J.; Martínez-Peña, F.; Romero, C.; Diaz-Balteiro, L. Participatory Goal Programming in Forest Management: An Application Integrating Several Ecosystem Services. Forests 2014, 5, 3352–3371, doi:10.3390/f5123352.

• Lakicevic, M.; Srdjevic, Z.; Srdjevic, B.; Zlatic, M. Decision making in urban forestry by using approval voting and multicriteria approval method (case study: Zvezdarska forest, Belgrade, Serbia). Urban For. Urban Green. 2014, 13, 114–120, doi:10.1016/j.ufug.2013.11.001.

• Nousiainen, I.; Tahvanainen, L.; Tyrväinen, L. Landscape in farm-scale land-use planning. Scand. J. For. Res. 1998, 13, 477–487, doi:10.1080/02827589809383009.

• Lundström, J.; Öhman, K.; Rönnqvist, M.; Gustafsson, L. How reserve selection is affected by preferences in Swedish boreal forests. For. Policy Econ. 2014, 41, 40–50, doi:10.1016/j.forpol.2013.12.007.

• Bilbao-Terol, A.; Jiménez, M.; Arenas-Parra, M. A group decision making model based on goal programming with fuzzy hierarchy: An application to regional forest planning. Ann. Oper. Res. 2016, 245, 137–162, doi:10.1007/s10479-014-1633-3.

• Uribe, D.; Geneletti, D.; F. del Castillo, R.; Orsi, F. Integrating Stakeholder Preferences and GIS-Based Multicriteria Analysis to Identify Forest Landscape Restoration Priorities. Sustainability 2014, 6, 935–951, doi:10.3390/su6020935.

• Nikodinoska, N.; Mattivi, M.; Notaro, S.; Paletto, A. Stakeholders’ appraisal of biomass-based energy development at local scale. J. Renew. Sustain. Energy 2015, 7, 1–14, doi:10.1063/1.4916654.

• Acosta, M.; Corral, S. Participatory Multi-Criteria Assessment of Forest Planning Policies in Conflicting Situations: The Case of Tenerife. Forests 2015, 6, 3946–3969, doi:10.3390/f6113946.

• Mizaras, S., Mizaraite, D. Integrated Economic and Social Approaches for the Evaluation of Forest Management Sustainability: The Case of Lithuania. Balt. For. 2015, 21, 96–105.

• de Sousa Xavier, A.M.; Costa Freitas, M. de B.; de Sousa Fragoso, R.M. Management of Mediterranean forests—A compromise programming approach considering different stakeholders and different objectives. For. Policy Econ. 2015, 57, 38–46, doi:10.1016/j.forpol.2015.03.012.

• Rico, M.; González, A. Social participation into regional forest planning attending to multifunctional objectives. For. Policy Econ. 2015, 59, 27–34, doi:10.1016/j.forpol.2015.05.007.

• Kašpar, J.; Marušák, R.; Hlavatỳ, R. A Forest Planning Approach with Respect to the Creation of Overmature Reserved Areas in Managed Forests. Forests 2015, 6, 328–343, doi:10.3390/f6020328.

• Garcia-Gonzalo, J.; Bushenkov, V.; McDill, M.E.; Borges, J.G. A Decision Support System for Assessing Trade-Offs between Ecosystem Management Goals: An Application in Portugal. Forests 2015, 6, 65–87, doi:10.3390/f6010065.

• Segura, M.; Maroto, C.; Belton, V.; Ginestar, C. A New Collaborative Methodology for Assessment and Management of Ecosystem Services. Forests 2015, 6, 1696–1720, doi:10.3390/f6051696.

• Tamima, U. Performance evaluation of forest co-management: A case study of Chunati Wildlife Sanctuary, Bangladesh. J. For. Res. 2016, 27, 853–861, doi:10.1007/s11676-015-0198-6.

• Enache, A.; Ciobanu, V.D.; Stampfer, K. Stakeholders’ perceptions regarding evaluation criteria of forest road options in romania. Environ. Eng. Manag. J. 2015, 14, 1409–1421.

• KC, B.; Stainback, G.A.; Rayens, K. Comparison of Three Major Forest Types of Mid Hills Region of Nepal for Conservation and Local Benefits. Small-scale For. 2015, 14, 479–491, doi:10.1007/s11842-015-9299-5.

• Sarkar, S.; Dyer, J.S.; Margules, C.; Ciarleglio, M.; Kemp, N.; Wong, G.; Juhn, D.; Supriatna, J. Developing an objectives hierarchy for multicriteria decisions on land use options, with a case study of biodiversity conservation and forestry production from Papua, Indonesia. Environ. Plan. B Plan. Des. 2016, 0, 1–22, doi:10.1177/0265813516641684.

• Diaz-Balteiro, L.; Alfranca, O.; González-Pachón, J.; Romero, C. Ranking of industrial forest plantations in terms of sustainability: A multicriteria approach. J. Environ. Manag. 2016, 180, 123–132, doi:10.1016/j.jenvman.2016.05.022.

• Dwivedi, P.; Jagadish, A.; Schelhas, J. Perceptions of Stakeholder Groups about the Participation of African American Family Forest Landowners in Federal Landowner Assistance Programs. J. For. 2016, 114, 89–96, doi:10.5849/jof.14-152.

• Ahmadi Sani, N.; Babaie Kafaky, S.; Pukkala, T.; Mataji, A. Integrated use of GIS, remote sensing and multi-criteria decision analysis to assess ecological land suitability in multi-functional forestry. J. For. Res. 2016, 27, 1127–1135, doi:10.1007/s11676-016-0242-1.

• Nilsson, H.; Nordström, E.-M.; Öhman, K. Decision Support for Participatory Forest Planning Using AHP and TOPSIS. Forests 2016, 7, 100, doi:10.3390/f7050100.

• Tikkanen, J.; Hujala, T.; Kurttila, M. Potentials of collaborative decision support methodologies to enhance reconciliation of competing forest uses-An action research on Regional Forest Programme in Finland. Land Use Policy 2016, 55, 61–72, doi:10.1016/j.landusepol.2016.03.021.

• Silva, V.A.M.; de Mello, K.; Vettorazzi, C.A.; Ribeiro da Costa, D.; Valente, R.A. Priority areas for forest conservation, aiming at the maintenance of water resources, through the multicriteria evaluation. Rev. Árvore 2017, 41, 1–10.

• Wang, Z.; Du, X. Monitoring Natural World Heritage Sites: Optimization of the monitoring system in Bogda with GIS-based multi-criteria decision analysis. Environ. Monit. Assess. 2016, 188, 384, doi:10.1007/s10661-016-5391-3.

• Diaz-Balteiro, L.; Alfranca, O.; Bertomeu, M.; Ezquerro, M.; Giménez, J.C.; González-Pachón, J.; Romero, C. Using quantitative techniques to evaluate and explain the sustainability of forest plantations. Can. J. For. Res. 2016, 46, 1157–1166, doi:10.1139/cjfr-2015-0508.

• Felardo, J.; Lippitt, C.D. Spatial forest valuation: The role of location in determining attitudes toward payment for ecosystem services policies. For. Policy Econ. 2016, 62, 158–167, doi:10.1016/j.forpol.2015.10.004.

• Bayram, B.Ç.; Üçüncü, T. A Case Study: Assessing the Current Situation of Forest Products Industry in Taşköprü Through SWOT analysis and analytic hierarchy process. J. For. Fac. 2016, 16, 510–514.

• Borges, J.G.; Marques, S.; Garcia-Gonzalo, J.; Ur Rahman, A.; Bushenkov, V.; Sottomayor, M.; O.; Carvalho, P.; Nordström, E.-M. A Multiple Criteria Approach for Negotiating Ecosystem Services Supply Targets and Forest Owners’ Programs. For. Sci. 2017, 63, 49–61, doi:10.1186/1752-0509-2-75.

• Grošelj, P.; Hodges, D.G.; Zadnik Stirn, L. Participatory and multi-criteria analysis for forest (ecosystem) management: A case study of Pohorje, Slovenia. For. Policy Econ. 2015, 71, 80–86, doi:10.1016/j.forpol.2015.05.006.

• Franco-Maass, S.; Burrola-Aguilar, C.; Arana-Gabriel, Y.; García-Almaraz, L.A. A local knowledge-based approach to predict anthropic harvesting pressure zones of wild edible mushrooms as a tool for forest conservation in Central Mexico. For. Policy Econ. 2016, 73, 239–250, doi:10.1016/j.forpol.2016.09.020.

• Focacci, M.; Ferretti, F.; De Meo, I.; Paletto, A.; Costantini, G. Integrating stakeholders’ preferences in participatory forest planning: A pairwise comparison approach from Southern Italy. Int. For. Rev. 2017, 20, 1–10, doi:10.1505/146554817822272349.

• Derero, A.; Worku, A.; Kassa, H. Genecological zones and selection criteria for natural forest populations for conservation: The case of Boswellia papyrifera in Ethiopia. J. For. Res. 2017, 29, 515–524, doi:10.1007/s11676-017-0466-8.

• Brudermann, T.; Sangkakool, T. Green roofs in temperate climate cities in Europe—An analysis of key decision factors. Urban For. Urban Green. 2017, 21, 224–234, doi:10.1016/j.ufug.2016.12.008.

• Valente, R.A.; Petean, F.C.S.; Vettorazzi, C.A. Multicriteria Decision Analysis for Prioritizing Areas for Forest Restoration. Cerne 2017, 23, 53–60, doi:10.1590/01047760201723012258.

• Uhde, B.; Heinrichs, S.; Stiehl, C.R.; Ammer, C.; Müller-Using, B.; Knoke, T. Bringing ecosystem services into forest planning—Can we optimize the composition of Chilean forests based on expert knowledge? For. Ecol. Manag. 2017, 404, 126–140, doi:10.1016/j.foreco.2017.08.021.

• Gumus, S. An Evaluation of Stakeholder Perception Differences in Forest Road Assessment Factors Using the Analytic Hierarchy Process (AHP). Forests 2017, 8, 165, doi:10.3390/f8050165.

Figures and Tables

Figure 1. Number of articles published.

Figure 2. Criteria selection method. 1 SWOT analysis: Strengths, Weaknesses, Opportunities, Threats.

Figure 3. Number of criteria used.

Figure 4. Number of decision-makers (DMs) in each interaction.

Figure 5. Number of groups in each study case.

Table 1

Percentages of GDM and multicriteria techniques most used in different reviews analysed.

Review	Authors	Year	%GDM	MCDM
[5]	Diaz-Balteiro and Romero	2008	20	AHP 1
[12]	Ananda and Herath	2009	10	MAUT 2
[10]	Segura et al.	2014	30	AHP
[7]	Udhe et al.	2015	58	AHP
[8]	Ezquerro et al.	2016	13	AHP-ANP 3/MAUT
[4]	Belavenutti et al.	2018	15	GP 4

¹ AHP: Analytical Hierarchy Process; ² MAUT: Multiattribute Utility Theory; ³ ANP: Analytic Network Process; ⁴ GP: Goal Programming.

Table 2

Article selection.

Articles identified using the Web of Science database	1226	Reasons for exclusion	No MCDM = 559Reviews = 40Duplicates = 294
Preselected	333	Reasons for exclusion	No case study = 12No MCDM and GDM = 169No download = 10No English = 3No forestry = 5

Table 3

Topic of the problem analysed.

Problem Topic	Papers Included
Forest management	78
Sustainability	29
Industrial forest plantations	13
Forest conservation	12
Ecosystem services	7
Social problems	7
Biodiversity	6
Environmental	4
Roads/infrastructures	3
Climate change	3
Urban problems	3
Restoration	2

Table 4

Participation methods.

Group Decision-Making Techniques	Papers Included	MCDM Method Employed as GDM Techniques	Papers Included	Interaction Method	Papers Included
Delphi method	3	Discrete Methods	196	Questionnaires	37
Ordinal Information		Outranking methods		Survey	36
Ranking	33	PROMETHEE	6	Software	30
Voting/Scoring	14	NAIADE	1	Interview	24
Approval voting	8	ELECTRE	1	Discussion	8
Borda	6	Hierarchical Methods		Group work	4
Cardinal Information		AHP	78	Meetings	6
Pairwise comparison	89	ANP	1	Field trips	2
Rating method	6	Average Optimisation		Brainstorming	1
Condorcet	5	WAM	76
		MAUT/MAVT	21
		HERO	6
		SMART	5
		SMAA	1
		Continuous Methods	20
		Goal programming	11
		Compromise programming	7
		Multiobjective programming	2

Table 5

Type of preferences.

Preferences	Papers Included
Individual	127
Collective	13

Table 6

Terminology employed.

Term	Papers Included
Decision-maker	98
Stakeholder	86
Expert	62
Respondent	4
Interest group	3

Table 7

Type of stakeholders selected.

Type of Stakeholder	Papers Included
Government—public administration members	51
Private owners	43
Environmentalists	37
Local stakeholders	34
Experts	22
Scientists	22
Forest industry members	21
Technicians	19
NGOs 1	19
Academics	17
Farmers	14
Hikers or recreationists	14
Tourists	13
Hunters	7
Business owners	6
Students	5

¹ NGOs: non-governmental organisations.

Table 8

Nature of case studies (%).

Type of Problem	% Case Studies
Illustrative examples, research projects, pilots	61.2
Real world problems	15.7
Not applicable	23.1