Evaluating the Potential of MAMCA as a Framework for Stakeholder Engagement during the Setup of Energy Communities

Abstract

The current energy transition is characterized by a high level of consumer and prosumer involvement. Energy communities (ECs) are instruments that fit into this trend, as they organize the collective and mainly citizen-driven exchange of clean energy. Most stakeholder engagement research for ECs focuses on one aspect such as awareness raising or deployment acceptance. Hitherto, no specific research has been conducted on a participatory approach that can be applied throughout all the phases of an EC setup and for different purposes. In our study, we determine how the Multi-Actor Multi-Criteria analysis (MAMCA) methodology can be used as an engagement tool for this purpose. By linking potential engagement goals and their connected tools and techniques to the corresponding MAMCA step, we have created a multi-layered practical framework that can be used by all types of EC initiators for stakeholder engagement throughout the setup of a new EC. As this practical tool stimulates solutions that cater better to stakeholder needs, it can contribute to smoother deployment and an associated increase in ECs in the general system. A theoretical evaluation and a performed case study demonstrate the utility of the methodology that is developed in this paper.

1. Introduction

Transitioning to a sustainable energy system is integral to the general sustainable development agenda [1]. This means not only putting a brake on the current climate change rate by emitting fewer greenhouse gasses but also implementing it in an inclusive way that enables everyone to play an active part in it [2,3].

The energy transition is characterized by, among other things, a shift from large, centralized assets to a large amount of decentralized energy production entities, the active transition of former energy consumers into prosumers, and a democratization of the overall energy system by shifting more decision and managing power over to these local prosumers [4].

New forms of organization and management of energy flows are emerging, with ‘energy communities’ (ECs) as one of the media through which local energy exchange can be formalized. They make it easier for different types of actors to jointly invest in (renewable) energy assets. The EC concept was introduced into the European Union legislation through the ‘Clean energy for all Europeans’ package, with definitions for citizen energy communities and renewable energy communities [5,6]. It is stated that ECs are noncommercial legal entities that can consume, provide, and manage their own energy locally and can offer energy services, but only with open, democratic participation and within a governance structure that provides environmental and social benefits for the community.

The ‘community’ aspect is at the heart of it, which means that an EC is defined by the cooperation of its members. Although this can take multiple legal forms (e.g., that of a cooperative, an association, or a nonprofit organization), without active collaboration between stakeholders the creation of an EC is not possible. The necessary empowerment of various stakeholders in the energy system, which forms a key aspect of ECs, can only be achieved through active involvement [7]. This means that appropriate participation and engagement strategies are essential, not only during the implementation of an EC but especially during the preparation, design, and setup. Without proper knowledge of community energy and its potential advantages and disadvantages, stakeholders will be hesitant to participate in an EC initiative, which is why awareness-raising engagement efforts from the very start of the project are essential [8]. The forms an EC can take are varied, and there is a large amount of (detailed) components around which a decision has to be made, such as the type and amount of assets that will be installed and the preferred energy trading form [9]. Additionally, the involved stakeholders often have different needs with respect to the end design. Therefore, stakeholder engagement during the decision-making phase is crucial to reach a widely supported result.

However, it can be noted that most participatory initiatives that are set up for ECs mainly focus on only one phase in the process, such as awareness raising at the start or deployment acceptance at the end. Some studies have set up interactive initiatives to raise general awareness on (community) renewable energy topics [10,11], while others center around the design of a community energy setup [12,13] or stimulating the uptake of an already developed EC initiative [14]. Nevertheless, none of the described approaches are all-encompassing, in that they represent a stakeholder engagement method that can be applied throughout the different stages of the EC setup process. The development of such a framework is, therefore, the goal of our study.

For participatory decision-making processes for complex problems with various stakeholders (preferences) Multi-Criteria Decision Analysis or Decision Making (MCDA/MCDM, hereafter: MCDA) methods are regularly used, with varying levels of potential stakeholder involvement [15]. One derivative, the existing Multi-Actor Multi-Criteria Analysis (MAMCA) methodology, focuses on consensus-building discussions and bringing into focus different viewpoints in cases where diverse stakeholders have varied objectives. It has been used as a stakeholder engagement tool in multiple other research domains. Therefore, we have chosen to explore the usefulness of this method in an EC context by further expanding and applying it to set up ECs in a participative way. The goal of our study is to present a framework for stakeholder engagement during the full setup of ECs, based on MAMCA, and test its applicability. The resulting framework can serve as a practical tool for future EC initiators (public or commercial institutions as well as bottom-up initiatives), offering the necessary instruments to actively involve all relevant stakeholders in the EC design. This guarantees a solution that caters better to the needs of the stakeholders, leading to wider support and smoother deployment. This way, the framework can contribute to an increase in ECs and an associated rise in renewable energy (RE) in the general system.

Depending on the specific engagement goals and the phase of the setup process, corresponding tools are linked to the MAMCA steps. This multi-layered practical framework is presented in Section 3. To test the usefulness of this MAMCA framework and determine whether it can lead to a deeper involvement of stakeholders and widely supported end results, it was first theoretically evaluated and then applied in a Spanish case study. The approach and results are described in Section 4 and Section 5.

2. Theoretical Background

2.1. MCDA/M and MAMCA

Given that multiple potential designs need to be assessed in an EC setup process, and various stakeholder objectives (or criteria) need to be taken into account, MCDA can be considered a potentially suitable technique to assist the decision-making process. MCDA comes in various forms and considers multiple criteria at once to evaluate and rank alternative outcomes in a transparent way. In many applications, it involves public participation [16]. The results can support decision making or consensus building in group discussions prior to it [17]. The selection of which specific MCDA method to use depends on the context [18,19]. For example, Analytic Hierarchy Process (AHP) has been used in stakeholder engagement processes that take on transport problems [20,21], the Analytic Network Process (ANP) has been used to actively involve stakeholders in urban planning decision making [22], and the Preference Ranking Organization Method for Enrichment Evaluations (PROMETHEE) was applied before in a participative process for the assessment of future renewable energy scenarios [23].

In our study, the Multi-Actor Multi-Criteria Analysis (MAMCA) framework was used as a basis because it allows for the inclusion of heterogeneous groups of stakeholders and their varied objectives within the MCDA. With MAMCA, the EC design alternatives can be evaluated for the objectives of each of the stakeholder groups separately [24]. The results are visualized side by side without attributing an importance weight to the various stakeholders. Consensus forming for the (group) decision making is hence conducted using a discussion based on the evaluation results and not in the prior MCDA step (such as, for example, the objectives selection).

MAMCA has been used to actively engage stakeholders in participatory decision-making processes in multiple research domains such as, for example, sustainable urban freight transport initiatives [25,26,27,28], construction logistics [29], and the implementation of sustainable development goals [30]. By using different approaches for each step, the method can be adapted or extended depending on the level of stakeholder involvement that is deemed suitable for a specific context or moment in time.

MAMCA consists of several steps (see Figure 1):

• Step 1: alternatives or scenario definition.

• Step 2: determining the stakeholders and their criteria.

• Step 3: criteria weighting.

• Step 4: linking measurable indicators to the criteria.

• Step 5: alternatives evaluation.

• Step 6: evaluation results and ranking with consensus building.

This process is iterated if necessary [31]. For each of these steps, various approaches can be used to provide the necessary input.

2.2. Evaluation of Engagement Initiatives

Throughout the decades, multiple authors have proposed various ways to categorize stakeholder engagement initiatives. Most of these categorizations entail a classification based on the intensity of involvement or actual power of the stakeholders and are linked to specific tools and techniques. Stakeholders here are defined as the actors who are affected by or can affect a decision in the matter [32]. Some of the categorizations are based on the objectives of the initiator when involving stakeholders in its process. Glass, for example, determines five potential objectives and matches engagement techniques with each of them [33]. Others identify different engagement purposes [34,35,36,37,38], sometimes linked to specific techniques, or focus on types of strategies and interpretations [39,40,41].

The most cited and influential model of public engagement is undoubtedly the ‘Ladder of Citizen Participation’, which was developed by Sherry Arnstein in 1969. The central idea behind this model is that citizen engagement in a democratic process requires the redistribution of power for it to be considered a genuine engagement effort. Only by power redistribution can the citizens who are currently excluded from decision processes achieve reforms that benefit them. When the reform is not part of an engagement process, no meaningful changes can be made and frustration arises [42]. Arnstein’s ladder describes three general forms of participatory citizen power in democratic decision making: nonparticipation (or no power), tokenism (or false power), and actual citizen power (Figure 2). A further subdivision is made into eight levels of citizen agency and control. The higher up on the ladder, the more actual power it represents in determining the end product. Manipulation and therapy are considered forms of nonparticipation. Informing, consultation, and placation can have their merit in a process but often do not provide the participating citizens with actual power. Delegated power and citizen control are considered the two forms of actual citizen empowerment at the top of the ladder. The model does not only apply to citizens and can also be a useful inspiration for engagement strategies for other relevant stakeholders. Initially set up as a criticism of many of the common engagement practices at the time, the format can also be regarded as an overview of different potential levels of engagement that might have their merit for specific purposes, rather than a strictly hierarchical ranking of forms of engagement.

As a conceptual tool, this Ladder of Participation influenced many later models of democratic decision making and citizen power building [43,44]. A well-known example is the ‘Wheel of Participation’ that was developed by Davidson and aims to provide a framework to link suited qualitative participation techniques to the specific objective of the engagement initiative [45]. The four main identified objectives are information, consultation, participation, and empowerment (Figure 3). Attention is drawn to the fact that there are several approaches to fulfill each of these objectives (the darker the color of the approach in the figure, the higher the level of stakeholder involvement and power it yields).

Contrary to Arnstein’s ladder, this wheel does not suggest a ranking between its four main components, suggesting the aim is to strive for a ‘higher’ engagement objective. This type of normativity is only expressed within the presented potential approaches of each of the components.

Both models can be regarded as useful tools to categorize and evaluate stakeholder engagement initiatives and therefore are used as a testing instrument to evaluate our framework.

3. MAMCA Stakeholder Engagement Framework for the Setup of ECs

Since we have established that stakeholder engagement is a crucial aspect for the creation of supported ECs, but no detailed methodology has been developed for the specific purpose of actively involving stakeholders throughout the full EC setup process, we propose our own framework.

MAMCA was chosen as the framework base for its promising premise of being able to take into account multiple potential outcomes, bring together and visualize varied actor views, and give a voice to all stakeholders involved. The methodology is further customized for the topic by adding specific tools to each of the MAMCA steps depending on the phase of the project and the corresponding engagement goal.

3.1. EC Process Phases and Accompanying Engagement Goals

The outcome of an engagement approach is sensitive to the way the process is conducted. A range of methods can be used, each with its own strengths and weaknesses. Good practice dictates that a method, and especially the level of engagement required, needs to be tailored to the specific context [46]. Various authors have also stressed the importance of linking engagement objectives to specific tools to increase the probability of a supported outcome [33,35,39]. Additionally, the stage of the process at which a method is used is an important consideration, since all phases have different outcomes and require specific engagement approaches that correspond with their goal. An all-encompassing stakeholder engagement framework needs to provide tools for all stages.

Therefore, we first determined the three major phases we experienced within an EC setup process, from the moment the initial idea is formed to start an EC until the actual long-term deployment of it. Next, we identified the potential goals of an engagement initiative for each phase (see Figure 4).

In the first phase, an initiator proposes to start an EC in a certain area, gathers information, and determines the potential general advantages that such an initiative could bring. In order to get the project started, potential stakeholders need to be enthused. Since common knowledge of ECs is generally limited, efforts need to be made to raise knowledge and awareness of what an EC entails. Before an actual setup with stakeholder engagement can be initiated, the interest of these actors needs to be sparked to motivate them to cooperate throughout the process.

In this awareness-raising phase (A), the main aim is to raise stakeholder knowledge of what an EC encompasses, make them aware of its potential benefits and barriers, and raise their interest in further involvement in the process.

In the second phase, it is determined what the most fitting and supported EC form is that can be deployed in a specific situation. All relevant stakeholders have already been determined and are willing to cooperate in providing their active input. The engagement efforts in this phase are geared towards involving these stakeholders in the process of designing the EC, translating their input into EC forms that fit their needs and wants, and deciding upon the one that will be implemented.

Within this design and setup phase (B) of an EC process, engagement initiatives can have the following goals:

• Gathering information on stakeholder needs to use as input for building potentially fitted EC designs;

• Building relevant EC designs (‘alternatives’) together with the stakeholders;

• Co-decision making on which EC design will be implemented and under what preconditions.

The third phase consists of the actual deployment of the EC after the implementation decision has been made. Since ECs are active cooperation forms, with democratic decision making and managing demands, stakeholders need to be kept engaged throughout the long-term deployment. Building further on the ‘community’ aspect is very important in this phase. Since ECs are a relatively new concept with few examples that have been running for multiple years or projects in which engagement tools have been tested out in the short term, we decided to mainly focus on the first two phases in this paper. In this implementation and community building phase (C), the goal of engagement initiatives is to let the EC members take control of the implementation and guarantee the continuation of the EC by strengthening the community feeling.

3.2. The MAMCA Framework for the Setup of ECs

Multiple authors have pointed out that when goal-specific engagement tools are used, the chances of a supported result increase [33,35,39]. In the developed framework, we therefore coupled existing engagement tools to each MAMCA step, linked to the corresponding goal as described above. The tools were all selected from the papers with engagement method reviews for which Reed gives an extensive overview [46]. The ones we believed to be potentially relevant in an EC setup context were added to the framework.

Table 1 gives an overview of the framework with an explanation of what each of the engagement tools entails and how they can be used in a specific EC context. Figure 5 visualizes this information in a schematic way.

The whole MAMCA process, with all the steps, can also serve as an awareness-raising tool in the first project phase. By guiding stakeholders through the different steps in a “mock” project, they learn about all the aspects that need to be taken into account, are incentivized to think about their own objectives, and get a first taste of what EC cooperation entails.

4. Methodology for the Evaluation of the Framework

The potential of the proposed MAMCA framework is evaluated in two ways: through a theoretical analysis based on existing stakeholder engagement theories and on its practical usability through a case study application.

First, it is tested whether all relevant engagement forms that are identified in the Ladder and Wheel of Participation by Arnstein and Davidson, respectively, can be linked to the identified engagement goals that are covered by the MAMCA framework. Then, it is identified how tools that comply with the defined engagement form can fulfill the engagement goal. This allows for assessing whether the framework offers the necessary tools to realize different forms and intensities of participation. Additionally, the framework features are analyzed through the eight components of best practice stakeholder participation, as identified by Reed [46]. It is determined for each of these components how the framework can comply with it. The resulting application propositions present the potential added value of MAMCA as an engagement tool.

In parallel, a first practical evaluation of the framework was performed by applying it in a case study in the Spanish town of Relleu, as part of the European Horizon 2020 project RENAISSANCE that aimed to develop a community-driven and replicable approach to setting up ECs in different places under varied conditions. An overview of the general results of these MAMCA initiatives is described by Lode et al. [47]. In this paper, to determine the potential future utility of the developed framework as an engagement tool, the potential application of the steps and the tools themselves are studied in more detail through one specific case study.

For the Relleu case, four engagement goals from the framework were selected by the project initiator: awareness and knowledge raising, information gathering for EC design input, joint alternative EC design building, and decision making. A participatory MAMCA workshop was organized in March 2022. Ten residents from a local compound for which an EC initiative was envisioned were invited to join, representing five households. Linked to the predetermined engagement goals, different MAMCA step approaches, visualized in Figure 6, were implemented in the workshop. The initiative started with an information session for all the participants on EC opportunities and barriers, as well as technical boundary conditions and general information, to raise their knowledge on the topic. The rest of the workshop (EC design) was set up in an interactive way and also served as an awareness-raising tool.

For the goal ‘Information gathering for EC design input’, a survey, a discussion, and a thematic workshop exercise were used to determine the stakeholder objectives and weights (MAMCA step 2 and 3). The survey contained an expandable list of 22 potential objectives for joining an EC, out of which the participants needed to select the ones they deemed essential. Next, a discussion round was initiated in which every stakeholder group presented their objectives selection to the others and could adapt their choices based on the knowledge gained from the discussion. As a third sub-step, the MAMCA software tool [48], and, more specifically, the pairwise weighting component, was used by all stakeholder groups to attribute a weight to their objectives according to importance.

For the engagement goal ‘Joint alternative EC design building’, EC alternatives were designed (MAMCA step 1) using a planning charrette tool (a collaborative designing session in which the stakeholders drafted potential EC solutions). The participants were divided into two groups and were both provided with an expert assistant and guidelines for building an EC design. The guide consisted of eight different questions on aspects that make up an EC (e.g., ‘What type and amount of participants do you want to take part in the EC initiative’ and ‘What legal form should the EC take on’), to which one or more answers needed to be given by the participants. They had to discuss each question within their group to come up with a response that everyone agreed upon. Potential answers and a more detailed explanation of the options were provided by the expert. Based on the responses, multiple types of EC forms (‘alternatives’ or ‘scenarios’) were created for each group.

Finally, the engagement goal ‘Co-decision making’ was shaped by organizing an evaluation round (MAMCA step 5) and a consensus-building discussion to determine a generally supported preference ranking for the EC designs (MAMCA step 6). All stakeholders evaluated the EC alternatives by scoring them on each of their objectives using the Analytical Hierarchy Process (AHP) technique [49] through the online MAMCA software. Experts assisted with the scoring of the technical, legal, and environmental objectives. A resulting ranking of the EC alternatives for each of the stakeholders was then visualized. The individual preference rankings were examined in detail with each stakeholder group, and the causes of the preference differences between the groups were explained. Based on this knowledge, a consensus discussion was held to determine a general preference ranking that could be supported by all. The discussion also resulted in some minor adaptations to the original designs.

Figure 6 gives a schematic overview of the selected engagement goals and corresponding tools for Relleu. In our case study evaluation, we describe the application observations we made during the application of the tools. All findings that are relevant for future application, such as encountered problems, questions raised by the participants, aspects that worked well, and voiced concerns are listed for each tool used. This input was gathered through observation during the MAMCA engagement process and through informal conversations with the participants after the workshop. Additionally, a short survey was presented before and after the workshop in which the specific questions were raised to assess the fulfillment of the engagement goals and the awareness and knowledge-raising effects of the workshop. Some questions were the same in both surveys to evaluate whether participation influenced their answers. The pre- and post-workshop survey questions can be found in Appendix A and Appendix B, respectively.

5. Evaluation of the Framework

5.1. Theoretical Evaluation Based on Existing Stakeholder Engagement Principles

5.1.1. Link with the Ladder and Wheel of Participation

Depending on the goal of the engagement process, different MAMCA steps and tools can be used. The identified potential goals can be coupled to Arnstein’s different ladder rungs and Davidson’s wheel quarters, as illustrated in Table 2.

Since it is possible to link each engagement form as defined in the Ladder and Wheel of Participation by Arnstein and Davidson, respectively, to an identified EC engagement goal, it can be stated that the framework offers the necessary tools to realize different forms and intensities of participation. Depending on the way the framework is used, and which tools are applied, a different level of engagement can be reached. The forms of window-dressing participation that are criticized in the existing theories are also a point of attention when utilizing the proposed framework. By making a link with the Ladder and Wheel of Participation, we want to highlight that using our framework is no guarantee that these forms of nonparticipation are shunned, but it offers the necessary tools to avoid them.

5.1.2. Best Practice Principles for Stakeholder Engagement in EC Design

The design of a customized stakeholder engagement plan at the start of the project is often regarded as a prerequisite for an efficient engagement initiative [33,36]. The plan and process have an important influence on the end result and determine whether the stakeholders feel their concerns are reflected in the final EC design and decision-making. Based on a Grounded Theory Analysis of the literature on various existing engagement methods, Reed [46] identified eight features of best practice stakeholder participation:

• “Stakeholder participation needs to be underpinned by a philosophy that emphasizes empowerment, equity, trust, and learning;

• Where relevant, stakeholder participation should be considered as early as possible and throughout the process;

• Relevant stakeholders need to be analyzed and represented systematically;

• Clear objectives for the participatory process need to be agreed among stakeholders at the outset;

• Methods should be selected and tailored to the decision-making context, considering the objectives, type of participants, and appropriate level of engagement;

• Highly skilled facilitation is essential;

• Local and scientific knowledges should be integrated;

• Participation needs to be institutionalized.”

These can form the starting points when setting up the engagement plan for the EC design process. An evaluation of the proposed MAMCA framework against these features shows that it offers the potential to fulfill all eight of them when the engagement process is planned and mindfully put into practice. They are integrated into the following framework aspects, respectively:

• The framework offers the opportunity to collect and take into account the input of every stakeholder and conduct an open process. Depending on the predetermined goal, stakeholder empowerment can take on many forms and intensities. In every phase, stakeholders can be provided with all the educational material needed to understand all elements that are important for decision making and with tools that are usable and easily accessible to everyone. Throughout the process, multiple interaction opportunities are created between everyone involved, which stimulates mutual learning and understanding.

• Stakeholder engagement is foreseen from the very start and takes on varying forms throughout the different phases of the process. This leads to EC alternatives that capture the various interests at stake, as is the case in the plural planning tradition [50].

• In step 2 of the MAMCA framework, all relevant stakeholders are identified to be represented and receive an equal voice in the process. Heuninckx et al. [51] give an overview of potential stakeholder groups.

• The premise of the framework is that the process starts with defining the goal of the engagement activities to be able to define the necessary steps and tools. This has to be clearly communicated to the stakeholders involved. Figure 4 shows the potential engagement goals that are identified for an EC setup process.

• Depending on the engagement goal, the MAMCA framework offers appropriate tools that can be used, all with active stakeholder involvement at their base.

• The overall engagement process needs to be managed by an experienced and preferably external facilitator. Impartiality is essential to guarantee an open discussion in which every stakeholder gets an equal chance to be heard. In some of the specific tools that deal with the more technical and complex inherent aspects of an EC (in steps 1, 4, and 5), the assistance of an expert in the field is mentioned as a prerequisite.

• The preparation phase and steps 1 and 5 of the framework are centered around the integration of local and expert knowledge. Since EC design has a large technical component, expert input is essential in the process and provides technical modeling, legal, and social support to guide stakeholders to well-informed decision making.

• Making the MAMCA framework the backbone of an officially written down and agreed-upon engagement plan at the start of every EC setup process can help battle window-dressing forms of stakeholder participation.

5.2. Evaluation through Case Study Application

5.2.1. Case Study Specifications

Relleu is a village of approximately 1300 inhabitants in the hills near Alicante, Spain. Within the village, a newly built compound of 37 houses can be found, which is the subject of this case study. The inhabitants, who are a mixture of permanent and temporary residents, indicated a strong interest in renewable energy systems but admitted to having very little knowledge of the topic. The engagement initiative that was set up here had awareness and knowledge raising, as well as information gathering, design, and decision-making on potential ECs for the site, as a goal. Table 3 gives an overview of the engagement tools that were selected to be part of each step of the MAMCA framework and observations from the application of each tool.

5.2.2. Case Study Findings on the Framework Application

Before the workshop trust in the (unknown) initiator and the (unknown) topic of ECs was low among stakeholders. A first general workshop invitation did not result in many responses. When a local representative that was known by the residents was engaged, the willingness to participate rose, together with the number of incoming questions on ECs. By explaining that an information session with all basic information would be given before the workshop, these questions could be bundled together, and stakeholders were also less anxious to participate in an initiative with a topic they did not know much about. The presentation was essential to raise stakeholders’ confidence in participation and trust in the interactive part of the workshop.

Step 2 of MAMCA (criteria determination) took the most time because participants were still not confident in their acquired knowledge of ECs. Because of the complexity of the topic, a preset list of potential objectives (that can be expanded by the stakeholders) is necessary. All participants indicated that they had a limited list of objectives in their head, but by seeing all the presented options, they were reminded of elements they forgot or gained insight into potential benefits and threats they did not know about but that are, nonetheless, important for them. By letting them present their own selection to each other, discussions were started that had an impact on certain viewpoints. From this, we learned that it was necessary to organize a second survey round to be able to take into account changed insights.

The criteria weighting in step 3 was aided by the pairwise comparison feature in the MAMCA software. There are alternative weight allocation methods, such as Direct Rating [48], but the participants indicated that they found pairwise comparison the easiest. The resulting visuals were deemed easy to understand and helped in steps 5 and 6 to better understand certain outcomes. Figure 7 shows two examples of the visual depiction of the criteria weights. In the first case, the household placed high importance on having its energy bill reduced and finding a solution with heightened safety measures. The second visual shows that household 4 has different needs and is more concerned with having reliable energy provision and direct decision-making power over the energy system through direct participation and autonomy.

Step 1 of MAMCA was applied after step 3. Because there were no preset EC alternatives both organizers and participants felt they needed a clearer insight into the stakeholder objectives before being able to compile potential EC designs. Expert assistance was essential in this step because of the technical complexity of the topic, in which the participants had no experience. It was apparent that clear guidelines on all the elements that need to be addressed and how they can be addressed are also indispensable. At the start of this workshop step, participants said they had no idea how to deliver useful input for a planning charrette, and that they felt overwhelmed by all the aspects that need to be considered when designing an EC. The provided toolbox and expert assistance helped to structure the exercise and lead to a useful outcome. The fact that the participants were assembled into two groups and did not have to individually design their own EC alternatives was also assessed positively. This resulted in discussions and consensus-making earlier in the process, leading to fewer conflict situations and more mutual understanding in the following steps.

The identification of evaluation criteria for each of the stakeholder objectives needed to be completed beforehand (step 4). The participants mentioned that they lacked the expertise to conduct this themselves, and because of the varied nature of the objectives (social, environmental, technical, and financial), experts in various fields needed to be consulted, which is difficult to conduct on the spot. The experts were also asked to indicate the circumstances that have a positive or negative impact on the functioning of their criterion. This information was used in the step 5 evaluation and can be regarded as a positive as well as a negative aspect: the input provided reliable help for the stakeholders in their evaluation of the EC alternatives, but they also depended heavily on this input. Since most of the criteria could not be assessed in a quantitative way, there is a chance that the evaluation results were heavily influenced by the partially subjective insights of a limited number of experts whose information was not critically questioned by the participants. The Simple Multi-Attribute Rating Technique (SMART) evaluation method used [52] was evaluated positively by the users since it allowed for an easy-to-understand qualitative assessment on a 1–10 scale. The evaluation resulted in a preference ranking of the EC alternatives for all participating stakeholder groups, as visualized in Figure 8. This result graph shows that alternative preferences varied substantially among the participating households, with households 1, 3, and 4 having similar outcomes, household 2 sharing a least preferred scenario with them but having another top pick, and household 5 depicting a fully inverse preference ranking. This indicates that a consensus-building discussion and potential adjustments to the initial alternatives are necessary to attain a broadly supported outcome.

In MAMCA step 6, the consensus-building workshop, the visuals that were generated in step 5 and step 3 by the MAMCA software were very helpful. They allowed for a structured discussion, with a focus on the reasons behind differing stakeholder views. Different components of the EC design were discussed individually and led to minor adaptations of the alternatives that could be supported by all. By going into detail in the consensus-building discussion, conflicting viewpoints were easier to grasp and overcome. By giving the participants the opportunity to build on a consensus themselves, rather than having an external analyst draw conclusions, the support for the end result can be much stronger. All participants in Relleu indicated that they felt heard, and that they felt their opinions were taken into account in the final outcome. This led them to be enthusiastic about the outcome and to take implementation steps immediately after the workshop. A neutral moderator to structure the discussion was, however, necessary, since the focus of the conversations often shifted away from the initial point of attention.

5.2.3. Case Study Findings on the Effects of the Used Framework Tools on the Engagement Goals

After the workshop, the participants were questioned about the effects of the applied tools. All 10 participants indicated that their knowledge of joint renewable energy initiatives in general, as well as for Relleu specifically, had increased thanks to the workshop. This also raised their awareness of the benefits and their willingness to join or even set up an individual or joint energy initiative in their neighborhood. Both awareness-raising aspects were assessed positively: the information part was considered a good introduction to the topic, while the interactive exercise urged them to reflect on the details and their own points of view. All except one person indicated that it was the fact that a familiar local informed them about the project that convinced them to join the workshop.

Their estimation of the probability that most neighbors could come to an agreement to start a joint project was also significantly increased through the workshop, as more than half of the participants assessed the chance of reaching a consensus higher at the end. The mutual discussions and consensus forming during the exercises showed that all participants’ intentions were relatively similar, and all points of view could be mutually understood and taken into account. Everyone indicated that their awareness of other participants’ viewpoints had increased and that they felt that their input was taken into account and could have an impact on the end result. This building of trust and confidence through stakeholder engagement can also be considered as an aspect of awareness raising.

The objectives weighting and discussion of the results provided useful information for the EC design. Although financial drivers are often generally regarded as the main motivation for citizens to join an energy initiative, social as well as technical objectives such as energy independence, grid reliability, safety, and energy poverty reduction were all mentioned multiple times as essential objectives, sometimes over financial benefits. This was a useful new insight, not only for the participants but also for the project initiator and experts. Additionally, the mutual discussions widened the initial range of personal motivations and hence the boundary conditions for the design.

The participants mentioned that the EC building workshop allowed them to better understand the complexities of (the setup of) ECs because it let them become acquainted with all the decisions that have to be made (collectively) and the information that needs to be collected. The design results turned out to be more detailed than many EC alternatives that are proposed by experts in other EC setup projects [47], and since many aspects were already discussed in the exercise itself, it made the consensus building during the decision-making phase easier. Some participants indicated that expert assistance was necessary considering the complexity of the topic.

The results of the MAMCA evaluation in step 5 showed that the designed EC alternatives scored better on the objectives of almost all participants than a business-as-usual scenario. By making the participants part of the evaluation and visualizing all results, they were able to better understand the outcomes and also support them. Multiple participants indicated that the consensus-building discussion for the decision making went smoother than they expected before the workshop, and that this was aided by the collective design and evaluation steps.

Although they were scared off a bit at first by the complexity of the matter, the applied step-by-step approach, as well as the knowledge gained, made them motivated to start taking concrete action for a joint energy initiative as a result of the workshop (they immediately contacted solar power system installers), whereas before they had little intention to take this step.

6. Conclusions and Suggestions for Further Research

This paper has presented a stakeholder engagement framework for the setup of ECs, with MAMCA at its base and various practical tools linked to the engagement goals and associated MAMCA steps. To determine if the framework is a steady foundation for engagement approaches in future EC initiatives, a theoretical and practical evaluation was carried out.

Theoretically, the framework contains the necessary tools to carry out various types and intensities of stakeholder participation. It is, however, important that the engagement goals are clearly defined in a plan at the start of the project and monitored further on to avoid window-dressing forms of participation.

During a test in the presented case study, positive feedback on the effectiveness of the framework for reaching the defined engagement goals was obtained, and the active stakeholder support for an EC went up. The Relleu case showed us that uncertainty about other stakeholders’ motivations and a lack of knowledge on what an EC (process) contains are still major drawbacks for many people. When a better insight into both can be offered, motivations go up. The developed engagement format, with the MAMCA framework as a base, worked on this account and was evaluated positively by the participants. It had a significant effect on the awareness and knowledge of the participants regarding joint renewable energy initiatives in general and specifically regarding energy communities. This directly resulted in a greater willingness to take immediate action towards a joint local initiative. The engagement goals ‘Information gathering for design input’ and ‘Joint alternative EC design building’ were also attained in that the used engagement tools provided the necessary information for the design phase, and the participants indicated that they felt their input was taken into account during the design phase. Regarding the ‘decision making’ aim, the provided evaluation method and consensus-building discussion resulted in a preference ranking of EC alternatives that all stakeholders understood and agreed upon.

In order to make more determined conclusions on the effectiveness of the proposed framework, it needs to be applied in more test sites. The presented case study represents a common residential case in which opposition against renewable energy is low, but knowledge levels and willingness to take action are also low. Other conditions, in which stakeholders are less open to sustainable initiatives or have fewer means to join an EC project, may need an adapted approach. Not all tools that are proposed as part of the MAMCA framework for the setup of ECs were tested out in the Relleu case either. Additionally, it is advised to also test the framework in a case with a mixture of different types of stakeholders.

Because the pilot case, as well as many other EC initiatives, only recently entered the design phase of its EC establishment, no actual onsite implementation has taken place yet. The direct results of the engagement process during the setup can, therefore, be analyzed in terms of design outcomes and stakeholder satisfaction with the process, but the effects on the actual implementation cannot be determined yet. A long-term study is needed to determine the effect of engagement initiatives during the EC setup on the support of the final implemented EC.

Author Contributions

Conceptualization, S.H. and C.M.; methodology, S.H. and C.M.; validation, S.H.; formal analysis, S.H.; resources, S.H.; writing—original draft preparation, S.H.; writing—review and editing, S.H., C.M. and G.t.B.; visualization, S.H.; supervision, C.M. and G.t.B.; funding acquisition, T.C. All authors have read and agreed to the published version of the manuscript.

Data Availability Statement

No new data were created or analyzed in this study. Data sharing is not applicable to this article.

Conflicts of Interest

The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript; or in the decision to publish the results.

Footnotes

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Figures and Tables

Figure 1. The different steps of the MAMCA methodology. Source: adapted from [24].

Figure 2. Original illustration of Sherry Arnstein’s Ladder of Citizen Participation as published in 1969, with eight ladder rungs that describe three general forms of citizen power in democratic decision making. Source: adapted from [42].

Figure 3. The Wheel of Participation. Source: adapted from [45].

Figure 4. Engagement goals for EC phases.

Figure 5. Schematic of the MAMCA framework and the corresponding EC engagement tools per goal.

Figure 6. MAMCA framework. The engagement goals and the corresponding tools that were tested in the Relleu project are indicated in gray and blue, respectively.

Figure 7. Example of criteria weighting results of two participating households.

Figure 8. Stakeholder preference rankings of the EC alternatives.

Table 1

MAMCA framework with engagement tools for the setup of ECs.

MAMCA Framework Step	Engagement Goal	Engagement Tool	Additional Explanation of the Engagement Tool
Preparation	Awareness raising	Specific information session with Q&A	Potential stakeholders are invited to attend an interactive information session in which the workings of an EC in general are explained, as well as the potential benefits and barriers for their specific case.
		Involving local project representatives	Local interested stakeholders assist in contacting and informing other potential stakeholders.
	Information gathering for EC design input	Personal or public consultative meeting	When the stakeholders have no prior knowledge of ECs, information is provided and concerns are captured through a consultative meeting. This can be part of the information session.
Step 1(scenario definition)	Joint alternative EC design building	Design workshop	A design workshop can take on many forms, with different types of input from the stakeholders.
		Brainstorming session	An expert translates the input from the stakeholders on their objectives and concerns into design alternatives that can be jointly discussed.
		Planning charrette	Stakeholders build the alternative EC designs themselves, assisted by an expert, and take joint ownership of the developed solutions.
		Expert cooperation	Assistance from experts in the field should always be part of this engagement stage seeing the technical components of ECs.
	Awareness raising	“Mock” design building workshop	Workshop participants design their own EC, aided by experts and practical information tools, so they learn by doing about the different aspects of an EC and the questions that have to be answered.
Step 2(stakeholder and criteria determination)	Information gathering for EC design input	Survey on objectives and perceived barriers	A list of potential objectives is provided out of which the most relevant can be selected and added by the stakeholders.
		Thematic workshop	During the workshop, stakeholders can present and mutually discuss in detail what they want to get out of an EC and which doubts they have. The objectives selection is also discussed.
Step 3(criteria weighting)	Awareness raising	Thematic workshop	During the workshop the stakeholders explain their criteria to each other, to create mutual insight into potentially differing viewpoints.
	Information gathering for EC design input	Exercise with weighting of stakeholder objectives (criteria)	Each stakeholder attributes a weight to its selected objectives according to the importance.
Step 4(linking indicators to criteria)	Awareness raising	Exercise with weighting of stakeholder objectives (criteria)	After the exercise the results of each stakeholder are visualized and explained to each other. This creates mutual insight and potential understanding.
	Co-decision making	Expert advice	Experts in the field determine indicators for the selected stakeholder objectives that allow for the evaluation of the EC alternatives.
Step 5(scenario evaluation)	Co-decision making	MCDA as decision (support) method	The evaluation of the EC alternatives is carried out by the stakeholders based on how the alternatives score on their selected objectives. Input on technical evaluation components is provided by experts.
Step 6(evaluation results and ranking)	Co-decision making	Consensus building workshop	The EC alternatives ranking resulting from the MCDA forms the basis of a stakeholder discussion that leads to a possible adaptation of the alternatives and a (group) decision on the EC design that will be implemented.

Table 2

Engagement goals linked to Arnstein’s [42] and Davidson’s [45] engagement types.

Engagement Goal during the EC Process	Engagement form by Arnstein [42]	Engagement form by Davidson [45]	How Tools That Comply with the Defined Engagement Form Can Fulfill the Engagement Goal
Phase A—Awareness raising
Awareness and knowledge raising	Informing	Information	By providing information on the topic, knowledge and awareness are raised.
	Consultation	Consultation	Two-way information exchange enhances the feeling of involvement and sparks stakeholder interest.
Phase B—EC design
Information gathering for design input	Consultation	Consultation	The gathered input is used for the EC design and decision making.
Joint alternative EC design building	Placation	Participation	The designed EC alternatives form the basis for the decision making.
Decision making	Partnership	Empowerment	Consensus building here is between all stakeholders (and technical experts) instead of between power holders and citizens. The end decision is made by the project initiator based on the results of the consensus-building process.
	Delegated power	Empowerment	The end decision is the result of the consensus-building process.
Phase C—Implementation
Implementation and continued active involvement	Citizen control	Empowerment	Initiatives are directed towards strengthening the community feeling and active involvement of the stakeholders, so the EC keeps running.

Table 3

The engagement tools that were applied in the Relleu case study and application observations.

MAMCA Framework Step	Engagement Tool	Specific Application of the Tool	Tool Application Observations
Preparation	Specific information session with Q&A	A 20 min presentation was given with general information on all EC aspects, as well as potential pros and cons for Relleu.	Many questions were raised and answered throughout the presentation.
	Involving local project representatives	The chairman of the Homeowners’ Association and a local resident personally contacted the other residents to explain the project and invite them to the engagement initiative.	All people who were contacted in this way agreed to attend the workshop.
Step 1(scenario definition)	Planning charrette	An interactive exercise was set up in which the stakeholders were divided into two groups that each received an expert assistant and a toolbox with building blocks that can make up an EC. Each group had to answer various questions and use the toolbox blocks to create alternative ECs.	The further into the exercise, the smoother it went. Conflicting views within the group were always discussed until a consensus was reached.
	Expert advice	An expert assisted each stakeholder group in answering their questions and guiding them through each step of the alternatives design building.	The expert was able to not only assist and answer questions but also to raise additional questions and make the participants contemplate more on certain important issues.
	“Mock” EC design building workshop	The process for the ‘EC design and decision-making’ phase as a whole, assisted by experts answering all stakeholder questions, helps to build knowledge and awareness.	Participants indicated they understood the complexities of ECs and the steps that need to be taken in an EC setup process much better after the design workshop.
Step 2(stakeholder and criteria determination)	Survey on objectives and perceived barriers	The stakeholders filled out a survey in which they indicated their most important objectives when joining an EC, as well as the obstacles they see.	Participants indicated they selected multiple elements they had not thought of themselves before the survey, even though they thought they had a clear view beforehand.
	Thematic workshop	Following the survey, the results were discussed in a group session in which each stakeholder was asked to elaborate more on their answers.	They could give more nuance to their survey answers, stress their most important issues, and even made adaptations after hearing each other’s points of view.
Step 3(criteria weighting)	Exercise with weighting of stakeholder objectives	The online MAMCA software tool, with the pairwise comparison feature, was used in an exercise in which all stakeholders gave weights to their selected objectives, according to importance.	The weighting results often were quite outspoken and put a specific focus on elements that might have otherwise been easily overlooked. The software was an important help to the participants, as were the resulting visuals for clarification.
Step 4(linking indicators to criteria)	Expert advice	Before the workshop, for each of the objectives in the survey list, a panel of experts defined corresponding quantifiable criteria that can be used in the evaluation.	It was deemed necessary by the participants that this step be carried out by experts since they felt their knowledge of this complex and technical topic was insufficient to take on this step themselves.
Step 5(scenario evaluation)	MCDA as decision (support) method	All stakeholders evaluated the EC alternatives based on their selected objectives, using the online MAMCA software supported by the AHP. Experts assisted with the scoring of the more complex topics. The resulting individual and joint preference ranking of the EC alternatives was then visualized.	The MAMCA software helped to better structure the evaluation process and was deemed very useful by the participants to ‘translate’ and input their evaluation views in a clear way. The expert assistance turned out to be necessary for the more technical parts of the assessment.
Step 6(evaluation results and ranking)	Consensus building workshop	A consensus-building discussion was started with the stakeholders in which the evaluation results were compared and analyzed in detail, and an EC alternative that can be supported by all stakeholders was determined.	A consensus on a preferred EC form that was to be implemented was derived from the discussion. The MAMCA software visuals were an important help in structuring the discussion and pointing out conflicting opinions.

Appendix A. Pre-MAMCA Workshop Survey Questions

1/What is your name? (this information is for comparison reasons between the before and after survey only. Your personal data will not be stored)

2/Would you be willing to individually invest in renewable energy for your house in Relleu?

• ○

  No

• ○

  I have the feeling I don’t know enough about the subject to answer

• ○

  Maybe

• ○

  Yes

3/How much do you know about shared renewable energy initiatives? (0 = I know nothing about it, 10 = I am fully informed about all the existing options)

4/Do you know what an energy community is? (0 = I have never heard of it, 10 = I am fully informed about what it encompasses)

5/Do you know about the other residents’ knowledge on and demands for a renewable energy initiative? (0 = I do not know any, 10 = I am entirely informed about the wants and needs of the other affected parties)

6/To what extend would you be willing to join a shared renewable energy project with your Relleu neighbours? (0 = I do not want to change my demands, 10 = I would entirely step back from my demands to reach an agreement)

7/How likely do you think it is to reach an agreement on this with your Relleu neighbours? (0 = Very unlikely, 10 = Certain)

8/Are there specific questions or doubts you want to have cleared out by means of the MAMCA workshop?

Appendix B. Post-MAMCA Workshop Survey Questions

1/What is your name? (this information is for comparison reasons between the before and after survey only. Your personal data will not be stored)

2/Would you be willing to individually invest in renewable energy for your house in Relleu?

• ○

  No

• ○

  I have the feeling I don’t know enough about the subject to answer

• ○

  Maybe

• ○

  Yes

3/How much do you know about shared renewable energy initiatives? (0 = I know nothing about it, 10 = I am fully informed about all the existing options)

4/Do you know what an energy community is? (0 = I have never heard of it, 10 = I am fully informed about what it encompasses)

5/Do you know about the other residents’ knowledge on and demands for a renewable energy initiative? (0 = I do not know any, 10 = I am entirely informed about the wants and needs of the other affected parties)

6/To what extent would you be willing to join a shared renewable energy project with your Relleu neighbors? (0 = I do not want to change my demands, 10 = I would entirely step back from my demands to reach an agreement)

7/How likely do you think it is to reach an agreement on this with your Relleu neighbors? (0 = Very unlikely, 10 = Certain)

8/Did you get an answer to the questions or doubts you wanted to have cleared out during this workshop?

9/Did your knowledge of what an energy community in general encompasses increase through this workshop? (0 = Not at all, 10 = A resounding yes)

10/Did your knowledge of what an energy community for Relleu can encompass increase? (0 = Not at all, 10 = A resounding yes)

11/Are you more aware of the benefits and challenges of an energy community for Relleu than before the workshop? (0 = Not at all, 10 = A resounding yes)

• ○

  Yes

• ○

  No, I already was aware and did not gain new insights through the workshop

12/Did you learn things about the viewpoints of other residents that you did not know before? (0 = Not at all, 10 = A resounding yes)

13/Has your awareness of other participants’ viewpoints increased? (0 = Not at all, 10 = A resounding yes)

14/Have some of your own viewpoints changed by participating in this workshop? (0 = Not at all, 10 = A resounding yes)

15/Do you feel your input was taken into account during the workshop? (0 = Not at all, 10 = A resounding yes)

16/Do you feel your input could have an impact when setting up an energy community in Relleu? (0 = Not at all, 10 = A resounding yes)

17/Anything you learned from this workshop? Or other things you would like to tell us?

References

1. United Nations. Transforming Our World the 2030 Agenda for Sustainable Development; United Nations: New York, NY, USA, 2015.

2. Siciliano, G.; Wallbott, L.; Urban, F.; Dang, A.N.; Lederer, M. Low-Carbon Energy, Sustainable Development, and Justice: Towards a Just Energy Transition for the Society and the Environment. Sustain. Dev.; 2021; 29, pp. 1049-1061. [DOI: https://dx.doi.org/10.1002/sd.2193]

3. Burke, M.J.; Stephens, J.C. Energy Democracy: Goals and Policy Instruments for Sociotechnical Transitions. Energy Res. Soc. Sci.; 2017; 33, pp. 35-48. [DOI: https://dx.doi.org/10.1016/j.erss.2017.09.024]

4. Van der Schoor, T.; Scholtens, B. Power to the People: Local Community Initiatives and the Transition to Sustainable Energy. Renew. Sustain. Energy Rev.; 2015; 43, pp. 666-675. [DOI: https://dx.doi.org/10.1016/j.rser.2014.10.089]

5. European Parliament & Council of the European Union Directive (EU). 2019/944 on Common Rules for the Internal Market for Electricity and Amending Directive 2012/27/EU. Off. J. Eur. Union; 2019; 158, pp. 125-199.

6. European Parliament & Council of the European Union Directive (EU) 2018/2001 on the Promotion of the Use of Energy from Renewable Sources. Off. J. Eur. Union; 2018; 328, pp. 82-209.

7. Caramizaru, A.; Uihlein, A. Energy Communities: An Overview of Energy and Social Innovation; Publications Office of the European Union: Luxembourg, 2020.

8. Koirala, B.P.; Araghi, Y.; Kroesen, M.; Ghorbani, A.; Hakvoort, R.A.; Herder, P.M. Trust, Awareness, and Independence: Insights from a Socio-Psychological Factor Analysis of Citizen Knowledge and Participation in Community Energy Systems. Energy Res. Soc. Sci.; 2018; 38, pp. 33-40. [DOI: https://dx.doi.org/10.1016/j.erss.2018.01.009]

9. Heuninckx, S.; te Boveldt, G.; Coosemans, T. Towards a Construction Kit for Energy Communities. 2023; under review, not publicly available yet

10. Rodrigues, L.; Gillott, M.; Waldron, J.; Cameron, L.; Tubelo, R.; Shipman, R.; Ebbs, N.; Bradshaw-Smith, C. User Engagement in Community Energy Schemes: A Case Study at the Trent Basin in Nottingham, UK. Sustain. Cities Soc.; 2020; 61, 102187. [DOI: https://dx.doi.org/10.1016/j.scs.2020.102187]

11. Ghazal Aswad, N.; Al-Saleh, Y. IJIKMMENA 2,2 Clean Energy Awareness Campaigns in the Uae: An Awareness Promoters PerspeCtive. Int. J. Innov. Knowl. Manag. MENA; 2012; 2, pp. 131-156.

12. Heaslip, E.; Fahy, F. Developing Transdisciplinary Approaches to Community Energy Transitions: An Island Case Study. Energy Res. Soc. Sci.; 2018; 45, pp. 153-163. [DOI: https://dx.doi.org/10.1016/j.erss.2018.07.013]

13. Mishra, R.; Naik, B.K.R.; Raut, R.D.; Paul, S.K. Circular Economy Principles in Community Energy Initiatives through Stakeholder Perspectives. Sustain Prod. Consum.; 2022; 33, pp. 256-270. [DOI: https://dx.doi.org/10.1016/j.spc.2022.07.001]

14. Marinakis, V.; Papadopoulou, A.G.; Doukas, H.; Psarras, J. A Web Tool for Sustainable Energy Communities. Int. J. Inf. Decis. Sci.; 2015; 7, pp. 18-31. [DOI: https://dx.doi.org/10.1504/IJIDS.2015.068115]

15. Kumar, A.; Sah, B.; Singh, A.R.; Deng, Y.; He, X.; Kumar, P.; Bansal, R.C. A Review of Multi Criteria Decision Making (MCDM) towards Sustainable Renewable Energy Development. Renew. Sustain. Energy Rev.; 2017; 69, pp. 596-609. [DOI: https://dx.doi.org/10.1016/j.rser.2016.11.191]

16. Estévez, R.A.; Walshe, T.; Burgman, M.A. Capturing Social Impacts for Decision-Making: A Multicriteria Decision Analysis Perspective. Divers. Distrib.; 2013; 19, pp. 608-616. [DOI: https://dx.doi.org/10.1111/ddi.12058]

17. McKenna, R.; Bertsch, V.; Mainzer, K.; Fichtner, W. Combining Local Preferences with Multi-Criteria Decision Analysis and Linear Optimization to Develop Feasible Energy Concepts in Small Communities. Eur. J. Oper. Res.; 2018; 268, pp. 1092-1110. [DOI: https://dx.doi.org/10.1016/j.ejor.2018.01.036]

18. Stewart, T. A Critical Survey on the Status of Multiple Criteria Decision Making Theory and Practice. Omega; 1992; 20, pp. 569-586. [DOI: https://dx.doi.org/10.1016/0305-0483(92)90003-P]

19. Te Boveldt, G.; Keseru, I.; Macharis, C. When Monetarisation and Ranking Are Not Appropriate. A Novel Stakeholder-Based Appraisal Method. Transp Res. Part A Policy Pract.; 2022; 156, pp. 192-205. [DOI: https://dx.doi.org/10.1016/j.tra.2021.12.004]

20. Ignaccolo, M.; Inturri, G.; García-Melón, M.; Giuffrida, N.; Le Pira, M.; Torrisi, V. Combining Analytic Hierarchy Process (AHP) with Role-Playing Games for Stakeholder Engagement in Complex Transport Decisions. Transp. Res. Procedia; 2017; 27, pp. 500-507. [DOI: https://dx.doi.org/10.1016/j.trpro.2017.12.069]

21. Le Pira, M.; Inturri, G.; Ignaccolo, M.; Pluchino, A. Dealing with the Complexity of Stakeholder Interaction in Participatory Transport Planning. Adv. Intell. Syst. Comput.; 2018; 572, pp. 55-72. [DOI: https://dx.doi.org/10.1007/978-3-319-57105-8_3]

22. Gonzalez-Urango, H.; Mu, E.; García-Melón, M. Stakeholder Engagement and ANP Best Research Practices in Sustainable Territorial and Urban Strategic Planning. Multiple Criteria Decision Making for Sustainable Development: Pursuing Economic Growth, Environmental Protection and Social Cohesion; Springer: Berlin/Heidelberg, Germany, 2021; pp. 93-130. [DOI: https://dx.doi.org/10.1007/978-3-030-89277-7_5]

23. Kowalski, K.; Stagl, S.; Madlener, R.; Omann, I. Sustainable Energy Futures: Methodological Challenges in Combining Scenarios and Participatory Multi-Criteria Analysis. Eur. J. Oper. Res.; 2009; 197, pp. 1063-1074. [DOI: https://dx.doi.org/10.1016/j.ejor.2007.12.049]

24. Macharis, C.; Turcksin, L.; Lebeau, K. Multi Actor Multi Criteria Analysis (MAMCA) as a Tool to Support Sustainable Decisions: State of Use. Decis. Support Syst.; 2012; 54, pp. 610-620. [DOI: https://dx.doi.org/10.1016/j.dss.2012.08.008]

25. Milan, L.; Kin, B.; Verlinde, S.; Macharis, C. Multi-Actor Multi-Criteria Analysis for Sustainable City Distribution: A New Assessment. Int. J. Multicriteria Decis. Mak.; 2015; 5, pp. 334-354. [DOI: https://dx.doi.org/10.1504/IJMCDM.2015.074088]

26. Verlinde, S.; Macharis, C. Who Is in Favor of Off-Hour Deliveries to Brussels Supermarkets? Applying Multi Actor Multi Criteria Analysis (MAMCA) to Measure Stakeholder Support. Transp. Res. Procedia; 2016; 12, pp. 522-532. [DOI: https://dx.doi.org/10.1016/J.TRPRO.2016.02.008]

27. Verlinde, S.; Macharis, C.; Milan, L.; Kin, B. Does a Mobile Depot Make Urban Deliveries Faster, More Sustainable and More Economically Viable: Results of a Pilot Test in Brussels. Transp. Res. Procedia; 2014; 4, pp. 361-373. [DOI: https://dx.doi.org/10.1016/j.trpro.2014.11.027]

28. Aljohani, K.; Thompson, R.G. A Stakeholder-Based Evaluation of the Most Suitable and Sustainable Delivery Fleet for Freight Consolidation Policies in the Inner-City Area. Sustainability; 2019; 11, 124. [DOI: https://dx.doi.org/10.3390/su11010124]

29. Fredriksson, A.; Janné, M.; Nolz, P.; de Radiguès de Chennevière, P.; van Lier, T.; Macharis, C. Creating Stakeholder Awareness in Construction Logistics by Means of the MAMCA. City Environ. Interact.; 2021; 11, 100067. [DOI: https://dx.doi.org/10.1016/j.cacint.2021.100067]

30. Almeida, A.C.L. Multi Actor Multi Criteria Analysis (MAMCA) as a Tool to Build Indicators and Localize Sustainable Development Goal 11 in Brazilian Municipalities. Heliyon; 2019; 5, e02128. [DOI: https://dx.doi.org/10.1016/j.heliyon.2019.e02128]

31. Macharis, C.; Baudry, G. Decision-Making for Sustainable Transport and Mobility Multi Actor Multi Criteria Analysis; Edward Elgar: Broadheath, UK, 2018; ISBN 9781788111799

32. Freeman, R.E. Strategic Management: A Stakeholder Approach; Pitman: Boston, UK, 1984.

33. Glass, J.J. Citizen Participation in Planning: The Relationship Between Objectives and Techniques. J. Am. Plan. Assoc.; 1979; 45, pp. 180-189. [DOI: https://dx.doi.org/10.1080/01944367908976956]

34. Innes, J.E.; Booher, D.E. Reframing Public Participation: Strategies for the 21st Century. Plan. Theory Pract.; 2004; 5, pp. 419-436. [DOI: https://dx.doi.org/10.1080/1464935042000293170]

35. Rosener, J.B. A Cafeteria of Techniques and Critiques. Public Manag.; 1975; 57, pp. 16-19.

36. Cundy, A.B.; Bardos, R.P.; Church, A.; Puschenreiter, M.; Friesl-Hanl, W.; Müller, I.; Neu, S.; Mench, M.; Witters, N.; Vangronsveld, J. Developing Principles of Sustainability and Stakeholder Engagement for “Gentle” Remediation Approaches: The European Context. J. Env. Manag.; 2013; 129, pp. 283-291. [DOI: https://dx.doi.org/10.1016/j.jenvman.2013.07.032]

37. Warner, M. ‘Consensus’ Participation: An Example for Protected Areas Planning. Ltd. Public. Admin. Dev.; 1997; 17, pp. 413-432. [DOI: https://dx.doi.org/10.1002/(SICI)1099-162X(199710)17:4<413::AID-PAD952>3.0.CO;2-P]

38. Carson, L. Deliberative Public Participation and Hexachlorobenzene Stockpiles. J. Env. Manag.; 2009; 90, pp. 1636-1643. [DOI: https://dx.doi.org/10.1016/j.jenvman.2008.05.019]

39. Burke, E.M. Citizen Participation Strategies. J. Am. Plan. Assoc.; 1968; 34, pp. 287-294. [DOI: https://dx.doi.org/10.1080/01944366808977547]

40. Pretty, J.N. Participatory Learning for Sustainable Agriculture. World Dev.; 1995; 23, pp. 1247-1263. [DOI: https://dx.doi.org/10.1016/0305-750X(95)00046-F]

41. Lawrence, A. “No Personal Motive?” Volunteers, Biodiversity, and the False Dichotomies of Participation. Ethics Place Environ.; 2006; 9, pp. 279-298. [DOI: https://dx.doi.org/10.1080/13668790600893319]

42. Arnstein, S.R. A Ladder Of Citizen Participation. J. Am. Plan. Assoc.; 1969; 35, pp. 216-224. [DOI: https://dx.doi.org/10.1080/01944366908977225]

43. Hart, R.A. Children’s Participation From Tokenism to Citizenship; UNICEF: Firenze, Italy, 1992.

44. Rocha, E.M. A Ladder of Empowerment. SAGE J.; 2016; 17, pp. 31-44. [DOI: https://dx.doi.org/10.1177/0739456X9701700104]

45. Davidson, S. Spinning the Wheel of Empowerment. Planning; 1998; 1262, pp. 14-15.

46. Reed, M.S. Stakeholder Participation for Environmental Management: A Literature Review. Biol. Conserv.; 2008; 141, pp. 2417-2431. [DOI: https://dx.doi.org/10.1016/j.biocon.2008.07.014]

47. Lode, M.L.; Heuninckx, S.; te Boveldt, G.; Macharis, C.; Coosemans, T. Designing Successful Energy Communities: A Comparison of Seven Pilots in Europe Applying the Multi-Actor Multi-Criteria Analysis. Energy Res. Soc. Sci.; 2022; 90, 102671. [DOI: https://dx.doi.org/10.1016/j.erss.2022.102671]

48. Huang, H.; Lebeau, P.; Macharis, C. The Multi-Actor Multi-Criteria Analysis (MAMCA): New Software and New Visualizations. Proceedings of the Lecture Notes in Business Information Processing; Springer: Berlin/Heidelberg, Germany, 2020; Volume 384, pp. 43-56.

49. Saaty, R.W. The Analytic Hierarchy Process–What It Is and How It Is Used. Math. Model.; 1987; 9, pp. 161-176. [DOI: https://dx.doi.org/10.1016/0270-0255(87)90473-8]

50. Davidoff, P. Advocacy and Pluralism in Planning. J. Am. Plan. Assoc.; 1965; 31, pp. 331-338. [DOI: https://dx.doi.org/10.1080/01944366508978187]

51. Heuninckx, S.; te Boveldt, G.; Macharis, C.; Coosemans, T. Stakeholder Objectives for Joining an Energy Community: Flemish Case Studies. Energy Policy; 2022; 162, 112808. [DOI: https://dx.doi.org/10.1016/j.enpol.2022.112808]

52. Edwards, W. How to Use Multiattribute Utility Measurement for Social Decisionmaking. IEEE Trans Syst. Man. Cybern.; 1977; 7, pp. 326-340. [DOI: https://dx.doi.org/10.1109/TSMC.1977.4309720]