Tools for Measuring Energy Sustainability: A Comparative Review

Abstract

This paper is intended to perform a comparative and qualitative review among eight tools to measure energy sustainability. Therefore, it was necessary to create a theoretical and conceptual framework based on four criterias of selection and six categories of comparison. In this work, the conceptual bases that supported the research and the methodology created to carry out the comparative review will be presented. This analysis was based on the intrinsic concepts of energy sustainability of each of the reviewed tools with a critical qualitative analysis. Some conclusions shown through the conceptual framework developed that it was possible to apply an innovative methodology to qualitatively compare different tools to measure sustainability. The importance of this reflects the difficulty of conceptualizing the subjectivity of sustainable development, as shown throughout the paper, where it is often not possible to obtain a measurable result since the measured phenomenon is too complex to reduce it to a numerical value.

1. Introduction

The indicators calculated based on the performance of energy systems provide important measures for the economic, social, and environmental development of a nation since human activities and sustainability problems are intrinsically related to the use of energy. Energy indicators are tools that assist in formulating sustainable development policies. They measure progress towards sustainability. The more in-depth knowledge of these methods allows them to be applied in consideration of their main characteristics, advantages, and limitations of each tool, which leads to more legitimate and reliable results [1].

Sustainability and sustainable developments are terms widely used in scientific literature and public policies, even though they do not have a consensus on a standard nomenclature and metric, due to the vast diversity of existing concepts. Despite this, there is a general understanding of the need to seek a balance in the relationship between human beings and the environment and their complex interaction dynamics [2,3,4,5].

The term sustainable also has different translations such as longevity in German (nachhaltend) and durability in French (durabilité) and also Dutch (duurzaamheid), which leads to different possible interpretations [6]. In this context, the term reflects a solution of the scarcity of natural resources, and the search for a continuous and perpetual use of these resources, which suggests a connection with the concept of energy due to the crises of the energy system that have occured since ancient times [7,8]. Thus, sustainable development can be defined as access to achieve sustainability, which is considered the final long-term goal [9,10]. Sustainability consists of a final objective defined employing scientific criteria, which is measured and accompanied by results generated by the use of sustainable development strategies [11].

To define energy sustainability, it is necessary to initially introduce the concept of sustainability. Sustainability is a characteristic of a system that allows it to remain, at a certain level, for a certain period. This concept becomes a principle according to which the use of natural resources to satisfy present needs cannot compromise the satisfaction of the needs of future generations. The concept of sustainability meets a set of interdependent variables, which must be able to integrate social, energetic, economic, and environmental issues [12,13,14]. Based on this concept, energy sustainability is interconnected with the use of renewable energy sources, and, in this way, it is possible to meet the main need about energy sustainability, which is the guarantee that future generations will not be affected since the resources are not exhausted and are available for free. Sustainability in addition to being related to clean energy generation is also related to social, economic, and environmental issues [15,16,17,18].

Energy sustainability cannot be understood as the energy used to achieve sustainable development [19]. This is the biophysical perspective of the human use of energy to enable the achievement of goals by considering environmental changes [20]. With regard to energy sustainability, the change from fossil fuels to renewable sources cannot be sustained over time. To obtain sustainability, it is necessary to reduce the useful energy supplied, which may occur by improving systems for more efficient methods, i.e., less energy is required to perform the same work [21,22,23].

Since energy is a vital input for the social and economic development of any nation, setting the goal of achieving energy sustainability is a major challenge for all countries [24]. While the debate over the proper definition of sustainable development continues, the fact remains that societies depend heavily on fossil energy sources, which has a negative impact on environment and health [25,26,27,28]. In this context, it is essential to seek an alliance between energy and sustainability, to assist countries in the development of sustainable energy policies, through the use of sustainable indicators and indices as tools to communicate energy issues for the political field and for the general public [29].

The purpose of this research was to conduct a comparative review between several indices and energy sustainability indicators chosen in the international review. This paper is organized in the following segments. In Section 2, the literature review is presented. Section 3 presents the methodology used in the research. Section 4 offers an analysis of the results with a detailed discussion. Lastly, the last section presents the conclusions reached.

2. Literature Review

Sustainable development and energy problems are inherent elements for the governments of all nations and, intrinsically, converge on the routes for the energy industry [30]. These elements constitute tools for the well-being of society. Development is not a consequence, but rather a state of things that are sought through time, regardless of the development paradigm assumed in each context [31,32,33].

2.1. Energy and the Dimensions of Sustainability

Sustainability is an act of equilibrium between social, environmental, and economic dimensions of human needs [34]. These balanced dimensions are constantly changing because of the growing human population and the fulfillment of their consumption needs [35,36,37]. Therefore, since the dimensions of sustainability are not independent of each other, the constant balance of sustainability makes it a dynamic concept and not a static state [38]. Due to the dynamic characteristic of sustainability, new ideas always emerge [39,40,41]. Additionally, for the three fundamental dimensions of sustainability (economic, social, and environmental), there are other definitions of sustainable dimensions in the literature, according to the need for the phenomenon to be measured. The institutional dimension and the technical dimension were incorporated to broaden the vision of sustainability [42,43,44], which totalled five dimensions that will be discussed in this paper [34].

The environmental dimension of sustainability focuses on natural biological processes, health, and ecosystem functionality as well as on continuous productivity with minimal environmental impacts [45]. This dimension seeks to reduce the negative impacts of the extraction of natural resources for energy production, consumption by society, and increase of the positive impacts. Global, national, and regional commitments show the important role of energy in sustainable development to reduce energy-related environmental degradation such as global warming, deforestation, air, land, and water pollution [46,47,48,49].

The economic dimension of sustainability evaluates whether the supply of energy is profitable and affordable or not [50]. Cost-effectiveness is necessary to ensure that energy investments are economically viable to encourage reinvestment in the system that promotes sustainability. Accessibility also ensures that the energy supplied is not only physically available, but accessible to society. The two complement each other to ensure the sustainability of the supply system. If the energy provided is very costly, society cannot consume it and, thus, the suppliers do not get any return for their investment [51,52]. Economic sustainability is achieved if the system offers a profitable and affordable energy supply in the present and the future. This is possible considering the cost of recovering the power supply potential, the operational costs imposed on users, and the need for financial support for the system [53].

In addition to the economic context, there is also a social context to assess the level of sustainability. The economic analysis includes the disparity in wages and costs of services between countries, which may mislead sustainability studies. Thus, the analysis of the level of sustainability may not be based entirely on the economic criteria. Different sustainability studies must go beyond economic analysis due to the intrinsic and complex nature of economic markets. For example, a photovoltaic system installed in Europe, but produced abroad, has an energy burden for society in terms of energy [54,55,56].

The social dimension of sustainability assesses the distributive effect of energy in society. It measures the acceptability, accessibility of the energy supply, and access to energy services for all segments of society [57]. It results in two main subdivisions: scope (extension) and inclusion. The scope defines the total area of society that is physically covered by energy services. Interconnected systems tend to exclude some areas either because they are in remote geographic locations or because investments and maintenance have a high cost for network extension [58].

Electrical exclusion can also occur due to the unequal supply of energy services. Parts of a society may be physically covered by energy services. However, they are still financially excluded from consuming these services. This type of exclusion comes from the unequal distribution of wealth in a country where a smaller percentage of the population has a greater share of its wealth [59]. About 2 billion people do not have access to modern energy services in the world today, and most of them still meet their essential energy needs such as cooking and heating from natural resources like burning wood and residual biomass [60]. These important social issues related to the use of energy, including poverty, quality of life, education, health, income inequality, and social justice, are themes that should be considered under the social dimension of sustainable development [61]. To improve access to electricity, researchers have assessed how electrical systems can be improved to reliably serve the population [62,63,64,65].

The institutional dimension of sustainability deals with the impacts and effects of institutions on social relations and how they can change the behavior of different social players. Institutions are influenced by players and their activities. On the other hand, institutions considerably influence players by shaping their interests and behaviors [66,67,68]. There is a mutual relationship between institutions and players in such a way that social values can shape institutional structures and, in response, institutional values appear as a measure for the rational assessment of players. Consequently, institutions act as an entity between actors and structures, which create new value orientations or reference points that influence the actors and their behavior [69].

From an economic point of view, the institutional dimension of sustainability can be defined as the capacity of an organization to produce results of value so that it acquires sufficient inputs to continue production at a constant or increasing rate [70]. The institutional scenario is a critical component of sustainability in which development policies are conceived, financed, implemented, and administered [69]. For the increasing public-private participation, institutions play an important role as a hub of governance and they balance the activities of the various dimensions of sustainability [71].

The energy crisis of the 1970s was the main driving force for expanding energy-oriented institutional capacities, particularly in the public sector. Government organizations at the national and local level develop their capacities both in decision-making and in managing energy-related issues. During the 1980s, government institutions became more experienced in managing energy-related activities with the adoption of energy policies, energy planning at national, regional, and local levels, and energy management in the industrial, building (commercial/residential), and transport arenas [72]. In addition to increasing public-private participation, institutions play an important role as a governance hub and balance the activities of the various dimensions of sustainability. With a focus on the sustainability of the energy system, a good institutional structure can play a key role in facilitating the implementation of energy plans and policies at international, national, regional, and local levels [73,74,75,76,77,78].

The technical dimension of sustainability measures the capacity of the system to provide energy to society reliably, efficiently, and with clean and renewable sources [79]. Technical sustainability is achieved if energy planning can meet the current and future energy needs of society [80]. This is possible if the power supply is reliable, efficient, and based on renewable energy [81] with locally available support services for the maintenance and execution of energy systems [82,83,84].

2.2. Data, Indicators, and Indices

The etymology of the word indicator comes from the Latin word indicare, which means to reveal, make public, and estimate. An indicator or set of indicators can transform basic statistical information to provide a deeper understanding of a problem or dimension and help develop a clear picture of the entire system, including its interrelationships and commitments [85]. When placed numerically, they are measured or derived from quantitative and/or qualitative measurements, which can be standardized for comparison with information from other areas or regions [86].

Indicators and indices are different, even though they may have a direct relationship. They are often used as synonyms, which can confuse several areas of application. An indicator is a tool that allows obtaining the information about a given reality, which can be individual data or a set of information [87]. A good indicator should be simple to understand, have statistical quantification with coherent logic, and effectively communicate the state of the observed phenomenon [88]. On the other hand, an index is defined as the final aggregate value of an entire calculation procedure in which the indicators are also used as variables that compose such an index, which is also referred to as high category indicators [89,90].

Figure 1 presents a visual representation known as the information pyramid, which shows how the amount of information relates to the increased condensation [88,89]. In Figure 1a, the existing relationship between primary data, indicators, and indices are presented. In Figure 1b, the relationship between the information and the intended public can be observed.

In Figure 1a, the top of the pyramid corresponds to the maximum degree of data aggregation, while the base represents the disaggregated data. The various assessment tools have their particularities. Some do not work with indexes or intermediate indicators because they use only primary and aggregate data to compose the final index, while others use all the steps of the information pyramid because they need a large amount of primary data to obtain the indicators, which are subsequently aggregated to obtain the final index [89].

Figure 1b shows the need to design indicators that provide relevant information for a specific target audience. This does not mean that different audiences receive different information, but that information must be written in a language that resonates according to the target audience. Scientists seek knowledge and understanding of the world, but legislators need to access information that helps them eleborate on viable policies, while the general public needs to know whether the goals and objectives are being met or not. Each audience has its language in terms of communication, and the criteria for using a specific indicator or index is whether it can communicate to the final users what they need to know [88,89,90,91].

The measurability of the sustainability is the key to implementing sustainable development [92]. Indicators and indices gain great importance and recognition as tools to formulate public policies and provide information of performance in areas such as economy, environment, society, and technology [93]. The indicators are adopted by countries and companies for their ability to summarize, focus, and condense the complexity of the dynamic environment. Thus, it can be a manageable amount of meaningful information [94]. The purpose of the tools for measuring sustainability is to provide decision-makers with a comprehensive and integrated assessment of the systems of nature and society such as medium and long-term perspectives to help them determine what actions should or should not be taken [95]. Energy sustainability indicators capture the complexity of energy systems and offer new insights about what can be communicated to the general public [96,97,98,99,100].

Some several indicators and indices exist for measuring energy and its relationship with sustainable development. The methods that created these indicators work as tools to measure sustainable development. The use of indicators is popular in many fields, especially economic indicators, indicators of poverty, health, environmental, and, more recently, energy indicators [91]. The theoretical framework created considered access to energy, energy poverty, energy consumption, energy security, the energy market, development, and energy integration, as some of the concepts used for the revised tools [101,102,103]. These concepts were chosen after conducting a literature review that considered recent research on indicators and indices of energy sustainability and also tools already consolidated in the scientific literature.

2.3. Energy Sustainability Indicators

The Latin American Energy Organization (OLADE), the Economic Commission for Latin America and the Caribbean (ECLAC), and the German Technical Cooperation Agency (GTZ) have presented eight indicators to capture energy sustainability including three indicators from the economic dimension, three indicators from the environmental dimension, and two indicators from the social dimension. The Energy Sustainability Indicators (ESI) are presented in Table 1. This set of indicators is of great historical importance, as they are part of the pioneers promoting the union between energy and sustainability, and serving as the basis for many types of research and development of other indicators and indices of energy sustainability [104]. In this methodology, all indicators have the same weight. There is no difference between them, even though they are related to different dimensions [105].

The ESI was applied in Latin America and the Caribbean to understand the positioning of energy sustainability in these regions [104] as well as used in Mexico to analyze local energy policies [105]. However, no aggregate index was calculated even though the indicators capture important aspects of the energy system under study. A critique of the methodology shows that these indicators were chosen arbitrarily, not being used to provide an international comparative framework [44].

2.4. Energy Indicators for Sustainable Development

Specifically designed by the International Atomic Energy Agency (IAEA) to measure energy supply for sustainable development, it is a project of energy indicators in cooperation with several international organizations, of which are: United Nations Department of Economic and Social Affairs (UNDESA), European Statistical Office (Eurostat), European Environment Agency (EEA), and the International Energy Agency (IEA) was initiated in 1999 [61].

The original name was Indicators for Sustainable Energy Development (ISED) and was subsequently modified, since some of the experts considered that the term “sustainable energy development” referred only to renewable energies, which limits choices related to energy. In a social dimension, the theme is equity with the sub-theme being accessibility, wherein it can be defined as “share of households (or population) without electricity or commercial energy, or heavily dependent on non-commercial energy.” In addition to the social context, the economic and environmental context must also be assessed.

In the first phase, the project was comprised of a set of 41 indicators adapted to measure sustainability in energy systems [86]. In the second phase, this original set of indicators was reduced and the Energy Indicators for Sustainable Development (EISD) consist of a set of 30 group indicators, classified in the dimensions of social, economic, and environmental sustainability [61].

The indicators represent an instrument for policymakers to evaluate and design programs and strategies and monitor progress toward a more sustainable future at a regional level. The Energy Indicators for Sustainable Development (EISD) can assist countries in their efforts to assess the progress made in implementing sustainable energy development strategies and identify areas in which measures and specific policies should be targeted. Case studies developed in several countries, including Brazil, demonstrate the advantages of using this integrated approach in the formulation and implementation of EISD [106,107,108,109].

The set of indicators of the EISD have their scope in the explanation of sustainable access to energy, but it has fundamental flaws, such as those found in the ESI indicators discussed earlier. The interpretation of changes in a large number of indicators can make it impracticable to compare the performance of countries or regions with this set of indicators [44]. A study using EISD was conducted for Brazil and, according to its authors [108], some flaws in the set of indicators may be shown, especially in the face of the inequalities in income that occur in the country and the existing regional differences because the lack of access to electricity mainly affects poorer regions. Because of this situation, concerning accessibility and energy viability, the fact that the lack of access to the electrical grid is mainly a regional problem, it is indirectly linked to the concentration of the population in rural areas and the conditions with poor distribution of income among the various regions of the country [110].

2.5. Energy Development Index

The Energy Development Index (EDI) was elaborated on by the IEA to better understand the role that energy plays in human development. It focuses on the transition made by a country or region to the use of modern fuels and energy services. It does not refer to whether the transition is sustainable, based on the economic, social, or environmental conditions of the country [111]. The composition of EDI is shown in Table 2.

The indicators that compose the EDI seek to quantify the availability of modern fuels, the consumption of these fuels, and how this consumption supplies basic needs and human development by segregating the energy development at domestic and community levels [102].

EDI was created to measure the transition to modern fuels in a country or region. It is calculated as the arithmetic average between four indicators that represent this transition, represented as output variables measured only in the consumption stage, without the inclusion of any time dimension [96]. The EDI shows the production of the country under analysis, but it does not show whether the effort channeled to modernize the fuels is correct or not. Since the limited supply of energy can restrict consumption to values lower than demand, EDI fails to address sustainability subjectively, and this is currently one of the biggest problems of developing countries [44]. A country can quickly increase its production or import energy resources and achieve a high level of EDI, while it can indent the index due to the depletion of its local resources or weaken its internal economy through increased imports.

2.6. Multidimensional Energy Poverty Index

Multidimensional Energy Poverty Index (MEPI) is a more recent metric for measuring access to energy. It focuses on depriving modern energy as opposed to accessing energy, and it captures both the incidence and intensity of energy poverty. It consists of six indicators in five dimensions, comprising the basic services of cooking, lighting, appliances, entertainment/education, and communication [112].

The MEPI methodology is derived from the literature on measures of multidimensional poverty of the Oxford Poverty and Human Development Initiative (OPHI) institution, which defends the need to concentrate on human poverty, considering the lack of opportunities and choices to live a basic human life [113,114]. Essentially, the MEPI captures the set of energy deprivations that can affect a person who is now identified as poor in energy if the combination of privations faced by such a person exceeds a predefined threshold.

The MEPI index focuses on energy services and is based on data related to energy deprivations, as opposed to deriving information indirectly through variables that are presumed to be correlated (e.g., energy or electricity consumption). Additionally, it captures both the incidence (number of poor people in energy) and the intensity (how much these people are poor in energy) [115,116,117].

In the research [112], the dimensions, indicators, and variables of the MEPI index are discussed. In this case, in the dimension, there is an indicator, a variable, and an evaluative item. In the dimension of the way the kitchen is used, the indicators are internal pollution and technology used in which it is evaluated whether the cooking is carried out with internal or external fire and what type of fuel is used. Lastly, the evaluation item can be answered simply as yes or no for the specific use of the technology, or it can be explanatory in terms of defining which fuel is used.

Specifically related to energy poverty, MEPI is a composite index that measures the deprivation of energy access [112]. This metric shows the communication to some countries that are poor in energy due to the direct measurement of useful energy needs but does not assign reasons for the cause or give suggestions on how the situation can be reversed [115]. Energy poverty is a dynamic and complex concept [118], and, although MEPI captures the reliability of the energy supply as part of deprivation, it does not deal with the issue of sustainability directly.

2.7. Sustainable Energy Development Index

The Sustainable Energy Development Index (SEDI) was created more recently to fill the gap between environmental indicators and the dimension of energy sustainability [44]. It comprises an index consisting of eleven indicators, distributed in five dimensions: technical, economic, social, environmental, and institutional. The study was initially carried out in 20 countries considered to be developing and then in 30 developing countries. A comparison of the results with the Human Development Index (HDI), the EDI, and the Environmental Performance Index (EPI) was also performed [43].

Due to the gap between sustainable development and energy development, the SEDI index was considered a proposal to measure sustainable energy development. However, as with the other revised tools, SEDI is not immune to the inherent weaknesses of data masking, where strong performances of some variables tend to conceal the weaknesses of other variables [44]. The value of SEDI should be understood as a relative value, whose total implication at the level of each country should be derived from the analysis of the various dimensions of sustainability. Therefore, if a country studies the performance of individual dimensions under the SEDI methodology, it will be able to identify political pathways to improve the weaker dimensions, which can help optimize the socio-economic benefits derived from access to sustainable energy.

2.8. Sustainable Development of Energy, Water, and Environment Systems Index

The Sustainable Development of Energy, Water, and Environment Systems Index (SDEWES) is a composite index created to assess the sustainable development of local energy systems [119]. It consists of a single set of seven dimensions and 35 main indicators and was initially applied to a sample of 22 port cities in the Mediterranean, based on three energy scenarios.

The scenarios considered the reductions in final energy consumption per capita for a given area of the city, based on intelligent energy measurements [120]. More recent studies have applied the proposed index to 12 cities in Southeastern Europe and 26 global cities, including Rio de Janeiro, to compare the performance related to SDEWES [121,122]. As a result of the application of the SDEWES index, this paper presents the classification of the best sustainable practices of the sampled cities, separated by dimensions. These best practices are focused on action-oriented measures that can be implemented by other cities to improve the performance of dimensions. The actions range from buildings based on cogeneration with combined heat and energy cycle, zero energy buildings, sustainable water management, large urban forests, urban planning, and incentive policies for research, development, and innovation [123,124,125,126,127].

The SDEWES emphasizes the need to implement best practices across a wide horizon of dimensions that will collectively serve for the sustainable development of energy, water, and environmental systems in cities [119]. It provides a foundation for cities to progress to smarter energy systems and, at the same time, cities need to be compared based on the magnitude, efficiency, and intensity of energy use and CO₂ emissions [128]. The SDEWES Index assesses the performance of local energy systems based on an integrated approach. The approach of a composite index includes the environmental, social, and technological context of local energy systems. The results indicate that well-articulated political efforts are needed to bring the local energy systems closer to sustainable development [129].

In the application of SDEWES Index dimensions and indicators, only two evaluation levels are presented including the dimension and the indicator. In the dimension, the concepts are broader such as when discussing energy consumption, in which there are several indicators, such as energy consumed in buildings, transportation, or per person. In addition to these metrics, other metrics can be used, such as how much energy is used for heating or cooling [121].

2.9. Regulatory Indicators for Sustainable Energy

Regulatory Indicators for Sustainable Energy (RISE) was created in 2014 with 28 indicators and 85 sub-indicators, divided into four categories and three dimensions, with pilot methodology initially applied in 17 countries [130]. In 2016, the methodology received some modifications, and RISE became defined as a set of 27 indicators and 80 sub-indicators in three dimensions. Its application had coverage in 111 countries, representing 97% of the world population, which provides an instrument to develop energy policies and regulations in each of the three dimensions used: sustainable access to modern energy, energy efficiency, and renewable energy [131].

In December 2018, a fourth dimension was incorporated (clean cooking), and 12 other countries were added to the overall study, which totaled 133 countries categorized into 32 indicators. Since this new dimension is a pilot, the methodology of calculating the indicators remained the same as the previous one with three dimensions [132].

Table 3 shows the dimensions and indicators of RISE in 2018. As a result of a partnership between the United Nation (UN), World Bank, and IEA, RISE uses the goals set out in the Millennium Development Goals (MDG) and in the efforts of the Sustainable Energy for All (SEforALL) initiative, which has led several stakeholders to commit to achieving three goals up to 2030. This was done to ensure universal access to modern energy services, which doubles the rate of energy efficiency improvement and doubles the share of renewable energy in the global energy matrix [133].

RISE ranks countries in a green zone of strong performance, a yellow zone of medium performance, and a red zone of poor performance. Each indicator targets an important policy element or regulatory regime to mobilize the investment, such as the establishment of planning processes and institutions, the introduction of dedicated incentives or support programs, and the assurance of funding energy access programs, and encouraging renewable energies. RISE indicators provide a comprehensive view of government support for sustainable energy and actions taken to transform this support into reality [131].

While many of the countries with the greatest impact on global sustainable energy outcomes are developing or develop strong policies and regulations, the results on access to energy are often distorted, especially in countries with particularly low electrification rates [44]. In the case of Brazil, although there is a high electrification rate, RISE classified the access to electricity with strong performance (100%). However, it did not consider the reality of isolated electrical systems, which correspond to more than half of the national territorial area and comprise thousands of people with no access to electricity [57]. This shows the need to regionalize the indicators for each reality. At the same time, it shows the difficulty of having an indicator to a country or continent.

2.10. Synthetic Index of Sustainable Energy Development

The Synthetic Index of Sustainable Energy Development (SISED) was created in 2016, based on the ESI methodology, which was proposed by the World Economic Forum in 2002. SISED was planned to be a synthetic index consisting of three indices from a set of indicators to promote sustainable energy policies to European countries [134]. The index is based on 33 variables, grouped into nine categories, comprising three distinct dimensions: energy supply safety (SES), competitive energy market (CEM), and environmental protection (EP). The variables were chosen according to the guidelines of the International Atomic Energy Agency (AIEA) [86] and are described in Table 4.

The SES dimension considers three categories: energy dependence, which shows the dependence on imports to meet the country’s energy needs, and considers that diversifying the energy supply is a determinant factor for safety. This is the primary energy production category, which assesses the future insertion of energy sources about the current availability of energy resources, and the power consumption category, which considers that reducing the amount of energy consumed can make society more independent when concerning the energy supply [135].

The CEM dimension comprises three categories. Energy efficiency is one of the foundations for seeking sustainable development, which has, in its energy intensity, its measurement form and relates the energy consumption of an economy with its global energy efficiency. The energy market category has the resource productivity variable as a way of measuring the availability of national resources with the corresponding production. The category of competitive prices has a direct relationship with efficiency gains in the economy because, theoretically, the liberalization of the electricity market aims to increase competition, which decreases the final price for consumers.

The Environmental Protection dimension (EP) has three other categories. The category of renewable energy sources is of fundamental importance for sustainable development, ranging from environmental protection to safety and diversification of energy supplies [136]. The category of greenhouse gas emissions, which evaluates the impact on the atmosphere of energy-related activities. The energy tax category, which aims to facilitate a change to renewable energy sources in the primary energy matrix.

Although the indicators discussed above have been created for a specific purpose, in some specific aspects, they have omitted the dimension of energy sustainability, responsible for the alliance between the technical dimension and other dimensions of sustainability (environmental, economic, social, and institutional). While the MEPI tries to capture the level of energy poverty, EDI has the dimension of access to energy, and SISED has environmental protection with a focus on energy security. They are not specifically designed to capture the dimension of sustainability. A country with a high index of human development and a high rate of access to modern energy may have little or no self-sufficienct natural resources. Even the domestic resources of rich countries could exhaust so quickly that it is uncertain how future supplies will be sustained. Therefore, high EDI does not necessarily indicate a sustainable energy future [44].

Conversely, a country with a low EDI or low MEPI can still have a moderate level of energy sustainability. In the current context in which sustainable development cannot be separated from energy concepts, the lack of energy sustainability needs to complement existing indices and indicators to provide a holistic view of the energy issues by environmental issues.

In general, the indices and indicators presented in this paper can be good complements to the energy metric tools on the path of sustainable development. Indicator systems, in any sphere, have become an important element in determining public and social development. When energy sustainability indicator systems are recognized and accepted, they become important components in the formulation of energy policies, which initiates an effective process of changes in priorities and behavior of the social players. In this sense, it is important to develop, test, and implement tools that capture all the complexity of concepts, such as energy and sustainability, without reducing the significance of each element that is part of any evaluation model.

3. Methodological Procedures

The research is qualitative in that it assigns numerical values to the observations of this investigation, considering that this research has an inductive and interpretive approach. Initially, a literature review on energy sustainability indicators was carried out, through documentary research in articles and documents of the institutions that created the studied methods in addition to critical studies on each tool [137].

Because of the review, eight tools were selected based on four selection criteria: (1) historical importance and origin (authors and institution), (2) the concept of sustainability contained in the tool, (3) the characteristics of each method, and (4) its theoretical and practical applicability. After selecting the tools, they were compared through six categories and six subcategories, as can be seen in Figure 2.

After criterion selection, a criterion comparison method is used. The comparison criteria are grouping, scope, sphere, data (subcategories typology and aggregation), participation, and interface (subcategories complexity, presentation, openness, and educational potential). Each of the categories and subcategories used provides a guideline for comparison between the different methodologies of the indicators and energy sustainability indices revised. Figure 2 is the representation of the methodological structure used.

As previously presented in Figure 1, the base of the pyramid represents the largest volume of data to be analyzed. These data are evaluated to obtain an indicator and then an index. In addition to the information categories presented in Figure 1, Figure 2 shows the existence of different types of information and this information is responsible for forming the indexes.

The research carried out is descriptive and exploratory with a predominantly qualitative approach [138]. For this purpose, only secondary data from bibliographic and documentary research was used, which served as a basis for categorizing the tools in the comparative study. It was initially organized by grouping with representation in group indicators and composite indices, which follow a chronological order of creation of the tools. Subsequently, based on the comparison criteria listed above, each analytical category is described separately, to show the results achieved in comparing the tools.

Referring to the grouping category, three major divisions can be found in the literature [109] such as individual indicators, which provide specific values that are easy to interpret and less susceptible to synthesis errors, when compared to other indicators of groups or compound indices. However, they are one-dimensional and, therefore, can omit vital information inherent to the degree of complexity of the concepts measured, such as sustainability and energy. The second major division includes disaggregated individual group or panel indicators, corresponding to a variety of indicators to deal with issues that require the use of many component variables of different dimensions. The third major division includes compound or aggregate indices, which are simple and easy to interpret as the one-dimensional indicators. At the same time, they are condensed to a set of variables that capture the various dimensions of the theme in question, which take advantage of the points and avoid the weaknesses of the previous categories [112].

The scope category comprehends the dimensions of sustainability encompassed by the evaluation tool and how they are used. It is based on what is measured, while the sphere category analyzes the field of application of the tool and which regions the system is designed for. The tools were classified according to geographic boundaries (global, continental, national, regional, or local).

The data category analyzes the typology of the data used by the tool and how they are treated in the evaluation, which entails the degree of data aggregation relative to the information pyramid shown in Figure 1. The typology refers to the use of qualitative and/or quantitative information [139]. The degree of aggregation of a system of indicators will be observed by the location of its indices, indicators, and data in the information pyramid [89].

The participation category analyzes the form and intensity of the participation of different social players in the development of indicator systems under analysis. It assists in the strategy of ordering knowledge and information processing, with the distinction between models based on a top-down approach, which uses aggregated data to predict purposes, while the bottom-up approach uses disaggregated data for exploration purposes [140].

In a top-down approach, guidance is given primarily by specialists, who perform the fragmentation of the larger system to improve understanding the composition of the subsystems. In the bottom-up approach, the largest weight is given to the players involved in the process, including the target audience. The most basic elements are described in detail and are later associated with a larger subsystem within a given level, while repeating the process until completing the highest level of the proposed objective [141]. Both approaches are extreme points of the same line, and each evaluation tool analyzed must be inserted into this context, which can carry characteristics of the two approaches. This results in a hybrid approach [142].

The interface category analyzes the tools reviewed by the user’s perspective. It considers the ease of observing and interpreting the results obtained. It represents how the tool communicates with the public as well as the degree of complexity and transparency of the tool, its presentation structure, and its pedagogical potential in terms of environmental education. This category is related to the degree of ease to interpret the results obtained in the clear and objective visualization of these results, with the view to better guide decision making. It also has a great influence on the ability to change social behaviors and act as a pedagogical tool in environmental issues. To detail the interface analysis category, four subcategories were divided.

• Complexity: The main characteristic of this subcategory is to evaluate the level of difficulty of the calculations involved in obtaining the studied indicators and indices to show a high, medium, or low degree of complexity. Objectively evaluating the degree of complexity of an isolated tool is difficult. However, a comparative study is possible to hand out a simplified evaluation.

• Presentation: It evaluates the visual presentation of the process to the end-user such as chart types, tables, maps, etc.

• Openness: Subcategory directly related to complexity and presentation, which seeks to show at what degree the implicit values of the evaluation system are disclosed to the public. The system will have greater openness and possibility of observing information in the tool interface, including the weights attributed to the data analyzed, at the different levels of the information pyramid (see Figure 1).

• Educational Potential: Verifies the learning capacity of the concepts involved in sustainability and energy by the user. Analyzes the pedagogical potential of application for the target audience or civil society.

The objective of this subdivision is to consider distinct and complementary aspects of the evaluation tools.

4. Data Analysis and Results

In group indicators, it is more appropriate to capture complex information than individual indicators due to the greater number of parameters used to measure complex issues such as human development and sustainability, which are multi-dimensional [143,144,145]. On the other hand, compound indices reduce the information set that would have been provided by a myriad of group indicators to a level that makes the analysis convenient and provides a uniform scale over which the comparative performance of the indicators can be measured [44]. Compound indices were created as a commitment between the simplicity of individual indicators, which are one-dimensional, and need to realize the multi-dimensional nature of some issues. They represent an attempt to overcome the shortcomings of the simple indicators and, at the same time, produce a result that condenses the information with easy-to-interpret metrics, which is a sort of composition between the simple and group indicators.

4.1. Grouping

Table 5 presents in chronological order the creation and the revised tools, with indicators and indices classified, according to grouping categories, their respective initiators, and references. As observed in the results of the grouping category, the group indicators were the pioneer methods used by the revised tools initially in 1997 with the ESI and soon after 1999 with the EISD.

The EDI appears as a compound index only in 2004, which is then followed by other tools. In 2014, RISE was created as a set of group indicators in which it resumes the use of pioneer methodologies, but with a modernized approach and interface. The choice of the best grouping method to measure energy sustainability depends on the opinion and experience of each user or researcher. Therefore, the knowledge of the qualitative characteristics of the tools is as important as the quantitative results, which are delivered.

4.2. Scope

Regarding the scope category, which analyses the dimensions in which the tool operates, the revised tools start with two dimensions (EDI) and vary between the three basic dimensions of sustainable development (ESI and EISD). For five dimensions inserting new scope criteria, such as the social and institutional dimensions (SEDI), it also completely alters the basic dimensions with scopes adapted to capture complex themes, such as cooking and lighting (MEPI), energy consumption and climate (SDEWES), access to energy and renewable sources (RISE), and environmental protection (SISED). Some dimensions are created according to the application and use of each tool. The scope category is summarized in Table 6.

A pertinent observation is a tool that uses multiple scopes. The importance of this dimension within the system is proportional to the number of dimensions used. Thus, each dimension of the SEDI has one-fifth weight in the final value of the index because it is composed of five dimensions in total. With this, an excess of dimensions can reduce the relative importance of them within the evaluation tool and impair the validity of the results. On the other hand, a system with few dimensions ends up having a limited relevance in the issue of capturing the complexity to be measured while something observed in the EDI has only two dimensions.

4.3. Sphere

The sphere category is associated with the range of the field of application tools. ESI and SISED have great range and can be applied at global, continental, national, regional, and local levels. The MEPI and SDEWES are more restricted to a local application, as presented in Table 7. Analyzing this item implies knowing the potential of a comparative use of the tool, which may or may not be applied between different scenarios, alone or in comparison with other systems.

4.4. Participation

In the top-down approach, a set of criteria and indicators initially generated is used by a team of experts who adapt and modify this set according to the local situation. In the bottom-up approach, local actors are actively involved in a participatory manner by proposing criteria and indicators based on their perception of the individual situation [146,147].

Although they are fundamentally different approaches, there is a possibility to use the elements of each of the previous approaches without creating conflicts. Top-down processes defined by experts may not have legitimacy for stakeholders, while a participatory approach by local actors can increase the legitimacy of the sets of criteria and indicators. In this way, a hybrid approach allows the improvement of mutual learning and the sharing of experiences to develop adaptive strategies over the entire process, which ends up resulting in the development of more practical and meaningful indicators [148].

In the participation category, the type of approach determines the players involved in the choice of indicators. This has a direct impact on the method used to measure energy sustainability because of the implicit concept of sustainable energy development of a tool that becomes apparent through its dimensions and its indicators. Therefore, evaluation systems reflect the values or beliefs of certain players about energy sustainability. Furthermore, according to Table 8, among the revised tools, there was a tree hybrid approach (MEPI, SDEWES, RISE, and SISED), and five top-down approaches (ESI, EISD, EDI, and SEDI). None of them exclusively uses the bottom-up approach.

4.5. Data

In Table 9, we have the comparative analysis according to data criterion, subdivided by the typology and aggregation. Referring to the data category, in most of the tools, the data worked is quantitative due to the need to reconcile the various existing tools to be compatible with each other [88]. Nonetheless, some tools can associate the concept of sustainable development with the quality of the system, which constitutes subjective and non-quantifiable quantities.

According to Table 9, we have five tools whose typology is uniquely quantitative (ESI, EISD, EDI, SEDI, and SISED) and three others with mixed typology, which work quantitatively and qualitatively (MEPI, SDEWES, and RISE). None are exclusively qualitative.

In the subcategory of data aggregation, the location of indices and indicators in the information pyramid (Figure 1) is the condition of the degree of aggregation. The tools whose grouping results in a general composite index have a maximum degree of aggregation such as EDI, MEPI, SEDI, SDEWES, and SISED. The existence of a highly aggregated index creates a condition to use intermediate indicators to compose the general index (EDI, SEDI, and SDEWES), intermediate indicators and sub-indicators (MEPI) or sub-indices, intermediate indicators, and sub-indicators (SISED). In turn, the tools that do not have a general aggregate index are composed only of indicators, such as ESI, or are formed by indicators and sub-indicators such as the case of EISD and RISE.

4.6. Interface

The interface category is also an important element in evaluating the tools. The efficiency of the results obtained by the indicator system is directly related to the capacity that the tool has in contributing to the increased critical consciousness of the players and users involved. Therefore, the interface category involved in the results regarding the data availability and visual presentation is of fundamental importance in the final process of what will be later released. Table 10 shows the interface category, its subcategories’ complexity, presentation openness, and educational potential.

With the advent of computer graphics and the advancement provided for data analysis, the tools have greatly evolved their visual presentation over time. This is clear when we observe the eight revised tools within the sub-categories: complexity, presentation, openness, and educational potential. As shown in Table 10, the ESI, EISD, and EDI tools contribute in a historical way to the development of methodologies in elaborating energy sustainability indicators but do not allow interaction with the end-user. In turn, newer tools such as SDEWES and RISE have interactive websites with rich visual material, richly illustrated reports, and elaborated methodologies that offer a high degree of openness.

Regarding the subcategory complexity, it was verified that half of the revised tools perform calculations as simple average and linear normalization to reach the final value of the index or indicator (ESI, EDI, SEDI, and RISE). Two tools have a medium complexity with decomposition methods (EISD) or Z-score (SISED) to reach the final value. Two other tools work with high complexity in the elaboration of the final result. The MEPI uses probabilistic functions and analysis of uncertainties because the weight of the indicators is attributed to qualities beyond quantities. The SDEWES has high complexity because it depends on five stages to construct the final index.

The complex themes covered by the tools must be understood and communicated simply and objectively, where interaction with the user and the results delivered to the general public need visual presentations, which facilitate the interpretation by considering the concepts of energy and sustainable development. This helps and impacts directly on the formulation of public policies based on the perception of the players involved, regarding the change in social behavior.

The potential educational subcategory verifies the ability to represent the results to the public, which are achieved by the tool under analysis. It also addresses how much this educational potential can serve to inform the class of scientists in addition to the general public. Thus, it is possible to compare this subcategory with the information pyramid (Figure 1). ESI and EISD have historical value proven by the pioneering in the application and theoretical basis of great educational value, which are characteristics that serve to educate scientists. However, it does not present an interface of interaction with the end-user. EDI has great pedagogical value for scientists because its concept is very close to HDI, which also has educational potential for the general public, even if it does not interact with the end-user.

MEPI has an innovation in the approach for the concept of energy deprivation. SEDI allows the comparison between countries in a simplified way and SISED makes it possible to direct the results toward the elaboration of public policies, which makes these tools have a great pedagogical value for both scientists and the general public, even though none of them have an interactive interface with the general public. This lack of interactivity was overcome by SDEWES and RISE. Both tools have a high educational value and allow a high degree of interaction with the end-user, which also serves as a pedagogical potential for scientists to develop new tools based on their visual interface.

5. Conclusions

The objective of this work was to conduct a conceptual review of eight tools to measure energy sustainability and to elaborate on a qualitative comparison between them. A conceptual framework was created based on selection and comparison criteria. The categories used in the comparison follow the general assumption that the revised tools should represent socio-environmental aspects relevant to society, which have a practical and efficient measurement process help the manager of energy policies in the decision-making process.

Energy sustainability indices and indicators serve as a basis for learning about society, the economy, the environment, institutions, technologies, and the interactions between these areas. When carefully chosen and effectively communicated, it can provide information in a politically neutral manner, contribute to engaging the government and citizens in a shared debate on the meaning of sustainability and energy, with the final purpose of developing common political objectives. As shown in this paper, the effective problem in measuring energy sustainability is related to the use of one or more tools that capture the entire complexity of the system, without reducing the significance of each of the scopes used.

In this context, a considerable challenge is to overcome the existing limitations in the use of predominantly quantitative methodologies. The concept of sustainable development is related to different dimensions, which are not necessarily associated with measurable physical quantities. Examples of this are the social and institutional dimensions because, even if it is possible to associate these dimensions with quantitative indicators, this association suffers limitations according to the variable that is sought to observe. In recent years, different systems have been looking to work with the dimensions of sustainability qualitatively and this approach almost always reveals aspects that are imperceptible in quantitative analysis. The great challenge regarding using the qualitative approach is being able to formulate tools that do not sacrifice the advantages of using quantitative indicator systems.

Some important points highlighted in the research show that the interface of the tools is also an important element within an evaluation system. The theoretical discussion of this work revealed that the effectiveness of the tool is related to the ability to increase the critical awareness of the actors involved, which is promoted by the level of user interaction with the tool. It was shown that the most current tools already have online interactivity and some allow real-time updating, according to the database used. The aggregation of data itself is usually carried out using mathematical models that contribute to simplifying the inherent complexity of the evaluated system.

Based on the comparative and qualitative approach of the work, it can be said that the implicit concept of a tool’s energy sustainability becomes apparent through its dimensions and selected indicators. The approach in terms of participation determines the actors involved in choosing the indicators. Therefore, an evaluation system reflects the values or beliefs of certain actors to the concept of sustainable development and it is important to verify which systems are used in selecting these indicators and their use.

The concepts related to sustainable development may not be quantitatively associated with measurable physical quantities, and, thus, a qualitative analysis helps to overcome these limitations. The results of the research showed characteristics such as dimensions used, typology and data aggregation, degree of openness of the tools, complexity in use, educational potential, and participation of the actors in each tool studied. This is one of the reasons for the emergence of new qualitative indicators, especially in the last decade. Increasingly, there will be a need for qualitative approaches to assist in interpreting complex natural phenomena and the various dimensions that comprise sustainability. The present research meets these needs and seeks to show a possible way to fill this gap.

Lastly, some suggestions for future research include using the proposed conceptual framework to compare new tools, perform a quantitative analysis of the evaluated tools and correlate the results with the qualitative analysis, expand the qualitative analysis proposed for some important concepts in the energy sustainability assessment processes and the actors that participate in it, such as legitimacy, culture, and power, evaluate the selected tools based on the proposed analysis categories in the perception of its users, and validate the tools with a practical application in different local and regional realities.

Author Contributions

R.N.M. was initially responsible for conducting the research. R.M.K., J.A.S.d.S. and B.R.P.d.R. reviewed the theoretical and technical concepts of the article. S.F.S., A.N., E.C.F. and L.H.M. reviewed the text and its organization. W.G.B. adjusted references and the template. All authors have read and agreed to the published version of the manuscript.

Funding

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Acknowledgments

The authors would like to thank CAPES (Coordination of Higher Level Personnel Enhancement) for supporting research through the Pro-Integration Project, which integrates the High Amazonian Studies Center (NAEA) and the Graduate Program in Electrical Engineering (PPGEE) of the Federal University of Pará (UFPA), the Graduate Program in Natural Sciences (PPGCN) of Pará State University (UEPA), the Space Research Institute of the Regional Center of the Amazon (INPE), and the Operations and Management Center of the Amazon Protection System (CENSIPAM). One of the authors would like to acknowledge FAPESC under project code 2019TR84 for its financial support in this research.

Conflicts of Interest

The authors declare no conflict of interest.

Figures and Tables

Figure 1. Information pyramid [88,89].

Figure 2. Methodological structure.

Table 1

Dimensions and meanings of ESI indicators.

Dimension	Indicator	Significance
Economic	Energy autarky	Own energy resources ensure energy supply over a long period and are measured by the degree of dependence on imports from one country
	Soundness in the face of external changes	Measures a country’s economic vulnerability due to changes in international market conditions
	Energy productivity	This indicator is the inverse of the energy intensity and measures the output per unit of energy
Environmental	Use of renewables	Measures the share of renewable energy in the energy matrix
	Scope of fossil resources and firewood	Measures the depletion of fossil fuels and firewood, considering the reserve-production ratio
	The relative purity of energy use	Measures carbon dioxide emissions from energy, with low emission levels indicating high sustainability
Social	Coverage of basics energy needs	It captures whether families use enough amounts of useful energy
	Electric power coverage	A high percent of electrified households

Table 2

Composition of Energy Development Index (EDI).

Index	Dimension	Indicator
EDI	Residential	Share of the population with access to electricity
		Modern fuel share of the residential sector
	Communitarian	Per capita consumption of public sector electricity
		Share of productive use of energy in total final consumption

Table 3

Dimensions and indicators of Regulatory Indicators for Sustainable Energy (RISE).

Dimension	Indicators-Policies and Regulations
Electricity Access	Existence and implementation of electrification plan	Grid electrification	Affordability of electricity	Utility creditworthiness
	Scope of electrification plan	Mini-grids	Utility transparency and monitoring	Standalone systems
Clean Cooking	Planning	Scope of planning	Standards and labeling	Incentives and attributes
Energy Efficiency	National energy efficiency planning	Mandates and incentives: utilities	Minimum energy performance standards	Transport energy efficiency
	Energy efficiency entities	Mandates and incentives: large consumers	Energy labeling system	Financing mechanisms for energy efficiency
	Information provided to electricity consumers	Mandates and incentives: public entities	Building energy codes	Carbon pricing and monitoring
	Types of electricity rate structures
Renewable Energy	Legal framework for renewable energy	Incentives and regulatory support for renewable energy	Network connection and use	Counterparty risk
	Planning for renewable energy expansion	Attributes of financial and regulatory incentives	Carbon pricing and monitoring

Table 4

Dimensions, categories, and variables of the SISED index [134].

Dimension	Category	Variable
Security of Energy Supply (SES) Evaluation	Energy Dependence	Import of fuels, oil, or natural gas
	Differentiation of the type of primary energy production	Renewable energies, gas, petroleum, nuclear energy
	Energy consumption	Gas consumption, consumption of petroleum products, electricity use, consumption of solid fuels
Competitive Energy Market (CEM)	Energy Efficiency	Energy used with regard to losses
	Power market	Productivity and resource management
	Competitive prices of energy	Price of electricity, gas, and fossil fuelsEnergy storage for resource managementLarge energy consumersEnergy planningInvestment in new energy sources
Environmental Protection (EP)	Renewable Energy Sources	Share of Renewable Energy Sources in electrical energy production, primary energy production, and final energy consumption
	Emitted CO2	Energy intensity of total Greenhouse Gas (GHG) emissions per capita Share of energy industry emissions and on total GHG emissions
	Taxes applied to energy	The tax rate applied to energy

Table 5

Summary table of revised energy sustainability assessment tools.

Tool	Acronym	Grouping	Initiator	Year Creation	Reference
Energy Sustainability Indicators	ESI	Group Indicators	OLADE, ECLAC, GTZ	1997	[104]
Energy Indicators for Sustainable Development	EISD	Group Indicators	IAEA	1999	[61,86,107]
Energy Development Index	EDI	Composite Index	IEA, OECD	2004	[111]
Multidimensional Energy Poverty Index	MEPI	Composite Index	OPHI	2012	[112]
Sustainable Energy Development Index	SEDI	Composite Index	Researchers	2012	[44]
Regulatory Indicators for Sustainable Energy	RISE	Group Indicators	World Bank	2014	[130,131,132]
Sustainable Development of Energy, Water, and Environment Systems	SDEWES	Composite Index	SDEWES Centre	2015	[119,123]
Synthetic Index of Sustainable Energy Development	SISED	Composite Index	Researchers	2016	[134]

Table 6

The comparative framework according to the scope criterion.

Tool	Scope
	Dimension	Dimension Name
ESI	3	Economic, environmental, social
EISD	3	Economic, environmental, social
EDI	2	Residential, community
MEPI	5	Cooking, lighting, household appliances, entertainment/education, communication
SEDI	5	Economic, environmental, social, institutional, technical
RISE	4	Electricity access, clean cooking, energy efficiency, renewable energy
SDEWES	7	Energy consumption and climate, penetration of energy and CO2 saving measures, renewable energy potential and utilization, water and environmental quality, CO2 emissions and industrial profile, city planning and social welfare, R&D, innovation and sustainability policy
SISED	3	Security of energy supply, competitive energy market, environmental protection

Table 7

The comparative framework according to the sphere criterion.

Tool	Sphere
	Global	Continental	National	Regional	Local
ESI	yes	yes	yes	yes	yes
EISD	--	--	yes	yes	yes
EDI	--	--	yes	yes	--
MEPI	--	--	--	--	yes
SEDI	yes	yes	yes	--	--
RISE	yes	yes	yes	--	--
SDEWES	--	--	--	--	yes
SISED	yes	yes	yes	yes	yes

Table 8

The comparative framework according to the participation criterion.

Tool	Participation
	Approach	Comments
ESI	top-down	Experts determine the final index without interference from social actors.
EISD	top-down	Experts determine the final index without interference from social actors.
EDI	top-down	Experts determine the final index without interference from social actors.
MEPI	hybrid	The final index is determined by specialists with the participation of social actors.
SEDI	top-down	Experts determine the final index without interference from social actors.
RISE	hybrid	Social actors participate in the selection of indicators but do not have access to their weights, being restricted to specialists.
SDEWES	hybrid	Experts determine the indicators and final index, but social actors can interact with dynamic tables and are encouraged to use the results of the index to assess the performance of a specific city.
SISED	top-down	Experts determine the final index without interference from social actors.

Table 9

The comparative table according to data criterion.

Tool	Typology of Data	Data Aggregation
		Index	Sub-indices	Indicators	Sub-Indicators	Aggregate Data	Primary Data
ESI	Quantitative	--	--	8	--	Yes	Yes
EISD	Quantitative	--	--	30	19	Yes	Yes
EDI	Quantitative	1	--	4	--	No	Yes
MEPI	Quantitative/Qualitative	1	--	6	6	Yes	Yes
SEDI	Quantitative	1	--	11	--	Yes	Yes
RISE	Quantitative/Qualitative	--	--	32	102	Yes	Yes
SDEWES	Quantitative/Qualitative	1	--	35	--	Yes	Yes
SISED	Quantitative	1	3	9	33	Yes	Yes

Table 10

The comparative table according to the interface criterion, with its four subdivisions.

Tool	Interface
	Complexity	Presentation	Openness	Educational Potential
ESI	Low—is a set of group indicators, with simple independent calculations and linear normalization.	It does not have a general numerical index because it is a set of group indicators. The visual presentation depends on each specialist, varying in comparative charts of bar, dispersion, or radar.	High—displays the data used with its respective sources. It also shows the weights and normalization methods of each indicator used.	It has its educational value associated with historical value, as it is a pioneering tool in energy-based sustainability analysis. It has no interaction interface with the end-user.
EISD	Medium—comprises a set of indicators, where each is calculated independently through its methods, ranging from a simple average to a decomposition method.	Since it is a set of group indicators, it does not have a general numeric index. Each application of the methodology allows obtaining area, line, bar, and dispersion graphs for visual presentation of the results.	High—with the primary data and aggregates referenced as well as the method chosen to calculate each of the indicators.	It is a tool geared to facilitate the elaboration of energy policies geared toward sustainability. It does not have an interactive interface with the user, and is used only by specialists.
EDI	Low—elaborated through four indicators, where each is expressed by the simple arithmetic average, with aggregation index varying between 0 and 1. Follows the logic used to obtain HDI.	It has a general numerical index that compares countries through the bar and scatter charts. It also compares the general index obtained with HDI.	Medium—it presents the primary and aggregate data, but it does not make clear the value of the weights of the indicators used to calculate the final index.	It has high pedagogical value because it is an approach similar to HDI where it relates to energy. It has no interface for end-user but has several richly illustrated reports.
MEPI	High—due to the weights being attributed to qualities beyond quantities, probabilistic functions are defined to calculate the indicators that compose the final index, as well as analysis of uncertainties through the Monte Carlo method.	It has a general numerical index. It uses maps, tables, bar charts, scatters plots, and probabilistic density functions for visual presentation of the results.	High openness—it shows the primary data, aggregates, and details the weights of the indicators used to calculate the final index.	High educational potential, being a tool of analysis differentiated from the others because its methodology is based on the concept of energy poverty and its intensity.
SEDI	Low—The general index is obtained by the simple average of five indicators referring to the dimensions worked. The aggregation technique follows the same logic used in EDI and MEPI.	Compares the general numerical index obtained with other tools and uses tables, bar, radar, and scatter charts to show the results.	High—providing information about the required data, data source, and the weight of each indicator used to calculate the final index.	High educational potential, with the possibility of comparing countries from different regions in a simple way. It does not allow user interaction with the tool.
RISE	Low—The score of the indicators is weightless and simple arithmetic averages. It allows the user to adjust the most relevant weights of each indicator through the website of the project.	It does not use a general numeric index, but rather a set of indicators. The visual part is well elaborated and shown in graphics (pizza, bars, radar, scatter) with a very interactive website.	A high degree of openness. Displays data sources and allows user interaction in the form of an online tool with access to data, spreadsheets, and graphics.	High educational potential with a strong degree of user interaction with the tool. It also produces illustrated reports available in an open form on the website.
SDEWES	High—Five stages lead to the construction of the index: selection, data collection, normalization by the Min-Max method, and aggregation of data.	It uses a general numerical index to rank with a visual presentation in the form of bar, radar, and spatial mapping charts. It has a dynamic website with a great presentation of the data and results achieved.	High openness with a detailed presentation of the input data of each indicator used and their respective weights.	An innovative tool with high educational potential for sustainable urban planning. Contributed to the formulation of public policies.
SISED	Medium—The final synthetic index is obtained by the arithmetic average of three indices, which are calculated by the Z-score method of each one.	It uses a general numerical index and three intermediate indices with a visual presentation in bar and radar charts (polar).	A high degree of openness. The tool shows the data used, its sources, and respective weights involved in the calculations of the indices.	High educational potential because it works directly with comparative results aimed at the elaboration of public policies.