Experimental Validation of Systems Engineering Resilience Models for Islanded Microgrids

Abstract

Microgrids are used in many applications to power critical loads that have significant consequences if they lose power. Losing power to medical centers, water treatment plants, data centers, national defense installations, airports, and other critical infrastructure can cause loss of money and loss of life. Although such microgrids are generally reliable at providing stable power, their resilience to disruption can be poor. Common interruptions include natural disasters like earthquakes, and man-made causes such as cyber or physical attacks. Previous research into microgrid resilience evaluation efforts centered on theoretical modeling of total electrical microgrid loading, critical electrical load prioritization, assumed capacity of renewable energy sources and their associated energy storage systems, and assumed availability of emergency generators. This research assesses the validity of two microgrid resilience models developed for analyzing islanded microgrids by using experimental data from a scaled microgrid system. A national defense context is provided to motivate the work and align with the intended purpose two microgrid resilience models. The results of this research validate that the simulation models are valid to use in some situations, and highlight some areas for further model improvement.

1. Introduction

Civilian infrastructure and national defense installations require resilient and reliable electrical power to ensure operations, and meet operational and mission requirements. While the power grid is cost effective in providing power for hospitals, ports, food storage facilities, water treatment and sanitation plants, national defense organizations such as the United States Department of Defense (DOD), and other critical infrastructure, the power grid is susceptible to disruptions which may impact operations. Weather events are a major contributor to power disruptions in the United States [1] and were found to cost billions annually [2]. Within the realm of national defense, disruptions risk exposing operational and strategic capabilities to interruption which are challenging to attach a monetary value to [3]. Recent extreme weather events have also exposed critical vulnerabilities in centralized grid power generation to lengthy disruptions where users were left without power due to extreme cold weather and wildfire risk. Examples include the Texas blackouts caused by gas power plant shutdowns in the extremely cold winter storms in February 2021 [4] and the California ’Public Safety Power Shut-Offs’ during periods of severe wildfire risk in 2019 [5].

There is sustained interest by many critical infrastructure sectors including the DOD and other national defense organizations to improve energy resilience [6]. Organizations such as the DOD also aim to consume more energy from renewable sources to substitute current strategies of using diesel generators for backup power. Microgrids utilizing renewable energy sources are an emerging system of interest to improve resilience. In the event of power interruption, the microgrid is able to continue to provide power in islanded mode operations using distributed energy resources [7] and without the need of diesel fuel resupply [8]. As such, there has been sustained interest in the analysis of critical infrastructure microgrids, including national defense microgrids, in order to improve resilience from a systems engineering perspective. Although there are several commercial tools available for microgrid design optimization, the main focus of the tools is to optimize the microgrid for cost [9,10]. These tools do not fully address resilience analysis in the context of national defense and other critical infrastructure settings where monetary damages due to an outage are difficult or impossible to define. Recent research into systems engineering measures of microgrid resilience for critical infrastructure and national defense installations has resulted in the development of simulation models to conduct resilience analysis [11,12]. However, these models have not been validated against real-world hardware data. To validate that the models closely match reality and build trust in the model results, experimental validation can be useful.

This article uses a scaled experimental microgrid setup to validate some of the simulation models used for analyzing microgrid resilience in critical infrastructure applications where monetary damages cannot be assigned to outages such as with national defense. Two microgrid resilience simulation models are discussed and the scaled experimental microgrid parameters are used as simulation inputs. The results of the models are then compared with data obtained from the experimental setup. A close result after comparison of the experimental model and the simulation model is used to validate the simulation models.

2. Background and Related Research

Many critical infrastructure sectors have facilities that contain advanced systems that require electrical and computing power to support complex system functions. The importance of electricity cannot be overemphasized as the lack of electrical power could disable all modern communications, cause hospitals to be unable to treat complex cases, stop water treatment and wastewater plants, spoil refrigerated and frozen food in warehouses, and disable national defense installations that are involved in radar defenses, sever lines of communication required for command and control of defense systems, and cause many other issues. The requirement for resilient energy for national defense is emphasized in Title 10, section 101 in the United States Code, which outlines the role of the armed forces defines energy resilience as “…the ability to avoid, prepare for, minimize, adapt to, and recover from anticipated and unanticipated energy disruptions in order to ensure energy availability and reliability sufficient to provide for mission assurance and readiness, including mission essential operations related to readiness, and to execute or rapidly reestablish mission essential requirements” [13]. Key points in the quote include the ability to minimize energy disruption and to recover from it. Microgrids can support energy resilience requirements for critical infrastructure including national defense. The remainder of this section provides background and discussion on microgrid architecture and microgrid resilience measures in a systems engineering context.

2.1. Microgrids

Electric utilities, regardless of ownership type (public or private), have been found to have similar performance and only differ slightly in pricing structure and reliability [14]. Regardless of utility ownership type, utilities must provide a reliable supply of power to customers at a reasonable cost to be viable. This balance between reliability and cost compels utilities to build infrastructure to operate under typical historical conditions. As such, occasional interruptions occur during abnormal weather which may inconvenience civilian customers and lead to some monetary loss to corporate customers [15]. Power interruptions at critical infrastructure facilities and installations including national defense installations may impact national security and the consequences may be more severe, and can be hard to quantify monetarily. National security requirements and the increasing number of abnormal weather events have elevated interest in improving electrical resilience though the deployment of microgrids. Recent publications have also asserted that microgrids are successful and practical in improving resilience [16,17,18].

The U.S. Department of Energy has adopted the widely cited definition of a microgrid which was developed by the Microgrid Exchange Group. It defines a microgrid as “a group of interconnected loads and distributed energy resources within clearly defined electrical boundaries that acts as a single controllable entity with respect to the grid. A microgrid can connect and disconnect from the grid to enable it to operate in both grid-connected or island-mode” [19]. Crucially, the microgrid acts as a single controllable entity that can function regardless of whether it is connected to another grid. It requires the microgrid to be distinctly identifiable from the grid, locally interconnected and controlled, and lastly, functionally independent [20]. For the microgrid to be distinctly identifiable, it must have clear physical and functional boundaries. This is defined by the hardware and functional components that form the external interface for coupling to the utility grid, also known as the Point of Common Coupling (PCC) [21]. The microgrid controller and its local interconnection fulfill the requirement of being locally interconnected and controlled to balance power availability with load demands [22]. Finally, the requirement of being independent is fulfilled by the microgrid’s capacity to sufficiently cater to load demands within its boundaries. Therefore, to meet the functional requirements of a microgrid, the basic components include (1) Distribution System, (2) Distributed Generation Resources, (3) Energy Storage Systems, and (4) a Control and Communication System [23]. A basic microgrid system is presented in Figure 1.

2.1.1. Microgrid Distribution System

Microgrids typically utilize AC and/or DC distribution systems that match with the connected loads and generation sources. DC distribution systems have some advantages in certain cases by providing lower losses and higher transmittable power, while AC distribution has been a standard for a century and is widely used. DC distribution advantages include that DC power need not be converted to AC for Distributed Generation sources such as photovoltaic (PV) to the energy storage systems (ESS) such as batteries. Therefore, research work on standardizing DC distribution and control is ongoing and, in the meantime, hybrid AC/DC microgrids are sometimes used [24].

2.1.2. Distributed Generation Resources

There are a range of technologies that are available for microgrid power generation. Widely used systems includes diesel generators, solar PV systems, wind turbines, and micro-hydro [20]. Diesel generators consume fuel to generate electrical power; however, they are not a renewable energy source and fuel resupply is required for continued operations which can leave them vulnerable to disruption in certain scenarios [8]. Solar PV generates DC electricity from solar energy. The performance of the PV system is dependent on the system’s location which determines solar intensity and cloud cover. Solar PV is also not able to generate electricity after nightfall and its performance is degraded in winter months [25]. Wind turbines harness kinetic energy from wind with rotor blades and transform it into electric energy though a generator. Similar to the Solar PV system, their performance is also location dependent and can only generate electricity when the weather allows. The last common power generation technology is micro-hydro, which generates electricity from the flow of water and is dependent on topography and rainfall of the area.

2.1.3. Energy Storage Systems

The rapid reduction in cost of ESS and its central role in many microgrids has driven the development and the successful operation of microgrids with a significant percentage of generation (or all generation) being handled by renewable energy. ESS allows for the balancing of power and energy demands while providing uninterrupted transition from utility supply to the microgrid supply. Further, ESS can store renewable energy that is intermittently generated for later use. Essentially, the primary functions of an ESS allow for some of the following within the microgrid:

• Handle load fluctuations and power transients, and provide some time for generation sources to respond to the fluctuations;

• Ensure power supply stability when the power source is unstable;

• Handle microgrid transition from utility connected to islanded operations.

There can be a slight change in the system AC frequency when loads are added or removed from the system. This must be handled by sufficient ESS capacity to ensure the microgrid with several power generation sources is able to balance energy demand with generation following system loading adjustments. Common ESS for microgrids that are practical include batteries, fuel cells, flywheels, and super-capacitors. Batteries are the most common microgrid storage solution as they are the most affordable type of system. The most common type of battery deployed for microgrids are lead-acid batteries as they can support high currents in a very short period to handle power transients [26] during microgrid decoupling and they are capable of saving reserve energy for future demands although lithium chemistries are being increasingly used in some microgrids. The next type of ESS, fuel cells, is rapidly rising in popularity [27] as the technology matures. Fuel cells provide high efficiency by directly converting chemical energy from a fuel into electricity through a chemical reaction. This improves the practical performance of the microgrid [28] by reducing cost, improving energy efficiency, and microgrid reliability. When a fuel cell is implemented with an electrolyser, it can supplement batteries for energy storage as it has a high specific energy that can be used to soak up spare energy generated on the microgrid [29]. The last two types of ESS—flywheel and super-capacitor—are usually employed to improve power quality and as uninterruptible power supply for small loads [30].

2.1.4. Control and Communication Systems

IEEE Std. 2030.7-20.7 [31] specifies the general functional requirements for microgrid control to allow for standardization. The core functions described in the standard are the “dispatch” and “transition” functions for the microgrid. The microgrid controller dispatch function ensures balancing power generation and load when the system is in islanded mode, rebalancing of generation and load when there are changes in profiles, and responding to external control orders to meet interconnection agreement requirements. The microgrid controller transition function enables the system to transit between grid-connected mode and islanded mode without delay or power supply disruption to connected loads.

To achieve dispatch control, a unified rule-based control strategy with separate rules for grid-connected mode and islanded mode operations can be adopted [32]. In grid-connected mode, the reduction of power variation at the point of interconnection to the grid is prioritized to meet interconnection agreement requirements. For dispatch in islanded mode, the control objectives change to maintaining power balance while ensuring the microgrid components are operating within limits defined by the predefined rules. This ensures power balance, safe, and efficient operations of the microgrid. Transition between grid connected and islanded mode is initiated by one of the following three processes with steps described:

• Planned Islanding—the microgrid controller receives the islanding command, proceeds to balance load and generation, configure local controllers, disconnect point of interconnection, and achieve steady state islanded power dispatch on the microgrid;

• Unplanned Islanding—the microgrid detects islanded conditions, disconnect point of interconnection to create an island, configure local controllers, execute pre-configured control commands such as load shedding and achieves steady state islanded power dispatch on the microgrid;

• Reconnection to the grid—the microgrid controller synchronizes to the grid power, configures local controllers, reconnects the point of interconnection, and achieves steady state grid-connected dispatch mode.

Microgrid controllers rely on a robust communication system to enable core control functions. Centralized communication architectures were initially developed for microgrid controls, implementation was straightforward, and it met the microgrid requirements [33].

2.2. Microgrid Resilience

Natural disasters which include flooding, earthquakes, and hurricanes among many others [34,35] can cause severe power disruptions. Although such events are rare, the disruptions are severe and continue to cause economic loss after the disaster. Power system resilience has been studied and defined to improve the design of power systems to be able to withstand external shock or damage events, and to recover quickly [36]. A recent review of microgrid resilience found that accurate and realistic simulations are needed to design microgrids with better resilience. The examination of more realistic and general simulation frameworks would enable accurate comparison of different microgrid design and employment strategies [37].

There are various performance measures for microgrids such as those proposed by Lu, Wang, Zhang and Cheng which include reliability, economic (cost), practicality, and environmental sustainability as performance indices [38]. The reliability index measures the ratio of total unmet load to total electric load demand. The economic index measures the system cost effectiveness by computing the ratio of annualized cost for power generation to the total electrical load demand. Next, the practicality index computes the ratio of the total microgrid system occupied area to the available area for the system. Finally, the environmental sustainability index is computed from percentage of load demands met by renewable sources. This may be a comprehensive matrix for microgrid performance which incorporates design architecture elements like space practicality and environmental sustainability. Most microgrid assessments optimize system performance by maximizing reliability to meet an objective reliability value and minimizing system cost. This is therefore achieved by computing a reliability–cost objective function with reliability as a constraint, or with a predetermined investment amount [39,40]. Such methods focus on the reduction of operational cost for historical normal loads and do not focus on microgrid resilience [41].

2.2.1. Energy Resilience Definitions

Energy resilience must first be understood and measured before microgrid resilience can be analyzed. Various studies have reviewed quantitative measures of energy resilience from different perspectives that include design, identified threats to the system, and from different time periods. These measures typically use the resilience curve shown in Figure 2 where the disruption impacts the microgrid at time t_d [42]. Ideally, a resilient microgrid system would either have no or a small drop in performance at time t_d at the onset of the disruption event. The system would then need to maintain a stabilized supply before it could recover and this period is the recovery time. The invulnerability (the drop in performance at t_d) and recovery time form the key measures of the microgrid resilience. These measures are also depicted in Figure 2.

A common measure for resilience uses the ratio of the area under normal operational performance and the actual degraded performance after the disruption [43]. Other researchers include the definition of the components of resilience like absorption capacity or invulnerability, and the time taken to recover from the disruption [42,44].

Resilience is also threat-dependent [45] and the threats can be widely categorized as intentional attacks, including physical and cyber-attacks, and low-probability high-impact events such as extreme weather. As such, some researchers have introduced operational resilience and infrastructural resilience to analyze different operational and infrastructural resilience strategies [36]. This also highlights the difference between the reliability and resilience measures for the microgrid although both terms have frequently been used interchangeably [46].

Clark-Ginsberg uses a cyber incident to illustrate the difference between power reliability and resilience. When the cyber incident disrupts power supply, the system reliability is impacted as reliability is generally measured by the power supply’s ability to meet load demands. Resilience then measures how much the cyber incident disrupts (invulnerability measure) and how quickly the system is able to recover (recovery time). He then argues that by implementing rolling blackouts, the system maintains a high resilience while reliability continues to degrade. However, this views resilience from a single-load perspective as the recovery, and in this instance, when the lights go back on the system is considered to be ‘recovered’ prior to the next blackout in a rolling blackout scenario. However, should the resilience measure account for the grid system supply, rolling blackouts are considered as part of the stabilization and restoration phase [47]. To summarize, reliability provides a quantified metric for a system to function as specified and does not assess the system’s degraded functionality or ability to recover from failure or disruption. Resilience provides a measure of functionality of the system when degraded from a disruption event and the ability of the system to return to a state that is able to meet the functional requirements of the system [46].

2.2.2. National Defense Energy Resilience

The deployment of microgrids in national defense installations can improve electrical power security, reduce energy costs by incorporating renewable energy generation, meet national defense mission objectives, and provide supply in remote installations [48]. Therefore, national defense requirements differ from civilian microgrid application requirements (including many other critical infrastructure sectors) which typically only define the value of resilience in financial terms and only account for low-probability high-impact events. The value of national defence microgrid resilience is national defense and this can make it an attractive target for intentional attacks [49]. This widens the range of threats to the national defense installation energy security and examples include grid disruptions, component failure, damage due to disaster, and intentional physical or cyber attacks. Since national defense microgrids are unable to use just monetary value to measure microgrid resilience, there is no standard to define the value of energy resilience within the DOD [41]. Some examples of methods used include attempts to quantify resilience for national defense microgrids using the cost of implemented generators, the cost to relocate the mission for the duration of the disruption, or developing a method to compute a damage function based on the disruption duration [50,51]. Although there are differences in the computation for the value of energy resilience, researchers agree that the implementation of microgrids with distributed energy resources improves energy resilience [52,53].

2.2.3. Resilience Improvement

Mahzarnia, Moghaddam, Baboli, and Saino conducted a study to review measures to enhance power system resilience and found that there was a lack of comprehensive studies that considered power system resilience holistically. Although the impact of the topology and employment of renewable systems on resilience needs to be better understood for power distribution, they found that the investment of distributed energy resources, development of smart grid technologies, and the employment of microgrids to be useful [54]. For distributed energy resources, a study for distributed energy storage systems has found considerable improvements to power resilience [55], while smart grid technologies for fault isolation and service restoration provide quick analysis and decision support [56] that also enhances resilience. Microgrids have been shown to be invaluable in servicing connected critical loads in Tokyo after major disruption caused by a tsunami in 2011 and are able to support power restoration, network formation strategies, and power disruption prevention measures [16].

2.2.4. Resilience Assessment Methodologies

There is keen research interest in national defence microgrids to enhance power resilience and the diverse studies have spanned from cost trade space, to assessing the impact of power resilience on mission operations and cyber security [11,12,57,58]. To conduct an assessment or analysis, simulation models are a common feature of most studies. This article focuses on two methods gaining traction within the US Navy’s Naval Facilities Command (NAVFAC) and elsewhere that are used for analysis of national defense mission resilience impact, and the cost trade space for microgrids on islands, and they are discussed next.

Peterson’s model developed for the analysis of microgrid resilience and demonstrated on a national defense microgrid computes an electrical disruption mission impact metric for a microgrid model by determining the power flow within the microgrid. The mission impact metric accounts for periods when the load demands are not met and load shedding occurs. This model uses reference building load models from the Department of Energy and solar radiation data from the National Renewable Energy Laboratory together with user input of the microgrid design parameters such as energy storage and generation capacity to simulate the impact to mission when a disruption occurs [11].

Anderson’s model computes islanded microgrid resilience using invulnerability and recoverability metrics [12]. A case study of a national defense microgrid is provided in Anderson’s work. Invulnerability is computed from the ratio of power delivered and load demand, and recoverability is the computed ratio of power demand that is not met after the disruption event. The model varies the power generation from three power sources (diesel, solar, and wind) together with the energy storage system capacity to generate the metrics. Further details of both models are provided in subsequent sections.

2.3. Validation Process

The system engineering process comprehensively describes the key activities for a structured development process to realize a successful system. It is an interdisciplinary approach to integrating various disciplines and specializations that is initiated from system conception to operation. The system engineering V-Model which was developed in the 1970s has roots in software engineering and some early researchers used it as a tool to emphasize the importance of verification and validation [59]. The right side of the V-model depicts the integration phase of the system and consists of various verification and validation processes alongside the integration processes. The INCOSE Systems Engineering handbook describes both the verification and validation processes which have some similarities. Although both processes aim to provide objective evidence that the system meets requirements, the aims and scope differ. The verification process seeks to produce evidence that the system meets system and technical requirements and is generally at the lower right side of the V-model. This phase consists of the development phase where the systems and sub-systems are tested for acceptance. The validation process then tests the system in its operational environment after the system has been verified to meet system requirements. This validation process assesses if the system is suitable and meets operational needs. In this paper, the focus is on initial experimental validation of systems engineering resilience models.

3. Methodology and Simulation

This section introduces the systems engineering process and focus on the verification and validation method used to validate Peterson’s and Anderson’s microgrid resilience models. The systems engineering method for verification and validation is presented together with the strategy, identified inputs, activities and outputs [60]. In short, the verification and validation method used in this article follows these steps: (1) The simulation models are first adapted to a scaled down experimental microgrid for direct comparison. (2) Next, the results from the experimental microgrid are collected. (3) Then, the simulation models are run using the data such as solar irradiance from the experimental microgrid. (4) Finally, the experimental microgrid data and the simulation results are compared. The simulation and experiment are set up to illustrate a 72 h power disruption to allow for the microgrid to operate in islanded mode. This validation effort focuses on validation of electrical energy resilience of the simulation models and does not assess additional functions such as mission impact or cost effectiveness.

3.1. Peterson’s Microgrid Resilience Simulation Model

Peterson’s simulation model was developed to quantify microgrid resilience and investigate the impact to national defense installation missions [11]. It uses an expected electrical disruption mission impact metric to analyze mission impact ($MI$) and does not account for peripheral issues such as power factor and phase imbalance. This allows for the model to explore the high-level engineering trade space between power resilience and mission impact.

Peterson suggests using the Mission Dependency Index (MDI) [61,62] to determine the severity of impact to of a specific load suffering a power failure. MDI is one way to prioritize loads on a 0–100 scale with 100 being the highest priority to maintain power. However, there are criticisms of using MDI for such situations [63].

For the simulation to mimic real scenarios as closely as possible, hourly historical models for solar energy from the National Solar Radiation Data Base [64] and facility load demands from the US Department of Energy were used as model inputs [65]. The other user-defined input variables include the mission impact assessment and the ratio of critical loads of each facility modeled, a set of power interruption scenarios and the assessed recovery time. The simulation could then be initiated to produce mission impact results for a single run or utilize a series of Monte Carlo simulations to compute a mean for a more representative general result. Peterson calculates the mission impact for a single scenario (s) as the $MI$ per unit time (T) of the entire duration of the scenario,

(1)$M_s=\sum _{t=1}^{T}M{I}_{s,t}$

Peterson further proposes that the total impact of disruption events over all considered failure scenarios (the expected electrical disruption mission impact ($EEDMI$)) should be calculated as

(2)$EEDMI=\sum _{s\in S}Pr(S=s)M_s$

which quantifies the resilience of the system for the analyzed microgrid disruptions.

If MDI is used for $MI$, $EEDMI$ is unitless. However, a practitioner may choose a different metric fro $MI$ which can add units. $EEDMI$ is meant to compare between different microgrid architectures when identifying the most suitable candidate microgrid architecture for a collection of loads that best serves the loads. For the purposes of the analysis conducted in this paper, the authors have reduced Peterson’s calculations to focus only on one load and have thus reduced the $MI$ component to an assumed impact based on the total load not served during a disruption. This simplification is appropriate to validate Peterson’s power balance model and other model aspects.

The baseline microgrid system design used in the model consists of (1) Utility Grid Connection, (2) Diesel Generators, (3) Photovoltaic Solar Arrays, (4) Energy Storage Systems, and (5) Multiple Facility loads [11]. To compute mission impact, the model then categorizes load priority based on assessed criticality and adds a mission impact figure if the microgrid was unable to meet critical loads for the simulation. The summation of the mission impact measure will then be the result of one simulation. It was assessed that conducting an experiment to generate results for computing mission impact was not feasible as part of this article because failure distributions for distribution line components and generating systems were used in the Monte Carlo simulations. The experiment in this article instead focuses on the validation of power flow and system battery charge results which is used by the simulation to compute mission impact. The power flow and battery charge status graphs will be compared with experimental result graphs for an assessment. This baseline model is shown in Figure 3, where loads are indicated as EP with the respective power sources shown. Peterson’s baseline model was adjusted to match the scaled down experimental set up shown in Figure 3 and was used to produce data to compare against the experiment system. To facilitate a direct comparison, switches to loads and source components were not used in the experiment model and were set open, a constant AC load was used, and the power generation and storage system ratings were changed to match the experimental microgrid.

A simple power disruption scenario with indefinite utility grid power loss and with a 20 h loss of the diesel generator was simulated. This produced the simulated baseline result graph for the model that mimics the experiment microgrid shown in Figure 4. The result is consistent with expected power delivery behavior as the load has to draw from the battery once the microgrid is islanded. In the initial 20 h with no diesel generator, the battery charge is consumed until there is sufficient solar power to meet power demands. After the diesel generator is recovered, demand is supplied by the diesel generator and does not further drain battery charge. When solar energy is available in the day, the photovoltaic solar system supplies power to the load and negative value power indicates power flow to the batteries for charging. However, it is noted that in this model, although there was surplus power from the 2 kW diesel generator after hour 20, the battery charge only increased when there was solar power in excess of the load of 0.7 kW. In other words, the model does not allow for ESS charging from the diesel generator. This becomes important in a subsequent section with experiments conducted on the physical microgrid.

3.2. Anderson’s Microgrid Resilience Simulation Model

Anderson’s microgrid resilience simulation model was developed to model resilience and system cost assessment to provide a resilience and cost trade space for high-level decision making [12]. The simulation model computes microgrid invulnerability and recoverability metrics to determine the resilience measure, and uses a cost model to estimate the cost of the modeled microgrid architecture. The model for resilience is stochastic and includes distributions for probability of damage to the microgrid components and resources. The recovery duration is also stochastic and based on the available repair resources generated in the simulation [66].

This paper evaluates Anderson’s simulation resilience model for verification. The model includes an equal weight for both invulnerability and recovery measures. The measures are computed as positive ratios shown in Equation (3). The simulated microgrid resilience curve function is shown in Figure 5 and it includes annotations for the invulnerability and recovery measures.

(3)$\mathrm{resilience}=0.5(\mathrm{invulnerability}+\mathrm{recovery})$

It should be noted that Anderson assigns equal weight to invulnerability and recovery. It is possible that a practitioner may wish to adjust this relationship to develop a resilience statistic that is more appropriate to their microgrid situation and priorities. Giving more weight to invulnerability may, for instance, drive decision-makers to fund microgrid upgrade projects that specifically address the depth of power loss during a power disruption. However, this may come at the cost of an extended recovery timeline.

The invulnerability measure is computed with the reduction in power delivered immediately after the disruption. The invulnerability measure is constructed by the ratio of power delivered ($P_{t_s}$) to load demands ($P_{t_d}$) shown in Equation (4). This is also described in various research work by Francis and Bekra as absorptive capacity and by Yodo and Wang as lost performance [42,67].

(4)$\mathrm{invulnerability}=\frac{P_{t_s}}{P_{t_d}}$

It is important to note that in Anderson’s model, power generated is not the total rated power available from all operating generation sources. Many microgrids that the authors have worked with are over-provisioned with generation capacity. For instance, at one microgrid the authors have investigated, more than five times the needed generation capacity is available and kept in reserve [12]. Thus, instead of potentially having invulnerability $>1$, Anderson recommends using actual power generated which corresponds to $P_{t_s}<=P_{t_d}$.

Recoverability is defined as the ratio of the area bounded by the demand and the reduced post disruption power delivered shown in Equation (5). This method provides an accurate indication of the microgrid’s ability to rapidly recover as it accounts for time after the disruption until it is recovered.

(5)$\mathrm{recoverability}=1-\frac{{\sum }_{t=t_d}^{t=t_r}{D}_t-G_t}{{\sum }_{t=t_d}^{t=t_r}{D}_t}$

Note that recoverability is only analyzed when $G_t<D_t$. Thus, recoverability $\in [0,1]$ where a microgrid that never recovers after a disturbance is represented by 0 and a disturbance that never disrupts power delivery to all microgrid loads is represented by 1. In the first case, this could represent when a microgrid on a remote island is completely destroyed and the decision to abandon the island is made rather than attempt to repair the microgrid and other damaged or destroyed critical infrastructure. In the second case, this could represent a microgrid that is built to successfully ride through a disruptive event such as a thunderstorm without dropping any attached loads.

The simulation resilience model requires 12 input variables which include energy generation resource parameters, energy storage capacity, probability of damage, and demand profile. Random variables like damage to the generation resource and component mean time to repair are generated by the simulation for use in the resilience model. The model then computes the resilience metrics of recovery, invulnerability, and time to recover. The simulation model includes a case study of a simplified national defense microgrid resembling a microgrid at a facility at Naval Station Rota, Spain, which has a diesel generator, a solar array, and an energy storage system to demonstrate the simulation model. This demonstration used historical demand data and used the model to generate a simulation of the loss of both solar and diesel generators in a tsunami event. Results of the system demand, system power rating, and power delivered are shown in Figure 6. As both the diesel generator and the solar generators have been damaged by the disruption, power can only be supplied by the energy storage system for the initial 24 h. Thereafter, the recovery of the solar and diesel generators allow the microgrid to meet demands at the time step of 65 h.

The baseline model inputs were then adjusted to match the scaled microgrid experiment and allowed to generate results with probabilistic random variable input for system damage and mean time to repair. This simulation model generates a 14-day scenario for the islanded microgrid and a sample result of the resilience curve is shown in Figure 6. The diesel generator in this scenario was damaged and was repaired at time 175 h. The battery was able to sustain the load until hour 12 and was fully depleted at hour 19 while the solar panels was only able to support part of the load requirements during the day. This solar power cycling between day and night can be seen between hour 24 to 175 until the diesel generator was recovered.

4. Scaled Microgrid Experimental System

The validation experiment was done on a scaled microgrid system with commercial off the shelf systems consisting of an integrated controller, inverter, and charger system; a photovoltaic array; a bank of batteries; and a generator. The system monitors the microgrid power and generates an hourly log of the respective component power generation and demands. This data log is then used for the experiment and to verify against the simulated model data. Details of the main components of the system include:

• 2 kW Integrated controller, inverter and charger system (FXR2524A, from OutBack Power, Bellingham, WA, USA);

• 1.2 kW Static array of twelve 100 watt solar panels are mounted on the roof with no obstruction to the light from the sun. (RNG-100D-R-BK, from Renology, Ontario, CA, UAS);

• 12 kWh Battery bank (SLR500-2, from GS Battery (USA) from Kyoto, Japan);

• 2 kW Gasoline generator (Hybrid Series H03651, from Firman, Peoria, AZ, USA).

The integrated controller, inverter, and charger are central to the system with solar, grid, and generator power generation sources. The battery is also connected and can be charged by any available source, and it can supply power when there is a disruption causing power generated to fall below the load demands. The experimental system setup diagram with power flow direction represented by arrows is shown in Figure 7 below.

To mimic power disruption on the experimental microgrid, the system is configured to consume battery power when grid power and solar power is insufficient to support load power demands. The system will then attempt to utilize the generator to charge the battery when its state of charge falls below a set threshold. It is important to note that microgrids optimized for different outcomes such as reducing greenhouse gas emissions or providing maximum resilience may use diesel generators differently with respect to battery charging. In the case of optimizing for greenhouse gas emission reduction, the diesel generator would not be used to charge the battery. However, a microgrid that is crucial to a national security mission may be configured to use the generator to charge the battery when it falls below a certain threshold to maximize the resilience of the system. The authors have observed both operating configurations in microgrids currently in use at a variety of critical infrastructure facilities including national defense installations. The experiment utilizes a grid power disruption and hourly logs of system power are used to assess Peterson’s and Anderson’s resilience models.

5. Experimental Results

The scaled microgrid experimental system was integrated and tested in the configuration discussed in the methodology and simulation section above. This section discusses the results from the experimental system described in the previous section, and assesses the two microgrid resilience analysis models against the experimental system. The baseline microgrid operational performance without any power disruptions is first discussed before it is configured to simulate a power disruption. Key system parameters such as the battery state of charge and power within the microgrid are presented.

5.1. Experimental Microgrid Baseline Results

Initially, the microgrid is connected to the utility grid and uses grid power to meet power demands when grid power is available. With renewable energy resources connected, the microgrid controller utilizes power generated by the renewable resource and supplements it with utility grid power if power delivered by the renewable resource is not sufficient to meet load demands. The baseline scenario was set up with a 0.7 kW load demand over 32 h as shown in Figure 8. This experiment was conducted between 23 and 24 July 2022 in Monterey, CA, USA.

As there were no power disruptions, the battery state of charge (SOC) was maintained close to the full level and the load demand was supplied by the renewable resource and the utility grid. In this baseline scenario, the power delivered matches or exceeds the power demands. It can be seen in the data graph in Figure 8 that the power delivered from the utility grid is reduced when there is available power from the solar panels. In addition, total power delivered exceeds load demands when solar power is available. This small difference is due to inverter efficiency loss. This behavior of the microgrid under normal power conditions is within expectations and consistent with deployed systems [68].

5.2. Experimental Microgrid Simulated Disruption Results

As the experimental microgrid system is installed in Monterey, California, USA, with benign weather and a reliable utility grid, a simulated disruption is next conducted to carry out the experiment to examine microgrid resilience and power behavior. The test scenario used for the experiment was a power disruption of the utility grid power. To simulate the scenario on the microgrid, the system was configured to prioritise power from the batteries, and grid power was limited to 2 kW to simulate generator power. Data were collected over a 70 h utility supply disruption with the microgrid operating within battery charge constraints, with available solar power, and within simulated diesel generator constraints. The test was conducted on 15 July 2022. The microgrid maintained delivery of power to meet load demands for the whole test and charged the batteries when it fell below the 70% (8.4 kWh) threshold. Solar power during daylight hours was able to support load demands and reduce the rate of battery discharge, and this can be clearly seen in the results graph in Figure 9. The battery charge takes a longer time compared to night hours to fall below the threshold. Using the simulated generator to charge the battery is consistent with many existing operational national defense microgrids that the authors have worked with.

To assess the resilience curve of this simulated utility grid power disruption on the experimental microgrid, the overall power rating, demand, and power delivered is illustrated in Figure 10. Power demands were met for the length of the disruption as there was sufficient power capacity from the batteries, solar power, and the simulated diesel generator. The system power rating shows the available microgrid power capacity during the power disruption. This differs from the component power graph shown in Figure 9 as the microgrid controller balances power delivery to meet demands and only depicts the power transferred within the system. The system resilience matrix can be computed from the results between t_d and t_r, and this will be presented in the model result discussion sections below. The minor differences between the power delivered and the constant 0.7 kW load was also noted and this was assessed to be attributed to component power efficiency losses.

The parameters used for the experimental microgrid are next used on Peterston’s and Anderson’s simulation models for a comparison of results. The generator power is controlled to toggle on and off in the same time interval as the experiment, and the load demand is kept constant at 0.7 kW. This aims to generate simulations on the two models to mimic the experimental results for assessment.

5.3. Peterson’s Simulation Model Results

Using the experiment parameters, the results of Peterson’s simulation model show that the system was not able to meet load demands for 3 out of 71 h. This simulation result was not expected as the simulated microgrid was assessed to have more than sufficient remaining power resources to meet power demands. The battery charge results were plotted and they showed that the battery continued to deplete even after diesel generator power was available. Unlike the experimental results in Figure 9, where the battery charges on excess power of the diesel generator, Peterson’s Simulation Model limits diesel generator power delivery to load demands. Therefore, as shown in Figure 11 the battery charge continues to deplete and at time 68 h, the system was not able to meet load demands until solar power was available.

5.4. Anderson’s Simulation Model Results

The experiment parameters were also applied to Anderson’s Simulation Model and it showed that the power generated was able to support the load for the duration of the test. This result is similar to the experimental results. As this was a high-level, low-fidelity simulation, minor effects such as power loss within the system were not included and the power delivered matches the demand as shown in Figure 12. With the overlay of the experimental data, it can be seen that the simulation model results are similar to the experimental data.

6. Result Analysis

The microgrid resilience simulation results from Peterson’s and Anderson’s Simulation Models are now compared with the experimental results for assessment. Apart from comparing the computed metrics, the results for the identified parameters are also computed and presented. This validates and highlights areas where the respective simulation model results differ from the experimental results.

6.1. Peterson’s Simulation Model Result Analysis

Peterson’s Simulation Model is able to simulate the utility power disruption of 70 h. The results report that the system was not able to handle load requirements for 3 h which is not congruent with the experimental results where the experimental microgrid was able to support load requirements for the duration of the disruption. Two observations of the results are now discussed: (1) the reducing battery charge level in Peterson’s model compared to the ability of the experimental microgrid to maintain battery charge, and (2) the higher solar power generation in the simulation model. The results for Peterson’s Simulation Model battery charge were computed and they show a clear positive trend as shown in Figure 13. Upon investigation, it was found that Peterson’s Simulation Model was not able to utilize generator power to charge the batteries and this continued to deplete the battery charge. While many microgrids are configured to not use the generator to charge the battery, the authors have observed a number of national defense microgrids that do charge the battery in this manner. It was further observed that Peterson’s Simulation Model does not have the ability to change this behavior, which indicates that additional work is necessary to enable this feature.

The peaks for Peterson’s Simulation Model solar power generation was also found to be higher than the experimental microgrid. The trend line for the residuals of Peterson’s model solar power, as shown in Figure 14 is below 0, showing that the solar generation in Peterson’s model was higher than the experimental results. This is caused by lower solar power generation efficiency due to the position of the static solar panels and weather conditions during the experiment versus the solar data used in Peterson’s model. However, the residuals trend line is flat, and close to 0, showing that the model results for solar power generation are similar to the experiment results. Peterson’s model could be improved with further refinement of the solar power generation model although the authors suggest the model is sufficient for high-level system resilience analysis.

The difference in battery charge conditions for Peterson’s Simulation Model is assessed to only have an impact on specific conditions where there is a periodic need to draw power from the batteries due to fluctuations in power generation. Although there are various configuration schemes for the microgrid to ensure power delivery to meet demands, the microgrid controller would be able to monitor the system conditions and maximize available power generation resources to meet demand. In the event that the batteries are unable to charge, the controller would have allowed the diesel generator to run continuously to meet power demands.

6.2. Anderson’s Simulation Model Result Analysis

The resilience measure for both the experiment and Anderson’s Simulation Model was computed using Equation (3) and the results were 0.816 for the experimental microgrid and 0.809 for Anderson’s Simulation Model. The difference of 0.7% in resilience measure is assessed to be small and power demands are fully met in both the experiment and Anderson’s Simulation Model results. The results for Anderson’s Simulation Model power rating were also computed and the trend line is flat and close to zero, as shown in Figure 15. This shows that Anderson’s model power rating results are similar to the experimental results.

7. Conclusions

The experimental microgrid results were useful in helping to validate the simulation models used for microgrid resilience research. This validation effort for the microgrid resilience models supports the effort to improve energy resilience in the increasing risk environment from extreme weather events and adversarial threats. The two microgrid resilience models (Peterson’s and Anderson’s) studied in this research have been used in exploring the trade-off between energy resilience versus resource distribution and cost in other research. Thus, it is useful to experimentally validate the models.

Although there were some differences in the power control for battery charging between Peterson’s Simulation Model and the experimental microgrid, other aspects of the model were found to be similar. One option to fix this variance is to include in the model input an option to allow the user to indicate if the system is designed to allow for diesel generator power to charge the batteries. In larger-capacity systems where connected energy resources run continuously, the impact to the simulation results would be minimal as the battery charge is utilized for bridging gaps in power delivery early in the disruption phase. Thereafter, standby generators would supply power with minimal fluctuations in supply. The model input parameters could also be improved to include the system solar panel position and orientation to improve computation of solar power efficiency.

Anderson’s Simulation Model resilience results and associated metrics were found to be similar to experimental results. This provides physical evidence that validates resilience, invulnerability, and recoverability used in Anderson’s model. These experimental results support the high-level microgrid architecture assessment simulation model that Anderson developed to be used for design and cost trade-off analysis.

It is important to note that both Peterson’s Simulation Model and Anderson’s Simulation Model are systems engineering tools. While they do analyze microgrids, they are not detailed electrical engineering models. They are meant to be used early in a system design process when making large architectural decisions about a microgrid. Later design efforts can use models such as Fish’s electrical microgrid simulation tool [69] are more appropriate for detailed design and simulation work.

One challenge with validating systems engineering models is the ability to achieve some degree of statistical significance or confidence in validation results. The research presented in this paper looked at one microgrid and one outage scenario conducted one time. Attempting to conduct many scenarios hundreds of times across many microgrids is at best impractical. There is an ongoing and open discussion in the systems engineering community about how to validate systems engineering models and methods. During the 2019 National Science Foundation Engineering Design and Systems Engineering workshop held at Purdue University, this issue was discussed extensively. Currently, the Validation Square approach of deductive logic [70,71] is used by some, while others conduct limited experiments such as the one presented in this paper. The authors view the research conducted in this paper as a first step on a long road toward the goal of verification and validation of microgrid resilience models fit for systems engineering purposes.

Battery energy storage can be used as a grid-forming asset for islanded microgrids. However, the authors have personally observed several large battery systems that cannot operate in grid-forming mode for a variety of bureaucratic reasons beyond the scope of this article to describe. Especially for station black start situations, having grid-forming battery systems is critical. The authors are currently working with several microgrid operators to upgrade existing batteries or purchase new battery energy storage systems that can support grid forming. In situations where grid-forming battery storage is unavailable, either diesel generators need to be kept in spinning reserve (operational and idle or kept spinning and hot via other means so they can start almost instantaneously) in order to endure an initial grid disconnect event and keep renewable generation sources online. In the authors’ experience, some microgrids contain very large diesel generators that can take up to a half hour to come online and be ready to accept a load from a cold start, which means that a microgrid may be completely offline for that entire period if a battery energy storage system is not available and capable of grid forming.

While only one microgrid using a mix of PV and simulated diesel generator power with a battery storage system was examined in this research, Anderson’s and Peterson’s models are capable of examining many different microgrid configurations. Microgrids that are fully reliant on diesel generation, such as is the case for some remote arctic communities, can be simulated. Conversely, microgrids that are fully renewable can also be simulated. While only common generation sources (diesel, wind, PV) are available in Anderson’s and Peterson’s models, it is readily achievable to include other generation sources with small modifications to the models.

Future Work

While this article investigated one scenario to validate Peterson’s and Anderson’s models, there is a need for future studies to continue to utilize real-world data to improve microgrid resilience simulation models. As this research was conducted on a scaled microgrid with commercial off-the-shelf components, it may not be able to highlight issues that may be present in the operation of larger microgrid systems. Some recommendations for future work include repeating the experiment on a scaled-up physical microgrid system with a power capacity close to deployed microgrid system, or to collect power disruption data from deployed systems when there is scheduled downtime.

For a more holistic result, more system factors could be included for the design of this experiment. Possible factors include the number of renewable energy resources, energy storage systems, and several repetitions with measured environment data. This will help refine simulation models and identify significant factors that influence microgrid resilience. The next experimental microgrid could also explore different power generation systems and topology design for assessment with the simulation model results.

During a disruptive event and in the aftermath, a variety of contingency plans may be executed by personnel associated with microgrid operations and recovery. Previous work has investigated the time it takes line crews to rebuild microgrid infrastructure after a disruptive event [72]. It may be useful to integrate contingency plans and planning into resilience simulation models to gain a better understanding of how a microgrid can either reduce the severity of an event (the invulnerability) or minimize the duration of the recovery.

Author Contributions

Conceptualization, D.L.V.B. and A.P.; methodology, J.J.H.; software, J.J.H.; validation, J.J.H.; formal analysis, J.J.H.; investigation, J.J.H.; resources, D.L.V.B.; data curation, J.J.H.; writing—original draft preparation, J.J.H. and D.L.V.B.; writing—review and editing, D.L.V.B. and A.P.; visualization, J.J.H.; supervision, D.L.V.B. and A.P.; project administration, D.L.V.B. and A.P.; funding acquisition, D.L.V.B. All authors have read and agreed to the published version of the manuscript.

Data Availability Statement

Please contact the authors for data and software used in this study.

Acknowledgments

The authors extend their thanks to Chris Frackell, Brandon Naylor, and Robert Lindstrom who assisted at various stages of research and experimentation.

Conflicts of Interest

The authors declare no conflict of interest.

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Figures

Figure 1. Components of a basic microgrid architecture.

Figure 2. Typical resilience curve showing the phases of disruption, and the key measures of invulnerability and recovery time.

Figure 3. Peterson’s Simulation Model architecture. Components not used in this analysis are indicated in gray. Adapted with permission from [11]. Copyright 2021, copyright Peterson, Van Bossuyt, Giachetti, and Oriti.

Figure 4. Peterson’s Simulation Model adjusted architecture results for battery state of charge and power flow.

Figure 5. Anderson’s Simulation Model resilience curve function for a power disruption with annotations for invulnerability and recovery measures.

Figure 6. Anderson’s Simulation Model resilience curve showing simulation of diesel generator and solar power disruption.

Figure 7. Scaled Experimental Microgrid Diagram showing connected power generation and demand components.

Figure 8. Experimental microgrid baseline result for a clear day with no power disruption is shown in this figure. It shows power delivered from energy resources, and power delivered to load demands.

Figure 9. Experimental microgrid simulated loss of utility grid disruption results.

Figure 10. Experimental microgrid simulated loss of utility grid disruption resilience curve results.

Figure 11. Peterson’s Simulation Model battery charge and component power over length of simulation.

Figure 12. Anderson’s Simulation Model power rating, power demands and power delivered over length of simulation overlaid with experimental data.

Figure 13. Peterson’s Simulation Model battery charge results show a positive trend because the model batteries were not charged by excess diesel generator power.

Figure 14. Peterson’s Simulation Model solar power residuals trend flat and slightly below 0 due to difference in assumptions for solar efficiency and weather conditions.

Figure 15. Anderson’s Simulation Model power rating residuals trend flat and close to 0.

References

1. Shield, S.A.; Quiring, S.M.; Pino, J.V.; Buckstaff, K. Major impacts of weather events on the electrical power delivery system in the United States. Energy; 2021; 218, 119434. [DOI: https://dx.doi.org/10.1016/j.energy.2020.119434]

2. Campbell, R.J.; Lowry, S. Weather-Related Power Outages and Electric System Resiliency; CRS Report No. R42696; Congressional Research Service, Library of Congress: Washington, DC, USA, 2012.

3. Sagunsky, D.L. Risk on the Horizon, Rig for Dark: Solutions to Mitigate DoD’s Reliance on the Fragile Electric Grid. Master’s Thesis; Joint Advanced Warfighting School: Norfolk, VA, USA, 2013.

4. Douglas, E. Texas largely relies on natural gas for power. It wasn’t ready for the extreme cold. The Texas Tribune; 16 February 2021.

5. Goss, M.; Swain, D.L.; Abatzoglou, J.T.; Sarhadi, A.; Kolden, C.A.; Williams, A.P.; Diffenbaugh, N.S. Climate change is increasing the likelihood of extreme autumn wildfire conditions across California. Environ. Res. Lett.; 2020; 15, 094016. [DOI: https://dx.doi.org/10.1088/1748-9326/ab83a7]

6. Department of Defense. Department of Defense Annual Energy Management and Resilience Report (AEMRR) Fiscal Year 2020; 8-BC91200; Department of Defense: Washington, DC, USA, 2021.

7. Padmanaban, S.; Nithiyananthan, K.; Karthikeyan, S.P.; Holm-Nielsen, J.B. Microgrids; Taylor and Francis: Boca Raton, FL, USA, 2021.

8. Anuat, E.; Van Bossuyt, D.L.; Pollman, A. Energy Resilience Impact of Supply Chain Network Disruption to Military Microgrids. Infrastructures; 2021; 7, 4. [DOI: https://dx.doi.org/10.3390/infrastructures7010004]

9. Stadler, M.; Cardorso, G.; Mashayekh, S.; DeForest, N. Distributed Energy Resources Customer Adoption Model Plus (DER-CAM+); Version 1.0.0; Technical Report DER-CAM+; 004669MLTPL00; Lawrence Berkeley National Lab. (LBNL): Berkeley, CA, USA, 2016.

10. XENDEE|Microgrid Design and Decision Support Platform. Available online: https://xendee.com/ (accessed on 16 July 2022).

11. Peterson, C.J.; Van Bossuyt, D.L.; Giachetti, R.E.; Oriti, G. Analyzing Mission Impact of Military Installations Microgrid for Resilience. Systems; 2021; 9, 69. [DOI: https://dx.doi.org/10.3390/systems9030069]

12. Giachetti, R.E.; Van Bossuyt, D.L.; Anderson, W.; Oriti, G. Resilience and Cost Trade Space for Microgrids on Islands. IEEE Syst. J.; 2021; 16, pp. 3939-3949. [DOI: https://dx.doi.org/10.1109/JSYST.2021.3103831]

13. United States House of Representatives. 10 USC. 1—DEFINITIONS; United States Government Publishing Office: Washington, DC, USA, 2020.

14. Pazzalia, J. Sparking Debate: Private or Public? The Effect of Ownership on Electric Utility Performance. Ph.D. Thesis; West Chester University: West Chester, PA, USA, 2022.

15. Schneider, K.P.; Tuffner, F.K.; Elizondo, M.A.; Liu, C.C.; Xu, Y.; Ton, D. Evaluating the feasibility to use microgrids as a resiliency resource. IEEE Trans. Smart Grid; 2016; 8, pp. 687-696.

16. Bajwa, A.A.; Mokhlis, H.; Mekhilef, S.; Mubin, M. Enhancing power system resilience leveraging microgrids: A review. J. Renew. Sustain. Energy; 2019; 11, 035503. [DOI: https://dx.doi.org/10.1063/1.5066264]

17. Van Broekhoven, S.; Judson, N.; Galvin, J.; Marqusee, J. Leading the charge: Microgrids for domestic military installations. IEEE Power Energy Mag.; 2013; 11, pp. 40-45. [DOI: https://dx.doi.org/10.1109/MPE.2013.2258280]

18. Hussain, A.; Bui, V.H.; Kim, H.M. Microgrids as a resilience resource and strategies used by microgrids for enhancing resilience. Appl. Energy; 2019; 240, pp. 56-72. [DOI: https://dx.doi.org/10.1016/j.apenergy.2019.02.055]

19. Ton, D.T.; Smith, M.A. The US department of energy’s microgrid initiative. Electr. J.; 2012; 25, pp. 84-94. [DOI: https://dx.doi.org/10.1016/j.tej.2012.09.013]

20. Hirsch, A.; Parag, Y.; Guerrero, J. Microgrids: A review of technologies, key drivers, and outstanding issues. Renew. Sustain. Energy Rev.; 2018; 90, pp. 402-411. [DOI: https://dx.doi.org/10.1016/j.rser.2018.03.040]

21. Giachetti, R.E.; Peterson, C.J.; Van Bossuyt, D.L.; Parker, G.W. Systems Engineering Issues in Microgrids for Military Installations. INCOSE Int. Symp.; 2020; 30, pp. 731-746. [DOI: https://dx.doi.org/10.1002/j.2334-5837.2020.00751.x]

22. Caldognetto, T.; Tenti, P.; Costabeber, A.; Mattavelli, P. Improving Microgrid Performance by Cooperative Control of Distributed Energy Sources. IEEE Trans. Ind. Appl.; 2014; 50, pp. 3921-3930. [DOI: https://dx.doi.org/10.1109/TIA.2014.2313648]

23. Mariam, L.; Basu, M.; Conlon, M.F. Microgrid: Architecture, policy and future trends. Renew. Sustain. Energy Rev.; 2016; 64, pp. 477-489. [DOI: https://dx.doi.org/10.1016/j.rser.2016.06.037]

24. Planas, E.; Andreu, J.; Gárate, J.I.; de Alegría, I.M.; Ibarra, E. AC and DC technology in microgrids: A review. Renew. Sustain. Energy Rev.; 2015; 43, pp. 726-749. [DOI: https://dx.doi.org/10.1016/j.rser.2014.11.067]

25. Mondol, J.D.; Yohanis, Y.; Smyth, M.; Norton, B. Long term performance analysis of a grid connected photovoltaic system in Northern Ireland. Energy Convers. Manag.; 2006; 47, pp. 2925-2947. [DOI: https://dx.doi.org/10.1016/j.enconman.2006.03.026]

26. Venkataramanan, G.; Illindala, M. Microgrids and sensitive loads. Proceedings of the 2002 IEEE Power Engineering Society Winter Meeting (Cat. No.02CH37309); New York, NY, USA, 27–31 January 2002; Volume 1, pp. 315-322. [DOI: https://dx.doi.org/10.1109/PESW.2002.985005]

27. Wood, E. The Rise of Fuel Cell Microgrids: Special Report; Distributed Energy. 2017; Available online: https://www.microgridknowledge.com/distributed-energy/article/11431516/the-rise-of-fuel-cell-microgrids-special-report (accessed on 16 July 2022).

28. Li, Z.; Zheng, Z.; Xu, L.; Lu, X. A review of the applications of fuel cells in microgrids: Opportunities and challenges. BMC Energy; 2019; 1, 8. [DOI: https://dx.doi.org/10.1186/s42500-019-0008-3]

29. Akinyele, D.; Olabode, E.; Amole, A. Review of Fuel Cell Technologies and Applications for Sustainable Microgrid Systems. Inventions; 2020; 5, 42. [DOI: https://dx.doi.org/10.3390/inventions5030042]

30. Ekanayake, J.B.; Jenkins, N.; Liyanage, K.; Wu, J.; Yokoyama, A. Smart Grid: Technology and Applications; John Wiley & Sons: Hoboken, NJ, USA, 2012.

31. IEEE Std 2030.7-2017 IEEE Standard for the Specification of Microgrid Controllers. IEEE: New York, NY, USA, 2018; pp. 1-43. [DOI: https://dx.doi.org/10.1109/IEEESTD.2018.8340204]

32. Sun, C.; Paquin, J.N.; Al Jajeh, F.; Joos, G.; Bouffard, F. Implementation and CHIL Testing of a Microgrid Control System. Proceedings of the 2018 IEEE Energy Conversion Congress and Exposition (ECCE); Portland, OR, USA, 23–27 September 2018; pp. 2073-2080. [DOI: https://dx.doi.org/10.1109/ECCE.2018.8557761]

33. Fang, X.; Misra, S.; Xue, G.; Yang, D. Smart Grid—The New and Improved Power Grid: A Survey. IEEE Commun. Surv. Tutor.; 2012; 14, pp. 944-980. [DOI: https://dx.doi.org/10.1109/SURV.2011.101911.00087]

34. Sancaktar, S.; Ng, C. Risk Assessment of Operational Events: External Events; United States Nuclear Regulatory Commission: Rockville, MD, USA, 2017.

35. Wielenberg, A.; Alzbutas, R.; Apostol, M.; Bareith, A.; Brac, P.; Burgazzi, L.; Cazzoli, E.; Cizelj, L.; Hage, M.; Hashimoto, K. et al. Methodology for Selecting Initiating Events and Hazards for Consideration in an Extended PSA; Technical Report ASAMPSA E and EURATOM: Fontenay-aux-Roses, France, 2016.

36. Panteli, M.; Mancarella, P.; Trakas, D.N.; Kyriakides, E.; Hatziargyriou, N.D. Metrics and Quantification of Operational and Infrastructure Resilience in Power Systems. IEEE Trans. Power Syst.; 2017; 32, pp. 4732-4742. [DOI: https://dx.doi.org/10.1109/TPWRS.2017.2664141]

37. Wang, Y.; Rousis, A.O.; Strbac, G. On microgrids and resilience: A comprehensive review on modeling and operational strategies. Renew. Sustain. Energy Rev.; 2020; 134, 110313. [DOI: https://dx.doi.org/10.1016/j.rser.2020.110313]

38. Lu, J.; Wang, W.; Zhang, Y.; Cheng, S. Multi-Objective Optimal Design of Stand-alone Hybrid Energy System Using Entropy Weight Method Based on HOMER. Energies; 2017; 10, 1664. [DOI: https://dx.doi.org/10.3390/en10101664]

39. Adefarati, T.; Bansal, R.C. Reliability, economic and environmental analysis of a microgrid system in the presence of renewable energy resources. Appl. Energy; 2019; 236, pp. 1089-1114. [DOI: https://dx.doi.org/10.1016/j.apenergy.2018.12.050]

40. Wang, S.; Li, Z.; Wu, L.; Shahidehpour, M.; Li, Z. New Metrics for Assessing the Reliability and Economics of Microgrids in Distribution System. IEEE Trans. Power Syst.; 2013; 28, pp. 2852-2861. [DOI: https://dx.doi.org/10.1109/TPWRS.2013.2249539]

41. Rusco, F.; Lepore, B.J.; Ludwigson, J.; Durland, L.; Barnes, T.; Biskup, E.; Ettaro, L.; Gerken, E.; Guinane, T.; Hanford, T. et al. DOD Renewable Energy Projects: Improved Guidance Needed for Analyzing and Documenting Costs and Benefits; Technical Report U.S. Government Accountability Office Washington United States: Washington, DC, USA, 2016.

42. Francis, R.; Bekera, B. A metric and frameworks for resilience analysis of engineered and infrastructure systems. Reliab. Eng. Syst. Saf.; 2014; 121, pp. 90-103. [DOI: https://dx.doi.org/10.1016/j.ress.2013.07.004]

43. Gholami, A.; Shekari, T.; Amirioun, M.H.; Aminifar, F.; Amini, M.H.; Sargolzaei, A. Toward a Consensus on the Definition and Taxonomy of Power System Resilience. IEEE Access; 2018; 6, pp. 32035-32053. [DOI: https://dx.doi.org/10.1109/ACCESS.2018.2845378]

44. Vugrin, E.D.; Castillo, A.R.; Silva-Monroy, C.A. Resilience Metrics for the Electric Power System: A Performance-Based Approach; Technical Report SAND2017-1493; Sandia National Lab. (SNL-NM): Albuquerque, NM, USA, 2017; [DOI: https://dx.doi.org/10.2172/1367499]

45. Haimes, Y.Y. On the Definition of Resilience in Systems. Risk Anal.; 2009; 29, pp. 498-501. [DOI: https://dx.doi.org/10.1111/j.1539-6924.2009.01216.x]

46. Albasrawi, M.N.; Jarus, N.; Joshi, K.A.; Sarvestani, S.S. Analysis of Reliability and Resilience for Smart Grids. Proceedings of the 2014 IEEE 38th Annual Computer Software and Applications Conference; Vasteras, Sweden, 21–25 July 2014; pp. 529-534. [DOI: https://dx.doi.org/10.1109/COMPSAC.2014.75]

47. Clark-Ginsberg, A. What’s the Difference between Reliability and Resilience?; Stanford University: Palo Alto, CA, USA, 2016; [DOI: https://dx.doi.org/10.13140/RG.2.2.20571.46885]

48. Kashem, S.B.A.; De Souza, S.; Iqbal, A.; Ahmed, J. Microgrid in military applications. Proceedings of the 2018 IEEE 12th International Conference on Compatibility, Power Electronics and Power Engineering (CPE-POWERENG 2018); Doha, Qatar, 10–12 April 2018; pp. 1-5. [DOI: https://dx.doi.org/10.1109/CPE.2018.8372506]

49. Venkataramanan, V.; Srivastava, A.K.; Hahn, A.; Zonouz, S. Measuring and Enhancing Microgrid Resiliency Against Cyber Threats. IEEE Trans. Ind. Appl.; 2019; 55, pp. 6303-6312. [DOI: https://dx.doi.org/10.1109/TIA.2019.2928495]

50. Rickerson, W.; Gillis, J.; Bulkeley, M. The Value of Resilience for Distributed Energy Resources: An Overview of Current Analytical Practices; National Association of Regulatory Utility Commissioners: Washington, DC, USA, 2019.

51. Giraldez, J.; Booth, S.; Anderson, K.; Massey, K. Valuing Energy Security: Customer Damage Function Methodology and Case Studies at DoD Installations; Technical Report NREL/TP-7A30-55913; National Renewable Energy Lab. (NREL): Golden, CO, USA, 2012; [DOI: https://dx.doi.org/10.2172/1055367]

52. Mishra, S.; Anderson, K.; Miller, B.; Boyer, K.; Warren, A. Microgrid resilience: A holistic approach for assessing threats, identifying vulnerabilities, and designing corresponding mitigation strategies. Appl. Energy; 2020; 264, 114726. [DOI: https://dx.doi.org/10.1016/j.apenergy.2020.114726]

53. Li, Z.; Shahidehpour, M.; Aminifar, F.; Alabdulwahab, A.; Al-Turki, Y. Networked Microgrids for Enhancing the Power System Resilience. Proc. IEEE; 2017; 105, pp. 1289-1310. [DOI: https://dx.doi.org/10.1109/JPROC.2017.2685558]

54. Mahzarnia, M.; Moghaddam, M.P.; Baboli, P.T.; Siano, P. A Review of the Measures to Enhance Power Systems Resilience. IEEE Syst. J.; 2020; 14, pp. 4059-4070. [DOI: https://dx.doi.org/10.1109/JSYST.2020.2965993]

55. Beaton, D.T. Testing Whether Distributed Energy Storage Results in Greater Resilience of Microgrids. Master’s Thesis; Naval Postgraduate School: Monterey, CA, USA, 2021.

56. Xu, Y.; Liu, C.C.; Schneider, K.P.; Ton, D.T. Toward a resilient distribution system. Proceedings of the 2015 IEEE Power & Energy Society General Meeting; Denver, CO, USA, 26–30 July 2015; pp. 1-5. [DOI: https://dx.doi.org/10.1109/PESGM.2015.7286551]

57. Castelo Branco, C.A.S.; Moraes, F.P.; Oliveira, H.A.; Neto, P.B.L.; Saavedra, O.R.; de Matos, J.G.; Oliveira, C.B.M.; Ribeiro, L.A.d.S.; Oliveira, A.C.; Júnior, M.F.A. et al. Mission Critical Microgrids: The Case of the Alcântara Space Center. Energies; 2022; 15, 3226. [DOI: https://dx.doi.org/10.3390/en15093226]

58. Sarker, P.S.; Venkataramanan, V.; Cardenas, D.S.; Srivastava, A.; Hahn, A.; Miller, B. Cyber-Physical Security and Resiliency Analysis Testbed for Critical Microgrids with IEEE 2030.5. Proceedings of the 2020 8th Workshop on Modeling and Simulation of Cyber-Physical Energy Systems; Sydney, NSW, Australia, 21–21 April 2020; pp. 1-6. [DOI: https://dx.doi.org/10.1109/MSCPES49613.2020.9133689]

59. Weilkiens, T.; Lamm, J.G.; Roth, S.; Walker, M. Model-Based System Architecture; John Wiley & Sons: Newark, NJ, USA, 2015.

60. INCOSEWiley. INCOSE Systems Engineering Handbook: A Guide for System Life Cycle Processes and Activities; John Wiley & Sons: New York, NY, USA, 2015.

61. Al Antelman, R.; Miller, A. Mission Dependency Index Validation Report; Technical Report Naval Facilities Engineering Service Center: Port Hueneme, CA, USA, 2002.

62. Smith, C.W. Mission Dependency Index of Air Force Built Infrastructure: Knowledge Discovery with Machine Learning. Master’s Thesis; WRIGHT-PATTERSON AFB: Dayton OH, USA, 2016.

63. Kujawski, E.; Miller, G. The mission dependency index: Fallacies and misuses. Proceedings of the INCOSE International Symposium; Wiley Online Library: Hoboken, NJ, USA, 2009; Volume 19, pp. 1565-1580.

64. Wilcox, S.M. National Solar Radiation Database 1991–2010 Update: User’s Manual; Technical Report NREL/TP-5500-54824; National Renewable Energy Lab. (NREL): Golden, CO, USA, 2012; [DOI: https://dx.doi.org/10.2172/1054832]

65. Deru, M.; Field, K.; Studer, D.; Benne, K.; Griffith, B.; Torcellini, P.; Liu, B.; Halverson, M.; Winiarski, D.; Rosenberg, M. et al. U.S. Department of Energy Commercial Reference Building Models of the National Building Stock; Technical Report NREL/TP-5500-46861; National Renewable Energy Lab. (NREL): Golden, CO, USA, 2011; [DOI: https://dx.doi.org/10.2172/1009264]

66. Anderson, W.J. Resilience Assessment of Islanded Renewable Energy Microgrids. Ph.D. Thesis; Naval Postgraduate School: Monterey, CA, USA, 2020.

67. Yodo, N.; Wang, P. Engineering Resilience Quantification and System Design Implications: A Literature Survey. J. Mech. Des.; 2016; 138, 111408. [DOI: https://dx.doi.org/10.1115/1.4034223]

68. Khodayar, M.E.; Barati, M.; Shahidehpour, M. Integration of High Reliability Distribution System in Microgrid Operation. IEEE Trans. Smart Grid; 2012; 3, pp. 1997-2006. [DOI: https://dx.doi.org/10.1109/TSG.2012.2213348]

69. Anglani, N.; Oriti, G.; Fish, R.; Van Bossuyt, D.L. Design and Optimization Strategy to Size Resilient Stand-Alone Hybrid Microgrids in Various Climatic Conditions. IEEE Open J. Ind. Appl.; 2022; 3, pp. 237-246. [DOI: https://dx.doi.org/10.1109/OJIA.2022.3201161]

70. Pedersen, K.; Emblemsvåg, J.; Bailey, R.; Allen, J.K.; Mistree, F. Validating design methods and research: The validation square. Proceedings of the International Design Engineering Technical Conferences and Computers and Information in Engineering Conference; American Society of Mechanical Engineers: New York, NY, USA, 2000; Volume 35142, pp. 379-390.

71. Seepersad, C.C.; Pedersen, K.; Emblemsvåg, J.; Bailey, R.; Allen, J.K.; Mistree, F. The validation square: How does one verify and validate a design method. Decision Making in Engineering Design; Lewis, K.E.; Chen, W.; Schmidt, L.C. American Society of Mechanical Engineers Press: New York, NY, USA, 2006; pp. 303-314. ISBN 0791802469

72. HersterDudley, M.R. Building Resilience within DOD Microgrids by Considering Human Factors in Recovery Procedures. Master’s Thesis; Naval Postgraduate School: Monterey, CA, USA, 2021.