1. Introduction
Smart grid systems are intended to optimize the usage of electrical resources. These systems are complex Cyber-Physical Systems (CPS) exposed to various security challenges. An advanced metering infrastructure (AMI) is one of the main communication components of the smart grid that consists of communication networks connecting smart meters and collectors. The AMI collects and processes data from large number of devices and reports the results over communication networks. It can provide advanced services, such as monitoring, alarm, billing, remote home control, intrusion detection, fault tolerance, and software updates [1]. Monitoring the smart grid can contribute to grid stability. The alarm functionality can address alarms for components failures in the grid and/or alarms in the Smart Home. The billing functionality could be for total consumption every hour or max usage. The remote home control functionality can control home devices by interacting with, e.g., the heating system. The intrusion detection functionality can monitor hacking attempts to the home, the control center, and any entity in between.
While providing advanced services, AMIs also introduce several security risks including attacks on confidentiality. The report [2] shows that connected Internet of Things (IoT)/CPS devices will grow at a Compound Annual Growth Rate (CAGR) of 13 percent with 26.9 billion related to the IoT in 2026. The rise of large scale interconnectedness and often outdated design of the devices present a significantly expanded attack surface. Despite the significant efforts in securing IoT/CPS-enabled smart grid systems, many remain vulnerable to various advanced and evolving cyber-attacks [3,4,5]. There are several reasons for this. IoT/CPS-enabled smart grid systems rely upon wireless networks. It makes them vulnerable for eavesdropping. These devices may also be unattended for prolonged periods of time leaving them vulnerable to physical attacks, and most devices limited in energy and computing power do not allow for the implementation and use of complex security schemes [6,7]. A recent survey [8] has shown that False Data Injection (FDI) attacks also threaten state estimation in smart grids.
Furthermore, adaptive attackers will adapt their strategies to the current security situation, and to newly deployed countermeasures. Such emerging attacks can become very sophisticated and can be coordinated [9], persistent [10], and collaborative or cooperative with specialized attack expertise [11]. Examples of such coordinated attacks include data injection attacks concurrently occurring from multiple adversaries [12], large-scale stealthy scans, stuxnet worm outbreaks and Distributed Denial-of-Service (DDoS). Such adaptive, collaborative or coordinated attacks require adaptive, collaborative or coordinated defenses. Therefore, the confidentiality of AMI data must be protected within the AMI system, in transit or at rest, which requires significant collaboration, evolution and adaptation in the security of the AMI. Evolutionary game theory (EGT) lends itself to model the dynamic interplay between the way attackers adapt and evolve their behaviors as evolutionary attacks and the way defenders anticipate the unknown and prevent dynamic, adaptive attacks that evolve over time. Evolutionary game theory studies adaptive rules that govern dynamic behavior. It offers, a solid basis for realistic and intelligent decision making in an uncertain world, describing how individuals make decisions and interact in complex environments in the real world [13].
The components in the AMI need to collaborate to achieve a common goal in collecting, aggregating, transmitting and securing the data. The data confidentiality attack in the AMI is unauthorized access to sensitive information between utilities and users by targeting the AMI components, such as smart meters, data concentrators, communication networks, and a central, or head-end, system. Users’ consumption habits and other relevant information must be protected from access by unauthorized persons or companies. The data sets in the head-end system must only be accessed by authorized systems or users. Confidentiality is, therefore, one of the primary concerns in AMI. As demonstrated in Reference [14], integrity and confidentiality attacks cause monetary effects on the AMI which in turn have cascading effects to other interdependent critical infrastructures, such as health, finance, and telecoms. In this paper, we focus on evolutionary game for confidentiality attacks and defenses for the AMI.
Previously, we have introduced an evolutionary game framework [15] that models evolving attacks and defenses in connection with data integrity for smart grid systems. The novel contributions of this paper are:
- the formulation of the AMI evolutionary game and the derivation of a numerical scheme for this game,
- the simulations of the evolutionary game on realistic AMI cases for confidentiality,
- the identification of constraints, and
- analysis of the confidentiality evolutionary game allowing the defender to explore the space of strategies and to select the optimal set of solutions.
The remainder of the paper is organized as follows. Section 2 and Section 3 give a brief literature survey and present theoretical background for this work, respectively. In Section 4, we introduce our system and game model, formulate the confidentiality game as an evolutionary game using replication dynamics, and discretize the evolutionary game model and derive a numerical scheme for solving the evolutionary dynamics. In Section 5, we carry out simulations on relevant AMI cases for confidentiality and demonstrate how this can be used to inform the AMI security. The simulation results are presented and discussed in Section 6. Finally, we conclude and discuss the future work in Section 7.
2. Related Work The IoT/CPS system brings great benefits to the cyber physical IoT-enabled smart grids by connecting people, processes, services, devices, and data. However, the rise of large scale interconnectedness presents a significantly expanded attack surface. There exist significant efforts to secure the IoT-enabled AMI, which is the core component of the smart grid. In this section, we give a brief review of these efforts. The research that studies game theoretical approaches for modeling the evolving nature of cyber-attacks inside the IoT-enabled AMIs is not addressed sufficiently in the literature. In this section, we present the work on modeling security threats for IoT-enabled smart grids with constrained processing resources. In this work, we use evolutionary game theory. Therefore, this section also discusses literature review regarding applications of evolutionary game theory for IoT-enabled smart grid.
The IoT-enabled AMI is an integrated system of smart meters, collectors, communications networks, and data management systems which support the safe, efficient, and reliable distribution of electricity and advanced functionality to energy customers [16]. Unfortunately, the power grids have been the target of sophisticated cyber-attacks which could lead to grid shutdown, cascading failures, damage to the infrastructure, and potential harm to people [17]. Such targeted attacks could have devastating effects on government, trade, commerce, banking, transportation, and other important aspects, which rely on energy to operate. A compromise of AMI may also result in an invasion of privacy and provide a surface from which to extract information from users, such as Internet activity, financial, or health records [17]. The AMI poses several well known security threats [18,19,20,21].
As the IoT-enabled AMI is the core component of the smart grid, it is thus important to identify the attack surface and protect it from cyber-attacks. The cyber-attack surface of the AMI has been quantified and examined [17]. It is also important to measure the significance of threats and how they can transpire into attacks in the AMI environment. Different categories of attack types and analysis of the various countermeasures against these attacks have been studied [22]. A methodology for assessing security, privacy and dependability in a combined manner in the smart grid has been developed and measureable security in smart grids has been introduced [1]. A controlled Markov-Gaussian process has been suggested to minimize the damage of advanced persistent threats in cyber-physical systems [23]. He and Yan [24] provide a systematic review of the critical cyber-physical attack threats and defense strategies in the smart grid, as well as discuss a wide range of opportunities and challenges in enhancing energy security by maintaining the integrity of smart grid under complex cyber-physical attacks. Ismail et al. [25] presented a noncooperative game for attacks on data confidentiality for smart-grid AMI and studied the strategies of the attacker and the defender at the Nash equilibrium. Applying this model, the authors defined the optimal strategy of the defender and the minimum resources required for defending the assets.
Evolutionary game theory is a branch of game theory. Evolutionary game theory, rooted in classical game theory and the theory of evolution [26], has been effectively studied to model population dynamics in biology and economics domains, but its application to smart grid security has not been fully exploited. Santos et al. [27] argue that, by using a dynamical approach, such as evolutionary game theory, one is able to follow the self-organization process by which a population of individuals coordinates into a given behavior. Hoffman et al. [13] argue that evolutionary dynamics is a powerful tool for specifying changes in strategies over time in a population. Quijano et al. [28] addressed the advantages of evolutionary game theory in the role of population games and evolutionary dynamics in distributed control systems. Ficici et al. [29] present a game-theoretic investigation of selection methods used in evolutionary algorithms. The three main advantages of using EGT in engineering problems and an outstanding advantage of distributed population dynamics compared to distributed learning algorithms are described in Reference [28].
Evolutionary game has been successfully applied to the areas of Advanced Persistent Threats (APTs), evolving interactions between an attacker and a defender, detecting DDoS, and wireless sensor networks. Alabdel Abass et al. [10] studied APTs that represent stealthy, powerful, long-term, and well-funded attacks against cyber systems, such as smart grids, data centers, and cloud storage. The authors capture the long-term continuous behavior of the APTs on the cloud storage devices using evolutionary game. Bouhaddi et al. [30] model the evolving interactions between an attacker and a defender in MANET as an evolutionary game. In this model, each player learns about the behavior of its opponent over time and adjusts its strategy. Vejandla et al. [31] present evolving gaming strategies for attacker-defender in a simulated network environment. Detection of DDoS attacks using an artificial immune system-inspired multi-objective evolutionary algorithm has been investigated in Reference [32]. Evolutionary game theory has also been used for modeling wireless sensor networks [33,34,35,36,37,38]. An overview of evolutionary computation and other computational intelligence technology contributing to meet security challenges can be found in Reference [3].
Although our work is partly motivated by the related work above, there are some distinctions compared with them. The unique characteristics and usage scenarios of IoT-enabled AMI in the smart power grid introduce new security challenges. The increasing share of pervasive IoT devices which lack computing power, security, and privacy in such environments is a challenge—not to mention provisioning of adaptivity to tackle dynamicity and evolution. An accurate and resilient evolutionary game-based adaptive confidentiality assessment on IoT-enabled AMI entities is required. Given the dynamics in the AMI environment, the ability of the AMI nodes to adjust their confidentiality protection in response to their perception of the environment and the systems themselves should be provided. Although there are many research contributions about confidentiality in AMI systems, most of them have not considered these and fall short defining a framework for building dynamic and flexible defense for AMI with population dynamical methods for designing defense mechanisms for robust and reliable AMI cyber systems. Our work is similar to Ismail et al. [25]’s work in that it models the AMI as tree structure and in that it is a similar game model for confidentiality protection. It differs from their noncooperative classical game, which assumes player’s rationality, in that it is a population game with continuous game and no rationality assumption. The proposed solution is adaptive using the replicator dynamics for players to adapt their strategies, addresses the problem of evolutionary attacks, and uses a numerical simulation to solve the replicator dynamics.
Our analysis shows that the presented related work mostly studies scenarios where a single adversary attacks one resource at a time. In reality, multiple attackers can cooperate and launch joint attacks. They can share their knowledge about previous attacks, learn from each other’s experiences, and coordinate future actions by selecting successful strategies. The defenders can also collaborate and share the acquired experience to choose the optimal strategies for their defenses. We recognize that the collaborations between multiple adversaries and multiple defenders have not been fully researched in the previous work. This motivates us to study and to apply evolutionary game theory to these advanced scenarios to explore the space of strategies, as well as to select the optimal set of solutions. It will allow the defender to choose the best possible strategy and to continuously stay ahead of the attacker in defending the security intelligence. 3. Evolutionary Game Theory In this section, we give an overview of evolutionary game theory that is further used in our formulation of the AMI confidentiality game. While classical game theory has been traditionally applied to model attacks on smart grid systems, it is a static approach that computes Nash equilibria and the corresponding utilities for the participating players. The main idea behind classical game theory is how rational individuals are expected to behave in conflict situations.
Formally defined, a game consists of N players and a strategy space S. Each player can select a strategysi∈Si∈S, whereSiis the strategy space of the i’th player. When a player selects a particular strategy, the corresponding payoff depends on this strategy and on decisions made by other players. The utility function is defined asUi:S→R, whereS=S0×S1×⋯×SN.
A Nash equilibrium (NE) is a strategy sets*∈Ssuch that
Ui(si*,s−i*)≥Ui(si,s−i*)∀i,si∈S,
wheres−i∈S−Si, the strategy space excluding player i. In a Nash equilibrium, no single player can increase its utility by unilaterally changing strategy.
Certainly, the limitations, such as rational and static features, do not reflect the way the real world behaves in most situations. Inspired by the theory of evolution, evolutionary game theory [39] was introduced to overcome these limitations.
Evolutionary game theory borrows the notation from classical game theory, like strategy spaces, payoff matrices, and utility functions. Differently from classical game theory with its focus on rational individuals, evolutionary game theory considers populations of players that adopt various strategies and play contest against each other. It studies how successful these populations are in their choices of strategies and how more successful strategies are passed to the next generations. Therefore, it models the dynamics and evolution of populations of players given a distribution of strategies. Generations of population evolve based on the success of individual strategies compared to the success of overall population. This evolution process is governed by two key elements:
- mutation mechanism that is represented by the Evolutionary Stable Strategy (ESS) concept; and
- selection mechanism that is represented by the replicator dynamics.
The ESS concept is considered to be a refinement of NE and it represents an ability to evolve. It outperforms any alternative mutant strategies. In other words, a strategy x is defined as an ESS if, for any other strategy y, some threshold fraction of mutantsϵ¯y∈]0,1[ exists that Equation (2) is satisfied for allϵ∈]0,ϵ¯y[:
U(x,ϵ×y+(1−ϵ)×x)≥U(y,ϵ×y+(1−ϵ)×x).
Thus, the strategy x is defined as evolutionary stable if this inequality holds for any mutant strategy, if the share of mutants in this population is sufficiently small [40]. A group of players choosing ESS will not be replaced by players that choose a different strategy. It is shown [40] that a strong connection between ESS and NE exists. If a strategy x is an ESS, then x is a Nash equilibrium, and, if x is a strict Nash equilibrium, then x is an ESS.
The second important concept is the replicator dynamics [41] that governs evolution of the strategies and is defined as following.
∂xi(t)∂t=(U(xi)−UA(x))×xi(t).
Here,xiis the proportion of strategy i in the populationx=(x1,…,xn).U(xi)is defined as an expected utility of strategy i, andUA(x) is defined as an average population utility. Playing a game, different individuals from a population are able to compare their strategies to the average population result and learn from each others experiences. The replicator dynamics is applied to adjust their strategies. If ESS exists, the evolution dynamics leads to ESS [42].
Further improvement of the replicator equation was suggested in Reference [43]. The authors proposed to add stochastic elements to better address dynamic stability. In this work, we use the replicator equation with stochastic elements.
4. Models and Numerical Scheme Development This section presents the system, threat, and game models for the AMI. Further, it defines the evolutionary game for confidentiality attacks and defenses based on these models. We propose a numerical scheme by discretizing the strategy spaces and deriving a discrete version of the replicator equation. 4.1. System and Threat Model The work considers a network scenario where adversaries attack an AMI network trying to compromise confidentiality and obtain unauthorized access to the information transmitted inside the network. In this scenario, we assume that the adversary has knowledge about the network topology including transmission technology. Further, we assume that the system deploys an intrusion detection system (IDS) to detect malicious behavior.
The configuration of an AMI is modeled as a graph that connects nodes representing individual meters, collectors, and the head-end system. Further, we define a set of nodesN={0,1,2,…,n}that comprise the AMI network. The head-end system (HES) node is defined as the top noden0 . We have two sets of nodes: collectors and meters. Meters collect the data and forward the data to the HES node using collectors as transmitters. Some nodes can perform as both meters and collectors. Due to the hierarchical nature of the information aggregation taking place, we represent the AMI as a tree structure, as shown in Figure 1. The tree is static, meaning that vertices and edges do not change over time. Except for the HES node that does not have a parent, the rest of the nodes have one parent and may have multiple children.
Confidential data is collected by meters, aggregated at meters/collectors, and transmitted by the network to reach the head-end system. The information sent from each node has a quantified value and is the sum of the value of the information gathered at the node and the value of the information collected from the node’s children.
We denotef(i):N→Nas the parent of node i, and the set
Chi=j∈N:f(j)=i
as the children of node i.
As all meters are leaf nodes, the setChiis empty for these nodes.
The AMI nodes can run on different security levels determining the probability of protecting the data before transmitting. Due to its limited computational budget, an AMI node is not capable of protecting all messages. In addition, the AMI uses an IDS to detect possible attacks. Thus, these resources are also taking in consideration.
A set of adversary nodes exists that can connect to and attack the AMI. We assume that the attacker cannot access the cryptographic keys and has no control over the encryption process. To intercept a message generated by the meterni, the adversary node can attack either the meter or any of collectors that forward the message. Attacking a leaf node, or a meter, is less expensive than attacking a collector. However, a successful attack on a collector gives higher payoff. For each node, we define a probability for protecting the messages asti. For each adversary node j, we define a probabilty for attackings an AMI nodeniassi,j. For each node, the costs of defense and attack are given ascidandcia, respectively. The collected messages represent a certain value. For quantifying these values, we present an asset valuevifor each nodeni. We assume that these values are constant over time.
4.2. Game Model
We consider the following confidentiality game that involves two classes of players, attackers, and defenders. The attackers and defenders meet pairwise and play the game. The attacker and defender do not have any knowledge of the opponent’s choices and choose their strategies simultaneously. The game is a discretization of a continuous game and can be considered as a special case of a resource constrained network security game [44].
The attackers choose attack rates (or probabilities)si≥0for attacking the node labeled i. The attacks are assumed to be subject to the budget constraint
∑i=1Nsi≤BS,
where N is the total number of nodes in the tree, andBSis the attacker budget. Hence, the strategy space for the attacker is given by
S={s∈[0,1]N:∑isi≤BS}.
Similarly, the defenders choose defense ratesti≥0for defending node i. The defense rate is the proportion of data that is encrypted before transmitting it to its parent node. In particular, forti=1, the data sent from node i is assumed impossible for the attacker to obtain. The defense rates are assumed subject to the budget constraint,
∑i=1Nti≤BT,
whereBTis the defender budget. Hence, the strategy space for the defender is given by
T={t∈[0,1]N:∑iti≤BT}.
We model an AMI as a tree structure, and we assume that, in order to intercept data sent from node i, the attacker can choose to either attack node i directly or the parent nodef(i). We consider that the cost of attacking and encrypting data on node i are proportional to the value of the data.
The utility functionUA:S×T→Rfor the attacking player is given by
UA(s,t)=∑i=1N(vi(si+sf(i))(1−ti)−si CA,i)=∑i=1N(vi si(1−ti)−si CA,i)+∑i=1N∑j∈Chivjsi(1−tj),
whereCA,iis the cost of attacking node i, andviis the value of the information collected at node i. Note that a factor appearing in the work of Ismail et al.,(1−a)where a is the detection rate, is omitted here for simplicity. It is a scaling of the node values and has no quantitative influence on the model.
Similarly, the utility functionUD:T×S→Rfor the defending player is
UD(t,s)=−∑i=1N(vi(si+sf(i))(1−ti)+ti CD,i=−∑i=1N(visi(1−ti)+ti CD,i)−∑i=1N∑j∈Chivjsi(1−tj),
whereCD,iis the cost of defending node i.
4.3. Evolutionary Game Formulation
In this section, we proceed with the main novel contribution of this work, the application of the model [25] in an evolutionary game. To this end, we assume that there are populations of attackers and defenders, represented by probability measuresPs(S)andPt(T), respectively. The measures represent the distribution of the overall population over the attacker and defender strategy space. Different strategies of population evolution exist, and, for the current work, we use a replicator equation.
The replicator equation favors the choices of strategies which perform well (in terms of utility) relative to the overall population. Let
πA(s,Pt)=∫T UA(s,t′)Pt(dt′)
be the expected payoff for an attacker given a defender populationPt. We can formulate a similar payoff for the defending population, denoted D. For the defenders, we have the expected payoff
πD(t,Ps)=∫S UD(t,s′)Ps(ds′),
given an attacker populationPs . Note that Equations (11) and (12) form a couple set of equations defining the dynamics between the attacking and defending populations.
Equations (11) and (12) can be integrated over the attacking and defending populations, respectively, to yield the average payoff. Given the attacker and defender populationsPsandPtwe obtain
πA(Ps,Pt)=∫S πA(s′,Pt)Ps(ds′)=∫S ∫T UA(s′,t′)Pt(dt′)Ps(ds′).
Similarly, the average payoff for a defender given the same populations is given by
πD(Pt,Ps)=∫T πD(t′,Ps)Pt(dt′)=∫T ∫S UD(t′,s′)Ps(ds′)Pt(dt′).
We can combine Equations (11)–(14) with the evolutionary replicator dynamic with noise [43] to model the evolution of any set of strategies. For any subset of attacking strategiesS¯⊂S , the evolutionary replicator dynamic with noise [43] takes the form of
dPsdτ(S¯)=∫S¯(πA(s′,Pt)−πA(Ps,Pt))Ps(ds′)+δS(S¯),
for some time scaleτand a stochastic termδs(S¯). Similarly, the evolution of a subset of defending strategiesT¯⊂Tis given by
dPtdτ(T¯)=∫T¯(πD(t′,Ps)−πD(Pt,Ps))Pt(dt′)+δT(T¯).
The Equations (15) and (16) fully govern the evolution of the attacker and defender populations. The noise termsδS(S¯)andδT(T¯)introduce random fluctuations in the evolution of the attacker and defender populations, respectively.
4.4. Numerical Scheme We carry out numerical experiments in order to provide insight into the evolutionary dynamics of the confidentiality game and to demonstrate how it can be used in informing AMI security. To this end, we discretize the strategy spaces and derive a simple but computationally effective numerical scheme for the replicator equation.
To solve (15) and (16) numerically, we discretize the N-dimensional strategy spaces S and T. An attacking strategy is represented by
sk=k1K,⋯,kNK,ki∈{0,⋯,K}.
The i’th element of the vectorskrepresents the degree (or severity) of attack aimed at node i. We impose a budget constraint
1K∑i=1Nki≤1
to ensure that each attacker only has a finite amount of resources, i.e., cannot fully attack every node at once. Similarly, a defending strategy is represented discretely as
tk=k1K,⋯,kNK,ki∈{0,⋯,K}.
Similar to the attackers, the defenders also have a budget constraint
1K∑i=1Nki≤Y
imposed on their valid strategy space. This constrain ensures that the defenders cannot fully defend every node in the network and need, therefore, to select the strategy that provide the highest possible payoff.
It is convenient to introduce the discretized strategy space, i.e., the list of all possible strategies that also adheres to the budget constraint. We can denote the discretized attacker strategy space as
ΩSK=k=(k1,⋯,kN):ki∈{1,⋯,K},1K∑i=1Nki≤BS,
and, similarly, the discretized defender strategy space as
ΩTK=k=(k1,⋯,kN):ki∈{1,⋯,K},1K∑i=1Nki≤BT.
In the discrete setting the population of attackers and defenders can be represented as a probability distribution over the discrete strategy spaces. In particular, the population of attackers is represented by the probability distribution
Ps(sk)∈[0,1],∑k∈ΩSKPs(sk)=1,
and the defender population is represented as
Pt(tk)∈[0,1],∑k∈ΩTKPt(tk)=1.
Using this representation, the expected payoff for an attacker and defender can be written in the form
πA(s,Pt)=∑k∈ΩTKUAs,tkPttk
and
πD(t,Ps)=∑k∈ΩSKUDt,skPs(sk),
respectively. Similarly, the average payoff for the attacker and defender populations can then be written as
σA(Ps,Pt)=∑k∈ΩSKπA(sk,Pt)Ps(sk)
and
σD(Pt,Ps)=∑k∈ΩTKπD(tk,Pst)Pt(tk),
respectively.
Finally, the discrete versions of the population replicator equations take the form
dPsdτ(sk)=(πA(sk,Pt)−σA(Ps,Pt))Ps(sk)+δsk
for the attackers and
dPtdτ(tk)=(πD(tk,Ps)−σD(Pt,Ps))Pt(tk)+δtk
for the defenders.
Given initial attacker and defender populationsPs(0)andPt(0), respectively, the i’th generation of attackers and defenders,Ps(i)andPt(i), are computed iteratively according to the scheme
Ps(i)(sk)=Ps(i−1)(sk)+Δτ(πA(sk,Pt(i−1))−σA(Ps(i−1),Pt(i−1)))Ps(i−1)(sk)+ΔτδSk,
Pt(i)(tk)=Pt(i−1)(tk)+Δτ(πD(tk,Ps(i−1))−σD(Pt(i−1),Ps(i−1)))Pt(i−1)(sk)+ΔτδTk,
whereΔτis the time step length between each generation. For numerical stability, positivity ofPs(i)andPt(i)is enforced after each step, and the populations are re-normalized.
5. AMI Case Study 5.1. Simulation Setup
We consider a case study with a small but realistic AMI structure consisting of 15 nodes with edges shown in Figure 2 and node parameters given in Table 1. All leaf nodes represent meters. The intermediate nodes between the head-end system and the leaf nodes act either as a pure collector aggregating data from its child nodes, or as a hybrid collector/meter. We assume that the value of the information aggregated at each node the sum of any information generated at the node (if it is a meter) and the information aggregated from child nodes. The cost weights of attacking and protecting data on a node i are set to 0.2 and 0.05, respectively. These weights are based upon the original work of Ismail et al. [25]. We used a simular network configuration. However, we removed some leaf nodes in order to limit the strategy space. Further, we investigated several options regarding the defender’s budget.
The attacker’s budget is set to1.0and is the same value for all simulations. Regarding the defender’s budget, we vary the budget as1.0,2.0, and3.0. The simulation results are presented separately for different values of the defender’s budget.
5.2. Evaluation Metrics
To better interpret the results from the evolutionary game and the evolution of attacker and defender strategies over time, we consider the following generation-dependent game metrics. The population-averaged attack-rate for node i is given by
Ai=∑k∈ΩsKsik Ps(sk).
Similarly, the population-average defense-rate for node i is given by:
Di=∑k∈ΩtKtik Pt(tk).
To assess the success of attackers and defenders in the current population, the time-evolution of the average payoff (27) and (28) are also monitored.
6. Results
The simulation results that show the evolution of average payoff for both defenders and attackers are depicted in Figure 3.
The attack and defense rates for the different nodes are depicted in Figure 4, Figure 5, Figure 6, Figure 7, Figure 8 and Figure 9, respectively, for the defender’s budget equal1.0,2.0, and3.0. At the start of the game, the attack and defense resources are equally destributed between the participating nodes. Initially, there is a transient phase where the attackers alternate between attacking the Head-End System and the intermediate collector nodes. The defender population responds by increasing the defense rate for the Head-End System to the point where the attackers, on average, give up this node and instead focus on the intermediate Collectors and Collector/Meters.
The results for the attack and defense rates show that both types of players favor nodes from a higher aggregation level, which increase their utilities. We also observe that when the defender’s budget increases the system distributes the new resources to the nodes that contribute more to the defender’s payoffs. On one hand, the leaf nodes are less valuable and, thus, less effort is exerted to protect these nodes. On the other hand, the attacker uses also less resource to attack the leafs since the attacker better benefit by attacking nodes from a higher aggregation level. For the same aggregation level, both players assign more resources to the nodes with higher aggregated value. The simulation graphs show that more effort was made to protect the node 3, which has higher resource value and also accommodates more children than the node 2. It is worth noting that the resources assigned to defend the nodes are not linearly proportional to their respective values and that this proportion also changes depending on the amount of the total budget assigned to the attacker. Further, we can see that the system finds a short unstable equilibrium state that happens around generation 70 for 1 defender, generation 60 for 2 defenders, and generation 50 for 3 defenders. For different values of the defender’s budget, both graphs start to converge to a stable state after approximately 80–90 generations and remain stable for more than 100 generations. For the defender, it means that the system has defined the solution that gives the best response to the adaptive attacks in the dynamic environment. The system, therefore, provides the adaptivity by using the replicator dynamics. We can also conclude that the ESS is reached for this system setup. 7. Conclusions In this work, we modeled confidentiality attacks and defenses as an evolutionary game and analyzed the behaviors of the attacker and the defender of the AMI system. By applying evolutionary game theory to this problem, we introduced an important dynamic and learning capabilities in the behavior of both attackers and AMI nodes, to explore the space of strategies, and to select the optimal set of solutions. We used the replicator equation to show the evolution of utilities for both type of players. Further, we outlined how the evolutionary game model can be used to evaluate the security threats in the AMI systems. In our simulation scenarios, we show that the solution converges to ESS for all investigated cases. The simulations also show that the behavior of the replicator dynamic depends not only on incentives but also on the network configuration and proportions of the protected assets. It is important that the results of this work give us the best possible defense strategy against evolving attacks. It allows the defender to to continuously stay ahead of the attacker in defending the AMI nodes. The next step in our research will be the investigation of more sophisticated scenarios, including dynamic AMI trees. We consider that these trees can have a dynamic configuration and change over time. A node can disconnect, connect to a new one, or a new node can be introduced. This condition introduces more variety and dynamism to the AMI system. This problem should be taken into account for preparing the strategy spaces. Applying the developed solution to real networks for further field testing and validation will be part of the roadmap for implementing this solution as an industrial toolset. Combining the evolutionary game analysis with machine learning algorithms, especially with federated learning and autonomy, is a step forward to overcome this limitation and the scaling problem. In our future work, we intend to evaluate and implement a combination of applying machine learning and evolutionary game theory for modeling adaptive attack-defense dynamics. It will also include a development of suitable quantitative metrics to evaluate game simulations.
Enlarge this image.Figure 1. Illustration of the components in the Advanced Metering Infrastructure.
Enlarge this image.Figure 2. The Advanced Metering Infrastructure (AMI) tree structure used in the case study. It contains the head-end system (HES). The intermediate nodes are either pure Collectors (C) or hybrid Meter/Collectors (M/C). The leaf nodes are meters.
Enlarge this image.Figure 3. Evolution graphs for average utility for the attacker and defender populations for the case study. The results are shown for the defender's budget equals to 1.0, 2.0, and 3.0. The x-axis presents the generations. The y-axis presents the evolution of average utilities.
Enlarge this image.Figure 4. Evolution graphs for average attack rate showing the results for the nodes of the case study. The defender's budget is equals to 1.0. The x-axis presents the generations. The y-axis presents the evolution of average rates.
Enlarge this image.Figure 5. Evolution graphs for average defense rate showing the results for the nodes of the case study. The defender's budget equals to 1.0. The x-axis presents the generations. The y-axis presents the evolution of average rates.
Enlarge this image.Figure 6. Evolution graphs for average attack for the nodes of the case study. The defender's budget equals to 2.0. The x-axis presents the generations. The y-axis presents the evolution of average rates.
Enlarge this image.Figure 7. Evolution graphs for average defense rate for the nodes of the case study. The defender's budget equals to 2.0. The x-axis presents the generations. The y-axis presents the evolution of average rates.
Enlarge this image.Figure 8. Evolution graphs for average attack for the nodes of the case study. The defender's budget equals to 3.0. The x-axis presents the generations. The y-axis presents the evolution of average rates.
Enlarge this image.Figure 9. Evolution graphs for average defense rate for the nodes of the case study. The defender's budget equals to 3.0. The x-axis presents the generations. The y-axis presents the evolution of average rates.
| Node | vi | CA,i | CD,i | s1* | t1* | s2* | t2* | s3* | t3* |
|---|---|---|---|---|---|---|---|---|---|
| #1 | 33.00 | 6.60 | 1.65 | 0.396246 | 0.51805 | 0.373106 | 0.524613 | 0.364412 | 0.536942 |
| #2 | 15.00 | 3.00 | 0.75 | 0.072595 | 0.04662 | 0.09603 | 0.107789 | 0.089545 | 0.511544 |
| #3 | 18.00 | 3.60 | 0.90 | 0.060055 | 0.05209 | 0.054538 | 0.48788 | 0.060831 | 0.515485 |
| #4 | 3.00 | 0.60 | 0.15 | 0.042049 | 0.03001 | 0.043214 | 0.074903 | 0.045137 | 0.12042 |
| #5 | 12.00 | 2.40 | 0.60 | 0.046897 | 0.029263 | 0.049434 | 0.07387 | 0.053685 | 0.119965 |
| #6 | 9.00 | 1.80 | 0.45 | 0.044304 | 0.028329 | 0.046061 | 0.071233 | 0.049054 | 0.119792 |
| #7 | 9.00 | 1.80 | 0.45 | 0.041939 | 0.018402 | 0.043156 | 0.065708 | 0.045231 | 0.119011 |
| #8 | 3.00 | 0.60 | 0.15 | 0.026952 | 0.04108 | 0.026954 | 0.077835 | 0.026961 | 0.121616 |
| #9 | 3.00 | 0.60 | 0.15 | 0.026952 | 0.04108 | 0.026954 | 0.077835 | 0.026961 | 0.121616 |
| #10 | 3.00 | 0.60 | 0.15 | 0.026952 | 0.04108 | 0.026954 | 0.077835 | 0.026961 | 0.121616 |
| #11 | 3.00 | 0.60 | 0.15 | 0.037328 | 0.035544 | 0.036896 | 0.074586 | 0.036151 | 0.119706 |
| #12 | 3.00 | 0.60 | 0.15 | 0.037328 | 0.035544 | 0.036896 | 0.074586 | 0.036151 | 0.119706 |
| #13 | 3.00 | 0.60 | 0.15 | 0.046802 | 0.027631 | 0.046602 | 0.070442 | 0.046307 | 0.117527 |
| #14 | 3.00 | 0.60 | 0.15 | 0.046802 | 0.027631 | 0.046602 | 0.070442 | 0.046307 | 0.117527 |
| #15 | 3.00 | 0.60 | 0.15 | 0.046802 | 0.027631 | 0.046602 | 0.070442 | 0.046307 | 0.117527 |
Author Contributions
All authors contributed to this paper as joint work in the context of the IoTSec project. H.A. conceived the research idea and the concept design; he also made the main contribution to the related work and scientific background study. P.A. and S.B. contributed to the scientific background, developed and programmed the evolutionary game model, conducted game analysis and performed the case study. The paper was written by S.B. with text contributions by all co-authors. All authors have read and agreed to the published version of the manuscript.
Funding
This research has been carried out in the context of the project IoTSec-Security in IoT for Smart Grids funded by the Research Council of Norway in the IKTPLUSS programme, grant number 248113.
Acknowledgments
The work presented here has been carried out in two research projects: The IoTSec (Security in IoT for Smart Grids, Nr. 248113/O70, 2015-2018), and FINSEC (Integrated Framework for Predictive and Collaborative Security of Financial Infrastructures) project funded by the EU under the Horizon 2020 programme (contract number: 786727). The authors wish to thank Wolfgang Leister for the advice during the preparation of this paper. The authors would also like to thank Ziad Ismail and Makhlouf Hadji for reading the draft of this paper and for their helpful comments that improved the paper.
Conflicts of Interest
The authors declare no conflict of interest.
Abbreviations
The following abbreviations are used in this manuscript:
| AMI | Advanced Metering Infrastructure |
| CAGR | Compound Annual Growth Rate |
| CPS | Cyber-Physical Systems |
| IoT | Internet of Things |
| FDI | False Data Injection |
| DDoS | Distributed Denial-of-Service |
| EGT | Evolutionary game theory |
| ESS | Evolutionary Stable Strategy |
| APTs | Advanced Persistent Threats |
| NE | Nash equilibrium |
| HES | head-end system |
| C | collector |
| M | meter |
1. Noll, J.; Garitano, I.; Fayyad, S.; Asberg, E.; Abie, H. Measurable Security, Privacy and Dependability in Smart Grids. J. Cyber Secur. Mobil. 2014, 3, 371-398.
2. Ericson, A.B. Ericsson Mobility Report 2020. Available online: https://www.ericsson.com/4adc87/assets/local/mobility-report/documents/2020/november-2020-ericsson-mobility-report.pdf (accessed on 11 December 2020).
3. Sadeghi, A.R.; Wachsmann, C.; Waidner, M. Security and Privacy Challenges in Industrial Internet of Things. In Proceedings of the 52nd Annual Design Automation Conference, San Francisco, CA, USA, 8-12 June 2015; ACM: New York, NY, USA, 2015; pp. 54:1-54:6.
4. He, H.; Maple, C.; Watson, T.; Tiwari, A.; Mehnen, J.; Jin, Y.; Gabrys, B. The security challenges in the IoT enabled cyber-physical systems and opportunities for evolutionary computing amp; other computational intelligence. In Proceedings of the 2016 IEEE Congress on Evolutionary Computation (CEC), Vancouver, BC, Canada, 24-29 July 2016; pp. 1015-1021.
5. Mavroeidakos, T.; Chaldeakis, V. Threat Landscape of Next Generation IoT-Enabled Smart Grids. In Artificial Intelligence Applications and Innovations, Proceedings of the AIAI 2020 IFIP WG 12.5 International Workshops, Neos Marmaras, Greece, 5-7 June 2020; Maglogiannis, I., Iliadis, L., Pimenidis, E., Eds.; Springer International Publishing: Cham, Switzerland, 2020; pp. 116-127.
6. Abie, H.; Balasingham, I. Risk-based Adaptive Security for Smart IoT in eHealth. In Proceedings of the 7th International Conference on Body Area Networks, Oslo, Norway, 24-26 February 2012; pp. 269-275.
7. Zeitz, K.; Cantrell, M.; Marchany, R.; Tront, J. Designing a Micro-Moving Target IPv6 Defense for the Internet of Things. In Proceedings of the Second International Conference on Internet-of-Things Design and Implementation, Pittsburgh, PA, USA, 18-21 April 2017; ACM: New York, NY, USA, 2017; pp. 179-184.
8. Deng, R.; Xiao, G.; Lu, R.; Liang, H.; Vasilakos, A.V. False Data Injection on State Estimation in Power Systems Attacks, Impacts, and Defense: A Survey. IEEE Trans. Ind. Inform. 2017, 13, 411-423.
9. Xiang, Y.; Wang, L.; Liu, N. Coordinated attacks on electric power systems in a cyber-physical environment. Electr. Power Syst. Res. 2017, 149, 156-168.
10. Alabdel Abass, A.A.; Xiao, L.; Mandayam, N.B.; Gajic, Z. Evolutionary Game Theoretic Analysis of Advanced Persistent Threats Against Cloud Storage. IEEE Access 2017, 5, 8482-8491.
11. Xu, S. Collaborative Attack vs. Collaborative Defense. In Collaborative Computing: Networking, Applications and Worksharing; Bertino, E., Joshi, J.B.D., Eds.; Springer: Berlin/Heidelberg, Germany, 2009; pp. 217-228.
12. Sanjab, A.; Saad, W. Data Injection Attacks on Smart Grids with Multiple Adversaries: A Game-Theoretic Perspective. IEEE Trans. Smart Grid 2016, 7, 2038-2049.
13. Hoffman, M.; Suetens, S.; Gneezy, U.; Nowak, M.A. An experimental investigation of evolutionary dynamics in the Rock-Paper-Scissors game. Sci. Rep. 2015, 5, 8817.
14. Tellbach, D.; Li, Y.F. Cyber-Attacks on Smart Meters in Household Nanogrid: Modeling, Simulation and Analysis. Energies 2018, 11, 316.
15. Boudko, S.; Abie, H. An Evolutionary Game for Integrity Attacks and defenses for Advanced Metering Infrastructure. In Proceedings of the 12th European Conference on Software Architecture: Companion Proceedings, Madrid, Spain, 24-28 September 2018.
16. Hansen, A.; Staggs, J.; Shenoi, S. Security Analysis of an Advanced Metering Infrastructure. Int. J. Crit. Infrastruct. Prot. 2017, 18, 3-19.
17. Foreman, J.C.; Gurugubelli, D. Cyber Attack Surface Analysis of Advanced Metering Infrastructure. arXiv 2016, arXiv:1607.04811.
18. Cleveland, F.M. Cyber security issues for Advanced Metering Infrasttructure (AMI). In Proceedings of the 2008 IEEE Power and Energy Society General Meeting-Conversion and Delivery of Electrical Energy in the 21st Century, Pittsburgh, PA, USA, 20-24 July 2008; pp. 1-5.
19. Goel, S.; Hong, Y. Security Challenges in Smart Grid Implementation. In Smart Grid Security; Springer: London, UK, 2015; pp. 1-39.
20. Li, F.; Luo, B.; Liu, P. Secure and Privacy-Preserving Information Aggregation for Smart Grids. Int. J. Secur. Netw. 2011, 6, 28-39.
21. Li, H.; Gong, S.; Lai, L.; Han, Z.; Qiu, R.C.; Yang, D. Efficient and Secure Wireless Communications for Advanced Metering Infrastructure in Smart Grids. IEEE Trans. Smart Grid 2012, 3, 1540-1551.
22. Baig, Z.A.; Amoudi, A.R. An Analysis of Smart Grid Attacks and Countermeasures. J. Commun. 2013, 8, 473-479.
23. Sayin, M.O.; Başar, T. Secure Sensor Design for Cyber-Physical Systems Against Advanced Persistent Threats. In Proceedings of the Decision and Game Theory for Security: 8th International Conference, GameSec 2017, Vienna, Austria, 23-25 October 2017; Rass, S., An, B., Kiekintveld, C., Fang, F., Schauer, S., Eds.; Springer International Publishing: Cham, Switzerland, 2017; pp. 91-111.
24. He, H.; Yan, J. Cyber-physical attacks and defenses in the smart grid: A survey. IET Cyber Phys. Syst. Theory Appl. 2016, 1, 13-27.
25. Ismail, Z.; Leneutre, J.; Bateman, D.; Chen, L. A game theoretical analysis of data confidentiality attacks on smart-grid AMI. IEEE J. Sel. Areas Commun. 2014, 32, 1486-1499.
26. Wang, Y.; Chen, X.; Wang, Z. Testability of evolutionary game dynamics based on experimental economics data. Physica A 2017, 486, 455-464.
27. Santos, F.; Encarnação, S.; Santos, F.C.; Portugali, J.; Pacheco, J.M. An Evolutionary Game Theoretic Approach to Multi-Sector Coordination and Self-Organization. Entropy 2016, 18, 152.
28. Quijano, N.; Ocampo-Martinez, C.; Barreiro-Gomez, J.; Obando, G.; Pantoja, A.; Mojica-Nava, E. The Role of Population Games and Evolutionary Dynamics in Distributed Control Systems: The Advantages of Evolutionary Game Theory. IEEE Control Syst. 2017, 37, 70-97.
29. Ficici, S.G.; Melnik, O.; Pollack, J.B. A game-theoretic investigation of selection methods used in evolutionary algorithms. In Proceedings of the 2000 Congress on Evolutionary Computation, CEC00 (Cat. No.00TH8512), La Jolla, CA, USA, 16-19 July 2000; Volume 2, pp. 880-887.
30. Bouhaddi, M.; Adi, K.; Radjef, M.S. Evolutionary Game-Based Defense Mechanism in the MANETs. In Proceedings of the 9th International Conference on Security of Information and Networks, New York, NY, USA, 20-22 July 2016; pp. 88-95.
31. Vejandla, P.; Dasgupta, D.; Kaushal, A.; Nino, F. Evolving Gaming Strategies for Attacker-Defender in a Simulated Network Environment. In Proceedings of the 2010 IEEE Second International Conference on Social Computing, Minneapolis, MN, USA, 20-22 August 2010; IEEE Computer Society: Washington, DC, USA, 2010; pp. 889-896.
32. Akyazı, U.; Uyar, A.Ş. Detection of DDoS Attacks via an Artificial Immune System-Inspired Multiobjective Evolutionary Algorithm. In Applications of Evolutionary Computation; Di Chio, C., Brabazon, A., Di Caro, G.A., Ebner, M., Farooq, M., Fink, A., Grahl, J., Greenfield, G., Machado, P., O'Neill, M., et al., Eds.; Springer: Berlin/Heidelberg, Germany, 2010; pp. 1-10.
33. Essaddi, N.; Hamdi, M.; Habib, S.; Boudriga, N. Evolutionary strategies for non-uniform deployment in wireless sensor networks. Int. J. Commun. Netw. Distrib. Syst. 2011, 7, 331-354.
34. Jiang, C.; Chen, Y.; Liu, K.J.R. Distributed Adaptive Networks: A Graphical Evolutionary Game-Theoret View. arXiv 2012, arXiv:1212.1245.
35. John, D.J.; Smith, R.W.; Turkett, W.H.; Cañas, D.A.; Fulp, E.W. Evolutionary Based Moving Target Cyber Defense. In Proceedings of the Companion Publication of the 2014 Annual Conference on Genetic and Evolutionary Computation, Vancouver, BC, Canada, 12-16 July 2014; pp. 1261-1268.
36. Li, Y.; Xu, H.; Cao, Q.; Li, Z.; Shen, S. Evolutionary Game-Based Trust Strategy Adjustment among Nodes in Wireless Sensor Networks. Int. J. Distrib. Sens. Netw. 2015, 11, 818903.
37. Shivshankar, S.; Jamalipour, A. An Evolutionary Game Theory-Based Approach to Cooperation in VANETs Under Different Network Conditions. IEEE Trans. Veh. Technol. 2015, 64, 2015-2022.
38. Wang, X.; Wu, Y.; Ren, Y.; Feng, R.; Yu, N.; Wan, J. An Evolutionary Game-Based Trust Cooperative Stimulation Model for Large Scale MANETs. Int. J. Distrib. Sens. Netw. 2013, 9, 245017.
39. Smith, J.M. Game theory and the evolution of fighting. In On Evolution; Edinburgh University Press: Edinburgh, UK, 1972; pp. 8-28.
40. Smith, J. Evolution and the Theory of Games; Cambridge University Press: Cambridge, UK, 1982.
41. Taylor, P.D.; Jonker, L.B. Evolutionary stable strategies and game dynamics. Math. Biosci. 1978, 40, 145-156.
42. Weibull, J.W. Evolutionary Game Theory; MIT Press: Cambridge, MA, USA, 1995.
43. Foster, D.P.; Young, P. Stochastic Evolutionary Game Dynamics. Theor. Popul. Biol. 1990, 38, 219-232.
44. Ismail, Z.; Kiennert, C.; Leneutre, J.; Chen, L. A Game Theoretical Model for Optimal Distribution of Network Security Resources. In Proceedings of the International Conference on Decision and Game Theory for Security, Vienna, Austria, 23-25 October 2017; pp. 234-255.
Svetlana Boudko
*,
Peder Aursand
and
Habtamu Abie
Norwegian Computing Center, P.O. Box 114, Blindern, 0314 Oslo, Norway
*Author to whom correspondence should be addressed.
© 2020. This work is licensed under http://creativecommons.org/licenses/by/3.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.