Scalable cyber‐physical testbed for cybersecurity evaluation of synchrophasors in power systems

Abstract

This paper presents a synchrophasor‐based real‐time cyber‐physical power system testbed with a novel security evaluation tool, pySynphasor, that can emulate different real attack scenarios on the phasor measurement unit (PMU). The testbed focuses on real‐time cyber‐security emulation using different components, including a real‐time digital simulator, virtual machines (VM), a communication network emulator, and a packet manipulation tool. The script‐based VM deployment and software‐defined network emulation facilitate a highly scalable cyber‐physical testbed, which enables emulations of a real power system under different attack scenarios such as address resolution protocol (ARP) poisoning attack, man‐in‐the‐middle (MITM) attack, false data injection attack (FDIA), and eavesdropping attack. An open‐source pySynphasor module has been implemented to analyse the security vulnerabilities of the IEEE C37.118.2 protocol. The paper also presents an interactive framework for injecting false data into a realistic system utilising the pySynphasor module, which can dissect and reconstruct the C37.118.2 packets. Therefore, it expands the potential of testing and developing PMU‐based systems and analysing their security vulnerabilities, benefiting the power industry and academia. A case study demonstrating the FDIA attack on the PMU measurements and the bad‐data detection technique is presented as an example of the testbed capability.

INTRODUCTION

Synchrophasor technology is a revolutionary advancement in power systems, enabling the collection of real-time phasor measurements with GPS timestamps. This technology comprises a phasor measurement unit (PMU), a phasor data concentrator (PDC), a communication network, and a control centre [1]. By utilising synchrophasor technology, state estimation, a mathematical tool introduced by Fred Schweppe in 1968 [2], can compute the current states of a power network based on redundant and noisy measurements. AC, DC, and linear state estimation are variants of static-state estimation. Linear state estimation reduces the computation complexity of AC state estimation by utilising synchrophasor technology. PMU built for transmission system are unsuitable for distribution network as it has different features including radial topology, high resistance to reactance ratio, and unbalances among phases [3]. Therefore, $$\mu $$ PMU was developed to address these issues. It is a high precision synchrophasor device with comparatively low cost that works well in distribution system [4].

Modern power systems use a various communication protocols depending on the specific requirements and standards of the industry. Some commonly used protocols include IEEE C37.118 for synchrophasor technology, IEC 61850 for substation automation, DNP3 (distributed network protocol) for SCADA (supervisory control and data acquisition) systems, and Modbus for device-level communication. Making these protocols works flawlessly rather than focussing on security was the only concern at the time of designing these protocols. As a result, many vulnerabilities [5] of these standard protocols have been enlisted and documented up to date. Moreover, the technology lifetime for cyber-physical power systems is around 15–20 years, whereas conventional IT infrastructure changes within 3–5 years [6]. For this reason, the power system communication infrastructure is more vulnerable to cyber-attack, and it is responsible for 20% of the reported breach in the power system in 2016, according to the literature [7].

Many types of cyber-attacks in power systems are listed in the paper [8], that is, command manipulation, code manipulation, malware injection, GPS spoofing, false data injection, denial of service, fuzzing, rouge node, and channel jamming. Still, there might be 0 day attacks that are unlikely to detect in honeypot or lab-experiment [9]. A denial-of-service attack (DoS) degrades the dynamic performance of the power system [10, 11]. The more extreme version of the DOS attack is the distributed denial of service (DDoS) attack, which is performed from different geolocation of the world utilising the botnet [12]. The DDoS attack makes it impossible for intrusion detection software to identify. The paper [13] presented a comprehensive review of different types of attacks and cyber vulnerabilities in the different vendor SCADA systems. It also presented 163 publicly enlisted vulnerabilities with CVE (Common Vulnerabilities and Exposures) records due to different types of buffer overflows.

FDIA attack is one of the major attacks on the modern power systems where adversary craftily injects malicious data into the system to compromise a part of the system or whole network. Liu et al. demonstrated in the paper [14] how FDIA attack could mislead state estimation and endanger grid operation. They also demonstrated different scenarios of FDIA attack in metre measurements. 2015 Ukraine blackout is the real implication of FDIA attack on large scale power system [15]. The attacker used BlackEnergy Version 3 malware to access the system and injected false command, which tripped seven 110 and 233 kV substation breakers, resulting in a blackout of more than 225 k people for 6+ hours [1]. European network of transmission system operators for electricity (ENTSO-E), which represents 42 transmission system operators (TSOs) across 35 states was hacked in early [16]. The attacker had the potential to jeopardise 42 TSOs. Another incident that made news in early 2021 was the attack on the Colonial Pipeline Co. The hacker got access to the Colonial Pipeline Co. Network and demanded cryptocurrency as ransom, forcing it to shut down the gas supplies to the East Coast [17].

The computer worm was first observed in 1988, known as Morris worm, which literally took down at least one computer in 20 of the whole Internet [18]. The Stuxnet is a notorious computer worm that was first uncovered in June 2010 by a security firm in Belarus [19] and caused substantial damage to the nuclear programme of Iran [20]. These incidents serve as stark reminder of the potential chaos for any jeopardy in the energy sector due to the single point of failure. The paper [21] also enlisted many cyber-attacks in power system, for example, 2003 Ohio, USA Slammer Worm penetration of nuclear power plant control system, 2007 Idaho National Laboratory, USA Aurora attack manipulation of circuit breaker of diesel generator etc.

Many widely used major communication protocols have vulnerabilities. Few vulnerabilities of the IEEE C37.118 protocol have been enlisted in the paper [22]: Authentication attack, establishing PMU and PDC communication without any authentication; MITM attack, involves hijacking session, altering, dropping, and injecting C37.118.2 packets; Reply attack, involves recording packet and reply it multiple times to hide the real scenario; DoS attack, involves overwhelming target by high-speed bulk packets which result in communication loss of PMU, PDC, and control centre. GPS spoofing attack on the PMU device has been presented in the paper [23]. Unlike military GPS, the civilian GPS signal can be predicted by a low-cost GPS receiver, so it is easy to forge the matched version of the corresponding GPS signal by the attacker. Finally, data tampering attack on C37.118.2 protocol has been presented in the papers [24, 25].

The testing of new applications must be done in environments that can characterise both the physical system and the cyber network due to the interconnected nature of the CPPS. Doing research on real-world context is a challenging and verifying security performance is costly [26]. Hence, cyber-physical testbed is an excellent alternative to collecting realistic data [27]. Therefore, testbeds are used to run strict, realistic, repeatable, and reproducible tests to verify new controls and applications and, most importantly, to find out security vulnerabilities [26]. It is vital to understand how an attacker crafts different tools and vulnerabilities to perform a successful attack. Honeypot is one way to study the attacks. It is used to analyse and learn new malware and generate anti-virus signatures. A low-interaction honeypot for detecting unauthorized traffic for the distribution system has been presented in the paper [28]. The problem with the low-interaction honeypot is that it emulates a small number of Internet protocols and services. That is why, there is tension between scalability and fidelity in a honeypot-based intrusion detection system. Machine learning-based anomaly detection is getting more popular day-by-day. So, it is necessary to mimic the real attack scenario in the power system to generate the synthetic dataset for the machine learning model. For this reason, high fidelity testbed plays a vital role [29].

The author in the paper [30] designed a testbed in a co-simulation environment for DER focussing on power system performance, security trade-offs, network segmentation, and encryption. The author in the paper [31] investigated on DNP3 for the case of MITM attack in emulation environments. They utilised Scapy extension of DNP3 protocol presented in the paper [32] to design the attack. The testbed paper [33] demonstrated how a hacker could develop a custom-made tool for performing a stealthy MITM attack against a synchrophasor device. Researchers in the paper [34] developed a modular architecture utilising a software-defined network, virtual machine, and pyPMU [35] to monitor data acquisition and closed-loop control in a wide area network. A hardware-based testbed was presented in [36] to develop an intrusion detection system. The testbed was modelled using a real-time digital simulator (RTDS), relays, PMU, PDC, and PC running a Snort intrusion detection system. In the paper [33], the authors developed a testbed scenario for attacking synchrophasor communication. They used Scapy to implement the MITM attack and a custom Python script to pack and unpack the IEEE C37.118.2 packets. They have also developed the testbed based on a microgrid scenario with a single PMU device and implemented a management server in Raspberry Pi. The survey paper [37] on smart grid listed a comprehensive review of different aspects of the smart grid, including physical power infrastructure, communication network, security and privacy, smart grid protocol, and cloud computing. A Fuss testing platform was developed in Queen's University cybersecurity testbed [38] using RTDS, actual IEDs, and merging units for testing IEC 61850 protocol.

It is obvious to understand the significance of testbed for holistically studying power systems. Yet, most testbeds mentioned before were designed to evaluate specific tasks [39]. These testbeds are not practically scalable. Also, these testbeds focused on conventional SCADA-based systems. Moreover, only a few testbed [29, 33, 34, 36, 40–43] experimented on the IEEE C37.118.2 protocol and synchrophasor technology. Among these testbeds, the paper [29, 40], demonstrated MITM attack, and the testbed paper [40] demonstrated MITM on IEEE C37.118 protocol. However, in this paper, the authors mostly focused on the MITM attack's effects on PMU measurements. They did not present how MITM, and FDIA happen in synchrophasor devices.

However, these testbeds are unsuitable for large scale deployment and vulnerability testing of synchrophasor technology. In addition, there are no open-source tools for analysing the IEEE C37.118 packets, according to our knowledge from literature reviews. Therefore, an emerging need for an IEEE C37.118 open-source tool that can perform different types of vulnerability testing, injection testing, and eavesdropping, false data injection attacks on the phasor measurements.

This research will demonstrate scalable cyber-physical system design in a simulation environment using different open-source tools like CORE network emulation, VirtualBox, Scapy, Vagrant, and Scapy. We will also demonstrate how to emulate cyber faults on a synchrophasor-based cyber-physical system. So, the main contribution of this research goes to.

• Designing a scalable CPPS distribution system testbed that incorporates the physical, cyber, and attacker layers employing different tools like Opal-RT, VirtualBox, Vagrant and CORE, Scapy, NetfilterQueue, and pyPMU. The testbed is named Smart Power System and Controls Testbed in short SPSC testbed.

• Implementation of the open-source python module named pySynphasor for analysing IEEE C37.118 protocol that can intuitively dissect, build and inject IEEE C37.118 packets in real-time.

• Demonstration of different cyber-attacks such as MITM, FDIA, FCIA, and Eavesdropping in the developed testbed by leveraging pySynphasor.

• Development of python based simple PDC module named pyPDC capable of collecting and aggregating data from multiple PMUs.

We will also present linear state estimation-based attack detection techniques for FDIA, enabling industry stakeholders and researchers to test similar attacks on real systems to find out the best detection technique.

So, the organization of the paper is as follows: Section 2 presents the methods of implementing scalable testbed and different types of attacks. Section 3 presents test case scenarios, section 4 presents a discussion, and finally, section 5 concludes the paper.

METHOD

The electric power system is intrinsically a cyber-physical system (CPS), with power flowing in the physical system and information flowing in the cyber network. The victim and attacker's points of view must be considered while designing a CPPS testbed. The victim's point of view consists of designing the physical, cyber, and detection layers. The attacker's point of view consists of designing different attack vectors on the communication protocols, identifying vulnerabilities in the system, and understanding the network topology to implement attacks effectively. Conventional testbed consists of two layers, that is, physical and cyber layers. Figure 1 depicts the designed SPSC testbed representing both the victims' and attackers' points of view. Therefore, the testbed consists of three layers, that is, (a) physical, (b) cyber, and (c) attack layers. The physical layer consists of IEEE 13-Node Test Feeder, PMU, control centre with PDC, and linear state estimator with bad data detection technique. The cyber layer connects all the equipment utilising a software-defined emulated network. Finally, the attack layer demonstrates deploying different cyber faults in a real system.

[IMAGE OMITTED. SEE PDF]

Physical layer design

The state of a power system can be expressed by the complex voltages of all buses [4]. It involves solving the network Quasi-Static model to find the states $$x$$ utilising the digital and analog measurements $${z}_{i}$$ from the system [44]. Power system measurement data $${z}_{i}$$ can be expressed with relation to the system states $$x$$ and measurement error $${e}_{i}$$. 1 $${z}_{i}={h}_{i}\,(x)+{e}_{i}$$

Here, subscript $$i$$ represents the $${i}^{th}$$ metre and $$h(.)$$ is the measurement function which expresses the relationship between measurements and state. A weighted least square (WLS) optimization problem can be formulated to solve this problem. By minimising $$j(x)$$, we effectively choose $$x$$ that best "fits” the measurements. 2 $$\mathit{min}\ J(x)=\sum\limits _{i}^{{N}_{m}}\,\frac{{\left({z}_{i}-{h}_{i}(x)\right)}^{2}}{{\sigma }_{i}^{2}}$$

Here, $${\sigma }_{i}^{2}$$ represents the variance of the metre measurements. Both equations can be written in matrix form for linear state estimation. 3 $$\overrightarrow{z}=H\overrightarrow{x}+\overrightarrow{e}$$ 4 $$\mathit{min}\,\overrightarrow{\ j}=\left(\overrightarrow{z}-H\overrightarrow{x}\right)W{\left(\overrightarrow{z}-H\overrightarrow{x}\right)}^{T}$$

Here, $$W={R}^{-1}$$ and $$R=diag\,\left\{\,{\sigma }_{1}^{2},{\sigma }_{2}^{2},\mathit{\dots }\right\}$$. The optimization problem can be solved by first-order optimal conditions. So, the estimated value of $${x}_{est}$$ can be obtained by $$\frac{\mathit{\partial }j}{\mathit{\partial }x}=0$$ [1, 4]. 5 $${\overrightarrow{x}}_{est}={\left({H}^{T}WH\right)}^{-1}{H}^{T}W\overrightarrow{z}$$

Linear state estimate is just a matrix multiplication with contrast to AC state estimation that reduces the computations complexities a lot. Yet, we need to build the $$H$$ matrices from system network [44]. $\begin{align*}H=[\begin{array}{c}II\\ M\end{array}]\\ M=yA+{y}_{s}\end{align*}$ Here $$II$$ represents voltage measurement bus incident matrix, $$A$$ current measurement bus incident matrix, $$y$$ represents series admittance matrix, $${y}_{s}$$ shunt admittance matrix. These matrices were formulated following the instruction from thesis [45].

One of the main goals of state estimation is to find out the bad metre measurements to eliminate those. As the state estimation problem is intrinsically over-determined in practice, there are more measurements than the states. So, bad measurements can be eliminated. For linear state estimation, the presence of a bad data detection [46] problem can be formulated as follows: 6 $$j=\sum\limits _{j}^{{N}_{m}}\,\frac{{e}_{j}^{2}}{{\sigma }_{i}^{2}}=\left(\overrightarrow{z}-H{x}_{\text{est}\,}\right)W{\left(\overrightarrow{z}-H{x}_{est}\right)}^{T}$$

$$j$$ follows the Chi-Square distribution, $${\mathrm{\mathrm{X}}}^{2}(K)$$. Here, $$K={N}_{m}-{N}_{s}$$ is known as degree of freedom, where $${N}_{m}$$ Number of total measurements and $${N}_{s}$$, Number of total states [47]. The presence of bad data can be estimated by checking the condition if $$j< {T}_{j}$$ , No bad data in the system, else, bad data exists. The threshold value, $${T}_{j}$$ can be obtained from the Chi-Square table utilising the degree of freedom (d.f) $$K$$ and metre accuracy, $$\sigma $$

There are multiple ways of detecting cyber-attack, such as network packet-based, data-driven, and physics informed. Firewall and intrusion detection systems are network-packet based and do not work efficiently in stealthy coordinated attacks [29] such as in the case of DDoS attacks. On the other hand, the bad-data-detection-based method is physics-informed detection mechanism. After detecting the presence of bad data in the measurement, it is also possible to identify the bad metre by hypothesis testing as follows: 7 $${\sigma }_{Y}=\vert \frac{{z}_{i}-{h}_{i}\left(\hat{x}\right)}{{\sigma }_{i}}\vert $$

If $${\sigma }_{Y} > 3$$, it identifies that $${i}^{th}$$ metre measurement is bad. The same method can also be utilised to detect FDIA attack on metre measurements.

Design the IEEE 13-node test feeder

We developed the testbed focussing on the IEEE 13 node test feeder which is a three-phase unbalanced distribution system operating in 4.16 kV [48]. The distribution system is designed in MATLAB Simulink and deployed in Opal-RT real-time simulator. Placing the $$\mu $$ PMU is an optimization problem to minimise the number of PMUs. The paper [4] identified the minimum number of PMUs for the IEEE 13 node test feeder and following that paper, we placed 5 $$\mu $$ PMUs in nodes 2, 3, 5, 9, and 10. These 5 $$\mu $$ PMUs measure voltage at five buses and current at nine lines in a total of 13 voltage and 20 current measurements for three phases. One phase is considered in the case of a transmission system due to its balanced nature. However, due to the unbalance in phases, the distribution system poses another challenge. The authors in paper [4] demonstrated that the three-phase unbalanced system could be decoupled into three-state estimation problems and possibly calculated by parallel computing. Figure 1a indicates the $$\mu $$ PMU placement with current and voltage measurements. The red diamond indicates the complex voltage measurements, and the green arrow indicates the complex current measurements. The yellow dashed border indicates the $$\mu $$ PMU. A single $$\mu $$ PMU measures multiple complex voltages and multiple line currents. We also added standard normal distributed noise with the simulated PMU measurements to mimic the real behaviour of the measurements. 8 $\begin{align*}\mathbf{N}=\sigma R(z)+\mu \\ {S}_{N}=S+\mathbf{N}\end{align*}$ Here, $$N$$ represents the generated noise utilising the standard normal distribution function $$R(z)$$, variance $$\sigma $$ and mean $$\mu $$. $${S}_{N}$$ is the noisy measurement that mimics the real metre measurements.

Implementation of pySynphasor

The latest synchrophasor standard has been split into two parts, IEEE Std. 37.118.1-2011, which covers measurement provision, and IEEE Std. 37.118.2™-2011, which covers data communication [49]. IEEE standard C37.244-2013 [50] suggested two communication protocols such as IEEE C37.118.2-2011 and IEC 61850-90-5 for PMU and PDC communication. However, IEEE C37.118.2 became the de-facto communication protocol for synchrophasor standard because of its compact packet size and low bandwidth requirement [8]. Yet, both protocols have unique features and limitations which are discussed in the paper [1].

With regard to the growing demand for an accessible open-source tool that can be used to implement different attack scenarios on the cyber-physical system of the smart grid, a notable progress has been made. Our team has successfully developed a Python module called pySynphasor, which is open-source and built upon the Scapy framework. This tool offers two essential functionalities for analysing synchrophasor packets. Firstly, it can effectively dissect the IEEE C37.118.2 packet from the network bitstream. Secondly, it has the ability to construct packets from raw measurements. The development of the pySynphasor module represents a significant advancement in the field and makes a substantial contribution to the research community.

The documentation and source code of the module are publicly available on GitHub at the specified URL, providing a valuable resource for further research and development within the community. To instal the module, users can simply execute the following pip command: pip instal pySynphasor, and it can be imported using the Python command: pySynphasor.synphasor import *. Detailed information regarding the API documentation and applications can be found on the module's website.

A simple demonstration of pySynphasor is shown in Listing 1 and 2. These listings illustrate the process of constructing a command packet and displaying its output in both human-readable and machine-readable formats. To stack communication layers, the div operator (/) is used between the synphasor and synphasor_cmd functions. Similarly, the data packet can be constructed by stacking the synphasor and synphasor_data layers. It is important to note that the module is built on top of the Scapy framework, providing full support for all Scapy functionalities.

The pySynphasor module is a versatile tool that can dissect and reassemble synchrophasor packets, allowing for various applications. Firstly, it can be utilised to analyse the security vulnerabilities of the IEEE C37.118.2 protocol, which is crucial for the protection and control of modern power systems. Secondly, the module can be used to design and implement attacks such as false data injection attacks, false command injection attacks, eavesdropping, and fuzz testing on the IEEE C37.118 protocol. These analyses help identify weaknesses and develop strategies to prevent such attacks. Lastly, the module enables the development of custom PMU and PDC applications tailored to specific needs, showcasing its versatility and potential in the development and analysis of modern power systems.

Implementation of synchrophasor devices

Power system tools typically lack the capability to model cyber-physical systems (CPS) with high fidelity. This often involves abstracting power system devices, such as networks, EMSs, IEDs, and field devices, to simplify the modelling process [7]. In our approach, we have designed PMU devices to closely mimic real CPPS environments. To achieve this, we deployed $$\mu $$ PMUs, PDCs, and network layers on virtual machines (VMs), which offer scalability and scripting advantages. The system requires the deployment of four types of VMs, each with distinct dependencies and setup mechanisms, posing challenges for scaling. To address these challenges and automate the VM deployment process, we leveraged Vagrant, a tool developed by HashiCorp for creating and managing portable virtual machine environments. Vagrant uses the Ruby programming language and provides instructions through a single file called Vagrantfile [51].

We leverage an open-source Python library named pyPMU [35] to develop the $$\mu $$ PMU. All the dependencies have been installed inside the virtual machine to mimic a real PMU. The developed $$\mu $$ PMU performs two significant tasks. First, it collects the phasor measurements from the distribution system inside the Opal-RT; second, pyPMU encodes phasor measurements into the IEEE C37.118.2 packet. Finally, it is waiting for the command from PDC to initiate the synchrophasor communication just as described in Figure 2. Manual installing so many PMU VMs is a cumbersome process. So, we utilised Vagrant to automate the PMU deployment process inside the VM.

[IMAGE OMITTED. SEE PDF]

All the steps of $$\mu $$ PMU installation is depicted in Figure 3a. From VM deployment to dependencies installation, all the steps are handled by the Vagrant script. The vagrant script is based on the Ruby environment. Therefore, it supports basic programming syntax like for-loop which elevates the scaling issue significantly. Hence, we implemented a base PMU device and then utilised a “for loop” to deploy the n number of PMUs. The command sequence of PMU deployment is essential because all the dependencies should be installed before changing the network card. Each PMU requires two network cards to work correctly. One network card works in bridge mode that connects with Opal-RT, and another network card works in internal network mode that connects the PMU with the smart-grid network. Then dependencies for the PMU, such as pyPMU, are installed in the script. Finally, static IP addresses are correctly set into both network cards to connect the physical and cyber layers.

[IMAGE OMITTED. SEE PDF]

As the pySynphasor can dissect and build IEEE C37.118.2 packets, we developed a simple PDC application name pyPDC utilising that. The pyPDC can receive data from multiple PMUs. The pyPDC is available in a subfolder of the pySynphasor. After installing the pySynphasor, the pyPDC can be import using the command from pySynphasor.pypdc import pyPDC. The pyPDC being a python application, gives us the advantage of deploying everything in a python environment utilising vagrant script. The steps for PDC deployment are shown in the Figure 3b.

First, VM deployment processes are written down in Vagrantfile sequentially utilising the vagrant script. Afterwards, a simple vagrant command “vagrant up” deploys the whole testbed except the CORE network. The script installs all the PMUs and PDC with dependencies and necessary setups within a few minutes. This automatic installation opens up the real potential of the testing cyber-physical power system in a hardware-in-loop and co-simulation environment with excellent fidelity and large scale. Both the PMU and PDC have critical network configurations that will be discussed in the subsection 2 (E).

Cyber layer modelling

Designing a highly scalable cyber-physical power system in the virtual environment is one of the main goals of this research. So, a software-defined network that runs in real-time is required to connect all the virtual machines and the physical power system. CORE (Common Open Research Emulator) is an open-source network emulator tool published by the US Naval Research Laboratory that runs on top of Linux. Being an emulator, it runs in real-time and connects many nodes, taking advantage of the Linux network namespace [52]. CORE has python API and a graphical user interface for building the emulated network. The paper [53] presented smart grid emulation where he compared different network simulators and found CORE to be more suitable for large-scale power system simulation. CORE has some advanced features such as (1) iptables [54] supports for firewall, Snort for intrusion detection, and SSH support for remote access. It also supports Docker containers that can be used to emulate routers, firewalls, personal computer and anything imaginable.

Figure 4 depicts the emulated network designed in the CORE emulator. The network is composed of routers, switches, and RJ45 connectors. Let us consider router n13. The network under router n13 is considered the substation network. Under the substation router, a switch connects all the devices in the substation, such as PMU, local PDC, relay, and RJ45 connector. The RJ45 connector is the interface between the virtual network inside CORE and PMU installed in VM. Therefore, the RJ45 connector is the main bridge between the independent host and emulated CORE network. The independent host can be any physical or virtual machine-like PMU VM. That means connecting hardware PMU and PDC with CORE network is possible, which is good for hardware-in-loop testing.

[IMAGE OMITTED. SEE PDF]

The VirtualBox network needs to be configured in a particular way to connect any host with the CORE virtual network through the RJ45 connector. VirtualBox has seven networking modes such as (a) Not Attached, (b) NAT, (c) NAT Network, (d) Bridge Network, (e) Internal Network, (f) Host-only network, and (g) Generic Network [55]. Different networking modes have different applications and different advantages and disadvantages. Internal network mode is suitable for this application as it allows connecting a particular VM with the CORE RJ45 ethernet port. While building the CORE VM, Promiscuous mode must be allowed during network interface card (NIC) configuration.

Enabling promiscuous mode is essential for CORE to receive incoming traffic from the physical network adapter and the rest of the PMU and PDC virtual machines. The IP address of the PMU-VM has to be set according to the network address of the substation router. Otherwise, the VM cannot connect with the CORE router. For example, the network IP of the link in Figure 4 connecting the enpos9 ethernet port is 192.168.0.1. Therefore, the allowable IP range is 192.168.0.2–192.168.0.255 for the PMU VM connected with the enpos9 RJ45 connector. The same rules apply to all the VM connected with the CORE network through the RJ45 connectors. Also, 192.168.0.1 will be the gateway address of that PMU. The network configuration mentioned above is automated in the Vagrant script for PMU and PDC except the CORE VM. We manually deploy and configure CORE network as it related to designing and configuring the network using GUI. It is also possible to automate the CORE VM deployment using the CORE python API.

Attacker modelling

We have developed an ARP poisoning script utilising Scapy. After deploying the script, all the packets from D1 (Alice) to D4 (Bob) are passed to the D3(Attacker) demonstrated In Figure 5. However, D4 (Bob) will not receive the packet unless the attacker enables packet forwarding. So, after enabling packet forwarding in the attacker machine, D1(Alice) and D4(Bob) can establish communication. In the case of MITM attack, the attacker convinces two victims that they are directly transferring data with each other [22]. This is the way MITM attack has been implemented in the SPSC testbed.

[IMAGE OMITTED. SEE PDF]

To implement FDIA in SPSC testbed, we utilised Linux iptables, NetfilterQueue, and pySynphasor. We developed a python script combining all of the tools to automate the FDIA attack process. The iptables [54] is the basic firewall programme of the Linux operating system. The packet filtering mechanism provided by iptables is organised into three different kinds of structures: (a) tables, (b) chains, and (c) targets [54]. First, the table allows processing packets in specific ways. Second, the tables have a chain attached to them, and it inspects traffic at a different point. Third, the target decides the fate of a packet, such as allowing, rejecting packets, or passing the packet to a queue. NetfilterQueue [56] is a filtering mechanism to get access to the packets by iptables rule so that user can accept, drop, alter, or reorder packets from kernel space to application space. Finally, we created iptables rules such a way, so that, any packet passing through the FORWARD chain; will be passed to the NetfilterQueue buffer (NFQ) just like depicted in Figure 6b.

[IMAGE OMITTED. SEE PDF]

After reading the NFQ buffer packet, the developed module, pySynphasor plays a vital role in synchrophasor packet injection. As pySynphasor can intuitively build and dissect the IEEE C37.118.2, the attacker leverages it to inject false data into measurement. It reads the network packet from NFQ buffer and builds an internal object structure in the Scapy framework.

The internal representation is helpful in the Scapy framework as it allows the user to modify and redesign the packet. After modifying or injecting the packet with false data, pySynphasor rebuilds the network packet. Another critical point while rebuilding the packet after the injection is updating the packet fields that depend on the packet's payload, such as IP length, TCP length, TCP checksum, and IEEE C37.118.2 CRC value. Listing 3 demonstrates a sample example, how pySynphasor implement such FDIA attack using few lines of Python script.

Figure 6b depicts the direction and flow of the packet in between Linux kernel space and user space. The red line indicates the path of the synchrophasor packets while passing through the attacker's machine. An automated script was built to perform the FDIA mechanism. The script begins with preparing the FDIA environment that includes (a) enabling packet forwarding, (b) resetting iptables (c) adding new iptables rules. Then, the script filters the TCP packet that is aimed at the target machine. Afterwards, the script injected false measurements. The next part is a little tricky. As the packet changes after the injection, a few fields, such as IP length, TCP length, IP checksum, TCP checksum, and IEEE C37.118.2 CRC, need to recalculate again. Otherwise, the receiver machine will discard the packet considering as invalid.

Hence, for a successful FDIA, the steps are (a) Capture packet from Linux For-ward chain and pass it to NetfilerQueue buffer, (b) When packets are available on the buffer, accept them and inject false data using pySynphasor. (c) Recalculate packet fields that changed due to injecting false measurements. (d) release the packet to the Linux For-ward chain.

RESULT

pySynphasor packet building and dissection results

We have developed the pySynphasor module on top of Scapy [57], which is an interactive packet manipulation programme based on python. It has three types of packet representation: internal, machine, and human. Machine representation is the actual raw packet that will be sent through the network. These are just binary bits that are not easy to deal with. Scapy manipulates the packet using internal representation that is just an object representing machine format and is not intuitive either. Human representation is the human-readable representation in plain text. So, this representation is easy to deal with for injecting packets. Figure 7 represents the human-readable representation of IEEE C37.118.2 data packet. Therefore, pySynphasor module along with the SPSC testbed made it intuitive to analyse synchrophasor packet.

[IMAGE OMITTED. SEE PDF]

The packet (pkts [4]) was captured during a sample PMU and PDC communication in the SPSC testbed. The Figure 7 demonstrates that the packet dissection was accomplished just by applying show (.) method on the packet (pkt [4]). Then pySynphasor decodes machine representation and presents the IEEE C37.118.2 packet in a human-readable format. The dissected packets show the different parts of data packet, such as, IP, TCP, IEEE C37.118.2 Comon Frame, IEEE C37.118.2 Data. The whole process is abstracted away inside pySynphasor. The data packet has five sub-segments: an Ethernet Header, IP header, TCP header, IEEE C37.118.2 common frame, and IEEE C37.118.2 data frame. Data frames represent the phasors measurements in complex number format. The Figure 7 just presents the dissection of data packet only. Similarly, pySynphasor is capable of dissecting IEEE C37.118.2 command, configuration, and header packets.

As mentioned earlier, pySynphasor not only dissects IEEE C37.118.2 packets but also builds the packet from raw measurement. Unlike pyPMU [35] and other tools, it is a bi-direction tool. Figure 8 demonstrated an example of intuitively building the IEEE C37.118.2 network packet from raw measurement just using a few lines of pySynphasor commands.

[IMAGE OMITTED. SEE PDF]

CORE network connection result

Figure 9 presents an example of how CORE connects the PMU and PDC that were deployed in the virtual machine. Figure 9a depicts a sample CORE network deployed in VM for connecting PMU and PDC. Figure 9b presents PMU deployed in a VM that transfers phasor measurement after a specific interval. The PMU device deployed here is built using the pyPMU [35] python module. This VM is connected to the RJ45 ethernet interface enp0s8 in the CORE network. Figure 9c is another VM connected with the enp0s10 RJ45 network interface where the PMU Connection Tester application is deployed for verifying the connection with PMU through the CORE network. PMU Connection Tester plotted four phasors measurements when it received data from the PMU device. All the phasors measurements are flat because PMU sends a constant value. So these results verify the successful connection of PMU and PDC through the CORE virtual network in our proposed testbed.

[IMAGE OMITTED. SEE PDF]

ARP poisoning result

ARP poisoning mechanism is used to implement MITM attack in our smart-grid testbed. The primary technique of this type of implementation is that the attacker has to poison two victims' ARP table. Figure 10 presents the result of ARP poisoning where PMU and PDC are deployed in a local network in two separate VMs. Figure 10 presents the ARP table of both victims before and after the poisoning. If we look into the ARP table of the PMU device, The IP address of PDC is 10.0.2.7, and the MAC address is 08:00:27:69:58:64 before the attack. The MAC address changed to 08:00:27:a7:1b:c3 in the PMU ARP table after the poisoning; that is, the MAC address of the attacker machine. That means the PMU ARP table is poisoned, and all packets from PMU to PDC will go through the attacker's machine. Table 1 presents the summary of ARP poisoning before and after the ARP poisoning. The same thing happened to the ARP table of PDC VM.

[IMAGE OMITTED. SEE PDF]

TABLE 1 ARP poisoning table.

Device	IP address	Original MAC	Spoofed MAC
PDC	10.0 0.2 0.7	$$08:00:27:69:58:64$$	$$08:00:27:\mathrm{a}7:1\,\mathrm{b}:\mathrm{c}3$$
PMU	10.0 0.2 0.4	$$08:00:27:c9:\mathrm{f}:61$$	$$08:00:27:\mathrm{a}7:1\,\mathrm{b}:\mathrm{c}3$$

IEEE C37.118.2 FDIA result

After the ARP poisoning, the attacker machine has full access to the victim packet. It created a temporary queue for manipulating and forwarding the packet afterwards. In subsection 2(F) we explained how the attacker gets access to the synchrophasor data from Linux kernel space to the user space through NetfilterQueue and iptables. Afterwards, utilising the pySynphasor module, the attacker injects the synchrophasor packets. Figure 6 presents scenarios of MITM and FDIA attack mechanism and results. Here, PMU and PDC are transferring packets normally. By poisoning the PMU and PDC ARP table, the attacker now steals the session of the PMU and PDC. After deploying the automatic FDIA attack script, it now gets access to the synchrophasor data. In this way, the attacker is eavesdropping on the synchrophasor packet.

Figure 6d presents PMU data packet before injection and Figure 6e presents the same packet after injection that was capture from PDC VM. From the Figure 6d, we can observe that the phasor measurements were [(2453 + 2444j) (2954 + 2780j) (2922 + 2079j)] before the attack, and after the attack, it was injected to [(2402+0j) (58,218 + 2860j) (58,218 + 12,675j)]. The packet injection mechanism is just a few lines of pySynphasor commands, just demonstrated in Figure 8. As the packet contents are changing, it is mandatory to update the fields that depend on packet content, such as TCP and IP packet length, checksum, and IEEE C37.118.2 CRC value. Updating of these signature fields is also performed in the script. After the injection, the script forwards the packet to the PDC. This is how the testbed smoothens the different attack scenarios for testing large system deployment.

Attack detection results

The distribution system is intrinsically unbalanced. This unbalanced system can be realized as the three separate equations [4] for designing state estimation problem. These three separate equations can be solved by parallel processing, reducing computation time. So, we formulated the three-state estimation problems and bad data detection mechanism and calculated the voltages and phase angles of the individual phases. Figures 11a and 11b presents the estimated and actual bus voltage and phase angle of phase A for IEEE 13 node test feeder. The results were generated based on the following scenarios: For phase A, buses 5,6, and 11 are not present. Therefore, 4 PMU measurements were used while calculating states. Seven current and four voltage measurements comprised in total of $${N}_{m}=11$$ measurements. Bus seven and eight are connected with a breaker, and bus 12 has no load. So, we need to estimate the voltage for bus 2,3,4,7,9,10,13. Therefore, the total number of states to estimate is $${N}_{n}=7.$$ The noise was added with the metre measurement utilising the equation (8) $$\sigma =0.05$$ and mean $$\mu =0$$ were considered while adding noise for this result.

[IMAGE OMITTED. SEE PDF]

We utilised the equation (6) and Chi-square distribution table for detecting presence of bad data. For the degree of freedom $$K={N}_{m}-{N}_{n}=11-7=4$$ and $$\sigma =0.05$$, the threshold value $${T}_{j}=9.49$$ obtained from the Chi-square distribution table. Figure 11c represents the metre error for all 11-measurement calculated using the hypothesis testing equation (7). But the value of $$j=19.6005 > {T}_{j}$$. That indicates that there is bad data present in the measurement. The Figure 11c also demonstrates the result of hypothesis testing on metre measurements. The error value $${\sigma }_{Y} > 3$$ for metre one. It proves that the attacker poisoned the metre one measurement.

DISCUSSION

Our main goal was to present a scalable cyber-physical testbed for smart grid system so that the SPSC testbed mechanism can be utilised to experiment on different cyber-attack detection mechanisms for large-scale system. The goal is to prevent attack like 2015 blackout in Ukraine due to the false command injection attack (FCIA) [15]. In the method section, we described how to develop such high-fidelity testbed that can mimic a real system. Unlike other CPPS testbed research, we not only focused on designing the testbed but also demonstrated how to design real-time attacks on the testbed using our developed module pySynphasor so that the system can be studied from both the attacker's and the victim's point of view. Another major part of the testbed was the demonstration of the pySynphasor module. It can be utilised in many ways because of its complete capabilities of building and dissecting the IEEE C37.118.2 protocol. As an example, application, we developed a simple PDC named as pyPDC utilising the module. Being a bi-directional packet manipulation tool, application of pySynphasor is limitless. We also demonstrated the FDIA attack utilising the module. It is also possible to design different types of attacks in IEEE C37.118.2 protocols using the pySynphasor. One of the potential testings might be implementing fuzz testing on synchrophasor devices. Therefore, pySynphasor has many usage cases to explore in future research.

Then, in the result section, we presented how the whole testbed works. In the III-B subsection, we presented how software emulated the network connecting the whole testbed. The smart grid is ready and equipped with a cyber and physical layer. In the III-A subsection, we presented how pySynphasor can build and dissect IEEE C37.118.2 packets intuitively. Performing cyber-attacks was one of the main goals of the SPSC testbed. In the III-C subsection, we presented how to access the PMU data packet by applying the ARP poisoning mechanism. In the III-D subsection, we presented how to inject false data on the phasor measurements utilising the pySynphasor module. Finally, we focused on the detection mechanism. So, subsection III-E presented how linear estate estimation based bad data detection can identify false data injection attacks on synchrophasor data. The synchrophasor standard suggested IEEE C37.118.2-2011 and IEC 61850-90-5 protocols for PMU and PDC communication. But, in this research IEEE C37.118.2 protocol has been explored and did not address IEC 61850-90-5 protocol.

CONCLUSION

The development of a scalable CPPS SPSC testbed has been presented. The effectiveness of such testbed has been demonstrated to identify the best detection mechanism for a more extensive system. We also presented the mechanism of building and dissecting synchrophasor packets, which is also useful in packet-based detection. Finally, we presented a physics-based detection mechanism, such as state estimation and bad data detection. Although the system is highly scalable, it can be made more robust and lightweight by using docker container instead of the virtual machine. Therefore, building the whole testbed on top of the docker container will be follow-up research. Designing the testbed around IEC61850 will be another improvement of current research.

AUTHOR CONTRIBUTIONS

Shuvangkar Chandra Das conceptualised the research, developed the pySynphasor module in Python, and built the testbed. Dr. Tuyen Vu supervised the project, revised the manuscript, and provided key suggestions for improving the testbed's attack scenario. Dr. Herbert Ginn co-supervised the project and revised the manuscript.

ACKNOWLEDGEMENTS

The information, data, or work presented herein was funded in part by the U.S. Office of Naval Research under award number N000142212239.

CONFLICT OF INTEREST STATEMENT

The authors declare no conflict of interest.

DATA AVAILABILITY STATEMENT

The IEEE C37.118 protocol implementation in Scapy is available open-source on these links.

.

References

Khan, R., et al.: Threat Analysis of BlackEnergy Malware for Synchrophasor Based Real‐Time Control and Monitoring in Smart Grid, pp. 1–11 (2016). MAG ID: 2514382028

Wu, F.F.: Power system state estimation: a survey. Int. J. Electr. Power Energy Syst. 12(2), 80–87 (1990). https://doi.org/10.1016/0142‐0615(90)90003‐t

Wang, H., Schulz, N.: A revised branch current‐based distribution system state estimation algorithm and meter placement impact. IEEE Trans. Power Syst. 19(1), 207–213 (2004). https://doi.org/10.1109/tpwrs.2003.821426

Chen, X., Tseng, K.J., Amaratunga, G.: State estimation for distribution systems using micro‐synchrophasors. In: Proceedings of the 2015 IEEE PES Asia‐Pacific Power and Energy Engineering Conference, pp. 1–5. APPEEC (2015)

Nayak, G.N., Samaddar, S.G.: Different flavours of Man‐In‐The‐Middle attack, consequences and feasible solutions. Proceedings of the 2010 3rd IEEE International Conference on Computer Science and Information Technology (ICCSIT) 5, 491–495 (2010). https://doi.org/10.1109/iccsit.2010.5563900

Yang, Y., et al.: Man‐in‐the‐middle Attack Test‐Bed Investigating Cyber‐Security Vulnerabilities in Smart Grid SCADA Systems, pp. 611. IET Conf. Publ. (2012). CP

Yohanandhan, R.V., et al.: A specialized review on outlook of future Cyber‐Physical Power System (CPPS) testbeds for securing electric power grid. Int. J. Electr. Power Energy Syst. 136, 107720 (2022). https://doi.org/10.1016/j.ijepes.2021.107720

Khan, R., et al.: Analysis of IEEE C37. 118 and IEC 61850‐90‐5 synchrophasor communication frameworks. ieeexplore.ieee.org, 1–5 (2016). publisher: PESGM. https://doi.org/10.1109/pesgm.2016.7741343

Bilge, L., Dumitra, T.: Before we knew it: an empirical study of zero‐day attacks in the real world. In: Proceedings of the 2012 ACM Conf. On Computer and Communications Security, Ser. CCS ’12, pp. 833–844. Association for Computing Machinery, New York (2012)

Liu, S., Liu, X., El Saddik, A.: Denial‐of‐Service (DoS) Attacks on Load Frequency Control in Smart Grids, pp. 1–6 (2013)

Kazemy, A., Hajatipour, M.: Event‐triggered load frequency control of Markovian jump interconnected power systems under denial‐of‐service attacks. Int. J. Electr. Power Energy Syst. 133, 107250 (2021). https://doi.org/10.1016/j.ijepes.2021.107250

Hoque, N., Bhattacharyya, D.K., Kalita, J.K.: Botnet in DDoS attacks: trends and challenges. IEEE Commun. Surv. Tutor. 17(4), 2242–2270 (2015). https://doi.org/10.1109/comst.2015.2457491

Upadhyay, D., Sampalli, S.: SCADA (Supervisory Control and Data Acquisition) Systems: Vulnerability Assessment and Security Recommendations. Elsevier Enhanced Reader’ (2019)

Liu, Y., Ning, P., Reiter, M.K.: False data injection attacks against state estimation in electric power grids. ACM Trans. Inf. Syst. Secur. 14(1), 1–33 (2011). https://doi.org/10.1145/1952982.1952995

Liang, G., et al.: The 2015 Ukraine blackout: implications for false data injection attacks. IEEE Trans. Power Syst. 32(4), 3317–3318 (2017). https://doi.org/10.1109/tpwrs.2016.2631891

Writer, C.S.: High voltage attack: EU’s power grid organisation hit by hackers. [Online]. https://techmonitor.ai/technology/cybersecurity/eu‐power‐grid‐organisation‐hacked (2020)

Turton, W., Mehrotra, K.: Hackers breached Colonial Pipeline using compromised password. Bloomberg.com (2021)

Orman, H.: The Morris worm: a fifteen‐year perspective. IEEE Secur. Priv. 1(5), 35–43 (2003). https://doi.org/10.1109/msecp.2003.1236233

Kerr, P.K., Rollins, J., Theohary, C.A.: The Stuxnet Computer Worm: Harbinger of an Emerging Warfare Capability, pp. 12 (2010)

Langner, R.: Stuxnet: dissecting a cyberwarfare weapon. IEEE Secur. Priv. 9(3), 49–51 (2011). https://doi.org/10.1109/msp.2011.67

Yohanandhan, R.V., et al.: Cyber‐Physical Power System (CPPS): a review on modeling, simulation, and analysis with cyber security applications. IEEE Access 8, 151019–151064 (2020). https://doi.org/10.1109/access.2020.3016826

Khan, R., et al.: IEEE C37.118‐2 Synchrophasor Communication Framework Overview, Cyber Vulnerabilities Analysis and Performance Evaluation. scitepress.org (2016). ISBN: 9789897581670

Fan, X., Du, L., Duan, D.: Synchrophasor data correction under GPS spoofing attack: a state estimation‐based approach. ieeexplore.ieee.org, 1–9 (2018). https://doi.org/10.1109/tdc.2018.8440488

Paudel, S., Smith, P., Z. f. I. S. C. S. S. T.: Data integrity attacks in smart grid wide area monitoring. scienceopen.com, 2016, publisher: BCS Learning and Development (2016)

Singh, V.K., Ozen, A., Govindarasu, M.: Stealthy cyber attacks and impact analysis on wide‐area protection of smart grid. ieeexplore.ieee.org, 1–6 (2016). ISBN: 9781509032709. [Online]. Available:. https://doi.org/10.1109/naps.2016.7747927

Govindarasu, M., Liu, C.‐C.: Cyber Physical Security Testbed for the Smart Grid: Fidelity, Scalability, Remote Access, and Federation, pp. 3 (2013)

Elanovic, N., et al.: Cyber physical systems approach to power electronics education. Electron 16(2), 125–129 (2012)

Koty, K., Gajewski, R.: SHaPe: a honeypot for electric power substation. J. Telecommun. Inf. Technol.(nr 4), 37–43 (2015). https://doi.org/10.26636/jtit.2015.4.983

Sahu, A., et al.: Design and evaluation of a cyber‐physical testbed for improving attack resilience of power systems. IET Cyber‐Phys. Syst.: Theory Appl. 6(4), 208–227 (2021). https://doi.org/10.1049/cps2.12018

Johnson, J., et al.: Assessing DER network cybersecurity defences in a power‐communication co‐simulation environment. ieeexplore.ieee.org 5(3), 274–282 (2020). https://doi.org/10.1049/iet‐cps.2019.0084

Wlazlo, P., et al.: Man‐in‐the‐middle attacks and defense in a power system cyber‐physical testbed. arXiv (2021). [Online]. http://arxiv.org/abs/2102.11455

Rodofile, N.R., Radke, K., Foo, E.: Real‐time and interactive attacks on DNP3 critical infrastructure using Scapy. Tech. Rep. (2015)

Khan, R., et al.: Demonstrating cyber‐physical attacks and defense for synchrophasor technology in smart grid. 2018 16th Annual Conf. on Privacy, Security and Trust (PST), 1–10 (2018). IEEE, ISBN: 9781538674932. https://doi.org/10.1109/pst.2018.8514197

Cui, H., Li, F., Tomsovic, K.: Cyber‐physical system testbed for power system monitoring and wide‐area control verification. ieee‐xplore.ieee.org 2(1), 32–39 (2020). https://doi.org/10.1049/iet‐esi.2019.0084

Andi, S., Krstaji, B.: pyPMU Open source python package for synchrophasor data transfer. ieeexplore.ieee.org (2016)

Adhikari, U., Morris, T.H., Pan, S.: A cyber‐physical power system test bed for intrusion detection systems. ieeexplore.ieee.org (2014). ISBN: 9781479964154

Cintuglu, M.H., et al.: A survey on smart grid cyber‐physical system testbeds. ieeexplore.ieee.org 19(1), 446–464 (2017). [Online]. Available:. https://doi.org/10.1109/comst.2016.2627399

Yang, Y., et al.: Cybersecurity test‐bed for IEC 61850 based smart substations. ieee‐xplore.ieee.org, 1–5 (2015). , publisher: IEEE. https://doi.org/10.1109/pesgm.2015.7286357

Hahn, A., et al.: Cyber‐physical security testbeds: architecture, application, and evaluation for smart grid. IEEE Trans. Smart Grid 4(2), 847–855 (2013). https://doi.org/10.1109/tsg.2012.2226919

Liu, R., et al.: Analyzing the cyber‐physical impact of cyber events on the power grid. IEEE Trans. Smart Grid 6(5), 2444–2453 (2015). https://doi.org/10.1109/tsg.2015.2432013

Adhikari, U., Morris, T., Pan, S.: WAMS cyber‐physical test bed for power system, cybersecurity study, and data mining. IEEE Trans. Smart Grid 8(6), 2744–2753 (2017). https://doi.org/10.1109/tsg.2016.2537210

Aghamolki, H.G., Miao, Z., Fan, L.: A hardware‐in‐the‐loop SCADA testbed. 2015 North Am. Power Symp. (NAPS), 1–6 (2015). https://doi.org/10.1109/naps.2015.7335093

Kezunovic, M., et al.: The Use of System in the Loop, Hardware in the Loop, and Co‐modeling of Cyber‐Physical Systems in Developing and Evaluating Smart Grid Solutions, pp. 10 (2017)

Monticelli, A.: Electric power system state estimation. Proc. IEEE 88(2), 262–282 (2000). https://doi.org/10.1109/5.824004

Tarali, A.: Bad Data Detection in Two Stage Estimation Using Phasor Measurements. Ph.D. dissertation. Northeastern University (2012)

Zhang, L., et al.: Design, testing, and implementation of a linear state estimator in a real power system. IEEE Trans. Smart Grid 8(4), 1782–1789 (2017). https://doi.org/10.1109/tsg.2015.2508283

Bandak, C.E.: Power Systems State Estimation. Ph.D. dissertation (2013)

Kersting, W.: Radial distribution test feeders. 2001 IEEE Power Eng. Soc. Winter Meeting. Conf. Proc. (Cat. No.01CH37194) 2, 908–912 (2001). https://doi.org/10.1109/pesw.2001.916993

IEEE: IEEE Std C37.118.2‐2011 (Revision of IEEE Std C37.118‐2005), pp. 1–53 (2011)

IEEE: IEEE Std C37.244‐2013, pp. 1–65 (2013). ISBN: 978‐0‐7381‐8260‐5

Hashimoto, M.: Vagrant: up and running: create and manage virtualized development environments, (2013)

Rosen, R.: Linux Kernel Networking: Implementation and Theory. Apress (2014)

Tan, S., et al.: SCORE: smart‐Grid common open research emulator. In: 2012 IEEE Third Int. Conf. On Smart Grid Communications (SmartGridComm), pp. 282–287 (2012)

Purdy, G.N.: Linux Iptables Pocket Reference: Firewalls, NAT and Accounting. O’Reilly Media, Inc. (2004)

VirtualBox: Chapter 6. Virtual networking. [Online]. https://www.virtualbox.org/manual/ch06.html

Ore, J.: oremanj/python‐netfilterqueue. [Online]. https://github.com/oremanj/python‐netfilterqueue (2022)

Biondi, P.: Network packet manipulation with Scapy, (2007)