Comparing the Use of Simu5G and 5G-Sim-V2I/N Modules When Analysing the Edge Computing Resource Management Efficiency

This is an open access article under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited.

1. Introduction

The continuous rise of mobile computing capabilities and the proliferation of smart, connected gadgets that created the internet of things (IoT) have made a tremendous change on end user expectations regarding the features and proficiency of available services that can be requested anytime, anywhere [1]. The main advances in this area are especially prominent with the rise of real-time or near real-time multimedia services, such as augmented reality, that have highly increased the requirements for the quality of service (QoS) that should be offered by the service providers [2]. To be able to support the QoS requirements of the new generation of mobile services, service providers can no longer afford to host the service in a high capacity distant cloud and rely on the wide network bandwidth for fast delivery, as the delay introduced by utilising the remote cloud data centre is not acceptable for near real-time responses. This situation has led to expanding the computing paradigm landscape with the concept of edge computing [3], wherein the services with high QoS requirements, specifically regarding minimum delay under heavy computing demands, are to be hosted close to the user, minimising the network distance. In essence, it is expected that the next generation of mobile communication will colocate micro data centres together with base stations at the very edge of the network to provide access to additional computing power available just one hop away from the users [4].

The deployment of services at the edge provides very high benefits in terms of ultralow latency and high bandwidth provisioned by the access network combined with ample computing and storage virtualised capacity in the micro data centre. Together, they help implement a new type of edge intelligence enabling fast, yet reliable, data collection, analysis, and response thus creating the foundation required for the Industry 4.0 [5] advancement. To take full advantage of the edge computing concept, it should be coupled with state-of-the-art mobile communication technologies (such as 5G and beyond) and be managed using a highly agile orchestration approach. This approach will maximise the utilisation of the available edge computing resources and enable smart, adaptive algorithms to deploy and manage edge service instances [6]. Therefore, one of the goals for the evolution of beyond 5G systems is to develop an intelligent orchestration solution of an integrated radio access network (RAN) and multiaccess edge computing (MEC) architecture that will provide a highly scalable, interoperable, collaborative ecosystem [7].

The effectiveness and overall edge service performances are highly dependent on ensuring continuous optimised communication between the user and the edge service, which entails smart decisions on the optimum placement of the edge service during deployment and on the continual migration that implements a follow-me behaviour for mobile users [8]. In an ideal scenario, the edge service used by the user will be running on the virtualised computing resources that are colocated with the base station the user is connected to. In highly mobile environments, this essentially translates to a very dynamic system behaviour as the edge services are being migrated in tandem with the handoff events generated between base stations, while the user moves from one zone coverage to another. Thus, it is imperative that the edge computing implementation includes an intelligent service orchestration that will be able to monitor the user location and QoS levels and make optimised decisions regarding the corresponding location of the instantiated edge services, while taking into account the restrictions of the available virtual computing resources in the micro data centres.

The typical process of design, modelling, and performance analysis of edge orchestration algorithms is performed in a closed environment focusing exclusively on the virtualised computing resources available and their optimised use. The simulation scenarios are created using varying artificial workloads fed to the orchestrator representing randomised users requests for allocation of new services and information regarding the users’ mobility needed for the follow-me continuous optimisation. This is, of course, an important first step to validate the approach and gauge its effectiveness when compared to existing approaches. In this way, one can measure the local edge orchestration performances such as effectiveness in the use of the available virtual resources and delay introduced by the placement of the edge service relative to the user position. In the absence of an underlying access network, it is not possible to analyse the end-to-end latency that describes the edge orchestration performances as perceived by the end users.

Thus, once the algorithms show promising results in isolation, their performance should be analysed when the edge computing system is deployed on top of an underlying 5G network, by creating a 5G+edge ecosystem. In this way, the edge orchestration performance analysis is taken to the next level, where the behaviour of the newly developed edge orchestration algorithms can be considered in detail using a large-scale mobile simulation scenario that is close to real-life developed use cases, such as intelligent transportation networks for an example. The purpose of this step is to validate the expected performances perceived in isolation in a more realistic simulation environment that also allows for extended analysis such as the measurement of the achieved end-to-end latency which is considered one of the most critical indicators of edge computing intelligent orchestration [9]. The 5G+edge ecosystem approach provides the possibility to analyse not just the overall end-to-end delay, but also its main constituting parts, namely, the 5G RAN or edge infrastructure [10]. It is also of great importance to understand the performances and stability of the ecosystem as a whole when deployed in a realistic, large-scale, urban environment.

From an edge orchestration design perspective, the edge computing infrastructure sits on top of an existing 5G network and is considered just a small subset of a large number of services that can be offered by the 5G service provider. Therefore, the 5G network should be added to the simulation scenario by using a readily available 5G simulation framework to model and simulate the 5G RAN. If the choice of a 5G simulation framework is made based on validated frameworks that implement the 5G standard, it is not expected that the concrete framework will have a major impact on the obtained results. Putting this reasoning into practice, however, is not as straightforward. The main goal of this paper is to offer an insight into the complexities of simulating the 5G+edge ecosystem, focusing on the effects of the choice of 5G framework that needs to be used to implement the RAN in a smart edge orchestration scenario. Specifically, we analyse the differences in terms of end-to-end delay obtained when running large-scale urban simulation scenarios using OMNeT++ [11] for the networking part with two different extensions that can be used to model the 5G RAN: the Simu5G [12] and the 5G-Sim-V2I/N [13].

Our results show that switching to a different 5G framework can significantly change the obtained macro results related to the end-to-end latency experienced by the edge service users. When comparing the obtained results from the two 5G RAN models, the average end-to-end delay values differ by a factor of 2 to 21, while some per-user delay differences reach several orders of magnitude. Whereas some variation in the obtained results is quite tolerable when taking into account the difference in the implementations of the 5G simulation frameworks, the obtained significant variations in the results show that one must be very careful in the choice of the 5G network. Furthermore, as the edge orchestration heavily depends on the information provided by the serving base station for its optimal scheduling of edge services, the workload generated for the edge orchestrator increases when using one framework compared to the other because of the differences in handling the base station assignments and handoffs. Thus, when focusing on the performance of smart edge service orchestration algorithms, one should ensure that the 5G framework used for the analysis implements close to real-life behaviour such as Simu5G in this cross-comparison.

The rest of this paper is structured as follows: In Section 2, we discuss the requirements and approaches to defining a holistic 5G+edge simulation environment for large-scale scenarios. In Section 3, we introduce the complete edge orchestration simulation model with toolsets and focus on the different alternatives for simulating the 5G network communication, describing the simulation environment used. Section 4 provides a quantitative and qualitative comparative analysis of the end-to-end delay results obtained using the two different frameworks and discusses the potential implications of the observed effects together with a cross-comparison of the two different frameworks and their parameters. Finally, the last section concludes the paper and provides potential future research directions.

2. 5G+Edge Vehicular Simulation Environments

The capabilities introduced with the combination of 5G and edge computing have a major impact on the development of new services and features in a wide variety of vertical markets [14]. Some of the major use cases stem from the automotive industry, wherein edge computing enables the transition from smart vehicles to intelligent transportation networks transforming the automotive landscape by incorporating it into the so-called internet of vehicles [15]. Use cases such as advanced driver assist systems and vehicle to anything (V2X) communication for enhanced connected cars or intelligent infotainment systems for augmented passenger experience are just some of the examples that can greatly benefit from the integration of edge computing and 5G [16]. While the 5G network is used to connect vehicles to intelligent infrastructure or other entities, the edge computing part provides the vehicle with extra computing power running edge services [17].

The automotive industry use cases are of special interest to edge designers as they define specific, strict requirements in terms of QoS for edge services such as reliability and performance guarantees which are essential for the safety of all participants in the next-generation smart cities. For example, one of the use cases described by the 5GAA [18] requires a service reliability of 99.9% with a maximum service delay of 10 ms in a scenario with over 1.000 vehicles per km². From a 5G perspective, these requirements should be easy to fulfil given that according to the 5G technology requirements [19], the service reliability is 99.999% with less than 1 ms end-to-end delay while supporting 1 million devices in 1 km². However, achieving these numbers in real-life scenarios requires very careful design and network planning based on small cells, massive multiple input–multiple output (MIMO) antennas that support beamforming, new radio frequencies, channel coding techniques, etc. In essence, there is a large number of parameters that need to be taken into account and attuned to achieve an efficient 5G network deployment. The success of the automotive use cases will also depend on the optimisations implemented in the edge service orchestration layer on top of the 5G infrastructure. With the edge orchestrator in charge of the edge service deployment and lifecycle management, the identified requirements must be satisfied across the full end-to-end ecosystem that connects the user with the corresponding edge service. From the edge computing perspective, it is therefore necessary that there is an efficient continuous feedback loop that provides information about the current user location and QoS to the intelligent edge orchestrator so that the edge services are distributed across the available virtualised edge resources in the most optimal way.

To analyse such scenarios in depth, modelling of both the 5G mobile connection and the edge computing orchestration is needed. A large corpus of the available edge computing research focuses only on the edge orchestration part, not taking into account the 5G communication network that is essential to establish a connection with the edge services user. Lately, there has been a rising number of attempts that focus on the synergy of 5G and edge computing and analyse the end-to-end performances. Many of these are based on vehicular use cases, and mostly deal with simplified environments such as strips of highways [20] or one intersection in the city [21], and other similar examples [22]. There are also a few examples where the simulation modelling is done for a larger scale urban city scenario such as [23] which show that the overall performances perceived by the user are greatly influenced by both the edge orchestration management (where the edge service is placed, is it being migrated to accommodate for user mobility or no, and what types of optimisations are implemented) and the 5G network performances (available bandwidth, distribution of base stations, coverage area, access network delay, etc.).

Therefore it can be concluded that the investigation of the edge orchestration components’ performance needs to be done by incorporating a model of the underlying wireless mobile access network that is used to implement the communication between the user and the edge services. Moreover, when the focus is put on edge orchestration management, researchers would like to be able to use readily available tools to model a “default” underlying 5G network.

Today, many more approaches tend to carry out the performance analysis using a MATLAB based or similar numerical computer simulation approach such as [24, 25] or [26] to provide just a few examples. On the other hand, there are options to use commonly known tools such as OMNeT++ [11] or NS [27] that offer the use of a full network stack simulation software. Such analysis can be found, for example, in [23, 28] and [29]. This approach ensures the incorporation of all relevant networking aspects into the simulation scenario, in contrast to them being omitted when focusing on the numerical aspects of the problem. For a large-scale urban vehicular scenario, the incorporation of a full network stack is of even greater importance because of the inherent highly dynamic nature of the environment. Namely, the mobility pattern can potentially greatly challenge the implementation of the intelligent edge orchestration by creating numerous events of new service requests, service migration requests, and service termination requests as the vehicles enter the edge computing service area, move from one base station to another, and then leave the service area. All of these events are represented with various types of network packets, generated on both the data and control plane, and need to be successfully transferred between the RAN and edge system components. A high vehicle density in an urban environment will further augment the frequent QoS changes. It is therefore a very good example of a scenario that can be used to test the limits of an edge orchestrator in terms of the efficiency of the implemented algorithms and latency introduced by the decision-making process.

When it comes to the implementation, the problem of using a full stack network simulation for the analysis turns out to be more difficult than expected. First, several different 5G frameworks can be used for these purposes, and one needs to make the correct choice. Furthermore, the question of how much the choice of the 5G framework influences the perceived edge service performances arises. In theory, the choice of a 5G framework should not have a great effect on the performance as all framework implementations should follow the 5G specifications and provide an acceptable model of a 5G network while using the default parameter values. These expectations, however, do not prove to be true in practice as different framework implementations approach the 5G modelling differently [30] and that can have a large impact on the overlying edge service orchestration behaviour.

Aiming to investigate how much the choice for a 5G framework influences the perceived edge service performances, especially in terms of end-to-end delay which is the main edge computing feature, we have opted to create a 5G+edge simulation model for a smart city vehicular scenario wherein we can cross-compare two different 5G frameworks discussed in details in the following section.

3. Simulating 5G Communication for Edge Computing Services

Establishing a 5G+edge ecosystem means bringing together the edge computing virtualised infrastructure with the mobile communications network infrastructure. The edge computing virtualised infrastructure is represented with a set of micro data centres that consist of a number of physical servers, that is, hosts, whose computing resources (CPU, RAM, disk, etc.) are virtualised forming a common pool of computing resources.

An edge orchestrator manages the edge services that run on the virtualised infrastructure and implements an intelligent placement algorithm that decides on where (in which micro data centre and on which host) to place a new instance of edge service. It also implements an intelligent migration algorithm that migrates an active edge service from one host to another according to the mobility of the users, thus ensuring optimal QoS at all times.

The access network infrastructure is implemented in the form of 5G base stations (BSs) colocated with the micro data centres that provide the coverage of the wireless network used as access network by the vehicles, that is, users of edge services (see Figure 1). These form the small cells which are the basis for the 5G implementation. These cells with their relative small coverage area can provide the ultrahigh density required in the next-generation systems.

[figure(s) omitted; refer to PDF]

The edge orchestrator makes decisions on placement and migration based on monitoring information gathered from the virtualised resources together with information from the 5G base stations, such as notifications about handoff events when a user moves from the coverage area of one base station into the coverage area of another base station. These handoff events trigger the orchestration algorithms that check the QoS status and decide if an edge service migration is necessary.

For the purposes of modelling and simulating this ecosystem for large-scale urban smart city vehicular scenarios, we have opted to use a combination of simulation tools, each of which will provide a certain set of the necessary features.

The urban vehicular scenarios in a realistic urban environment can be successfully modelled using SUMO [31] as it uses OpenStreetMap to set the scenario, including traffic rules, and it provides an engine that can model the random mobility of vehicles within the coverage area following the defined traffic rules. To model the full 5G network stack that provides mobile communication in the form of vehicle to infrastructure (V2I) by connecting the vehicles to the nearest base station, we have used OMNeT++ [11] network packet simulator. The edge servers and edge orchestration are modelled using an extension of CloudSim [32] that defines a network of micro data centres with a specification of the hosts in each micro data centre, together with the algorithms and additional components necessary to implement the intelligence of the edge orchestration and edge service lifecycle management. The full extension of CloudSim and the isolated analysis of its performances regarding the optimal management of edge services has been presented in [33]. Extending this environment with OMNeT++ in a verified simulation workflow in order to obtain the complete 5G+edge ecosystem has been presented in [23]. Note that although OMNeT++ has its own extensions for edge computing, such as ECSNeT++ [34], these do not support the possibility of migrating edge services which is a requirement for the implementation of the follow-me behaviour. Furthermore, the requirement to combine several simulation tools is considered to be more typical when transitioning from an edge only to a 5G+edge system because, as discussed previously, edge orchestration intelligence is typically designed in an isolated edge environment.

Here, we focus on the implications of choosing how the 5G network is modelled in OMNeT++ using a specific framework available for these purposes. Our main goal is to analyse if the change of the chosen 5G framework in OMNeT++ will significantly affect the edge orchestration layer in terms of performance, or, in other words, to gauge the influence of the choice of 5G simulation framework on the placement and migration optimisations in the edge layer.

5G networks are modelled in OMNeT++ as an extension of its SimuLTE module. As discussed in [30], there are four different options that enable the simulation of 5G vehicular scenarios based on SimuLTE [35]: OpenCV2X, Artery-C, 5G-Sim-V2I/N, and Simu5G. OpenCV2X is an open-source implementation of the 3GPP standard CV2X (Rel 14) Mode 4 [36] with its latest version being 1.4.1 from September 2021. Artery-C is used for cellular vehicle-to-anything communication based on LTE release 14 [37], and its latest changes on GitHub are from June 2021 with no official release. 5G-Sim-V2I/N implements the 5G User Plane according to 3GPP Release 15 for simulating use cases in the context of vehicle to infrastructure or network (V2I/N) with a latest version from October 2022 and release v0.4 [13]. Simu5G replaces the SimuLTE module of OMNeT++ and represents a 5G new radio (NR) and LTE/LTE-A user-plane simulation model [12] based on Release 16 with its latest version being 1.2.2 from April 2023. Given the newer release versions and official releases on GitHub, we have decided to explore in more depth the last two frameworks: 5G-Sim-V2I/N and Simu5G. In this way, we have one choice of a 5G implementation with specialised support for vehicular networks and a generic 5G implementation that can be used for any type of mobile communication. The implementation approach taken for each of them is explained in [30], while the goal of this paper is to cross-compare the results obtained when using one and then the other framework to analyse the performance of edge orchestration in a large-scale urban vehicular simulation.

With our research focus being the edge layer optimisation, we treat the 5G frameworks as a tool for the means of simulating the communication between the users (vehicles) and their respective edge services hosted on the edge servers colocated with the 5G base stations. Therefore, we have followed the basic instructions on installing the 5G frameworks as an OMNeT++ extension and used the provided default parameters set for a small cell-based urban scenario, which are specified on Table 1. Thus, we have managed to create two parallel simulation environments wherein the only differences in the simulation setup is due to the different choice for the 5G framework following a typical workflow used to analyse edge orchestration performance.

Table 1

Radio access network (RAN) parameters of OMNeT++ simulations.

RAN parameters	Value
NR channel model	3GPP38_901
Carrier frequency	2 GHz
Shadowing	Enabled
Fading	Enabled
gNodeB height	25 m
gNodeB transmission power	20 dBm
gNodeB antenna gain	5 dBi
gNodeB noise figure	5 dB
UE height	1.5 m
UE antenna gain	0 dBi
UE noise figure	7 dB
UE transmission power	26 dBm
Cable loss	2 dBi
Thermal noise	−104.5 dBm
Fading type	JAKES
Fading paths	6
Number of RB UL-DL	12
MAC scheduling UL-DL	PF
MAC pilot mode	Robust CQI

Considering the two chosen OMNeT++ 5G frameworks, 5G-Sim-V2I/N and Simu5G, there are some implementation details affecting to our simulation scenario that are worth mentioning and that are reflected on the modules’ stack represented in Figure 2. On one hand, based on the fact that 5G-Sim-V2I/N was released prior to Simu5G, it is implemented on top of the SimuLTE module that implements the LTE user-plane simulation model for INET and OMNeT++. Therefore, 5G-Sim-V2I/N updates SimuLTE parameters to 5G specifications, developing an NR library on top of the LTE library (more information about 5G implementation in framework 5G-Sim-V2I/N can be found in [13]). On the other hand, the Simu5G framework [12] includes a new implemented NR layer that completely replaces the SimuLTE library.

[figure(s) omitted; refer to PDF]

There are also some concerns about the development of the application level in one of the frameworks as all protocols of the network stack need to be implemented in the simulation tool in order to simulate the generation of application data. Therefore, all protocols involved during data transmission from and to the application server, which in both cases is based on the StandardHost library, and to and from the mobile user equipments, which in our simulations are defined as vehicles, need to be involved in the simulation process. However, we have observed that even though all layers are defined in both 5G RAN frameworks, there is a direct access to the TraCIScenarioManager included in the mobility libraries defined in the Veins manager module at the application level of the 5G-Sim-V2I/N framework in order to get information of the car involved in the communication. This shortcut is used instead of identifying the client applications from their own communication socket’s IDs, enabling the simulator to be much lighter as there is no need to keep on memory all related information of vehicles’ application layer during the simulation. However, this way of implementing the communication process at the application layer is a simplification that is distant from reality. It means a direct access to the simulation information of SUMO that is running in parallel to OMNeT++ simulation.

We show the differences of the layers’ implementation and communication process of both frameworks on Figure 3. In this way, the 5G-Sim-V2I/N framework defines a single socket to communicate the application layer of the simulated server with the equivalent layer of the vehicles that are moving through the coverage area instead of defining as many sockets as connections needed for the dense environment of vehicles. On the contrary, Simu5G implements the applications based on IApp of INET that defines socketIn and socketOut gates per application instance and vehicle. This also requires to define multiple sockets at the server’s application level.

[figure(s) omitted; refer to PDF]

To reach the objective of measuring the end-to-end delay in the system, we combine the measurements for delay in the access network from OMNeT++ with the delay in the core infrastructure network from the base station to the edge server obtained from CloudSim. In this way, we obtain the full detailed path from each vehicle to its corresponding edge service.

In Table 2, the versions of all necessary simulation tools including the operating system are provided. Please note that the 5G-Sim-V2I/N framework could not be run with a higher version of OMNeT++, and therefore, it is also based on lower versions of the supporting libraries such as Veins and INET. On the other hand, Simu5G is actively maintained, and thus, compatibility is ensured with higher versions of any related libraries. Both 5G implementations were tested with the same version of CloudSim using the same policies regarding placement and migration of edge services. In this way, the edge service performance can be cross-compared under equal conditions regarding the edge orchestration approach and techniques in place. The situation is similar with SUMO. The minor changes in the versions do not affect the nature of the vehicular mobility scenarios created under the same conditions using identical input parameters, traffic rules, and simulation area.

Table 2

Software libraries versions used in the comparison.

Parameter	Simu5G	5G-Sim-V2I/N
OMNeT++	6	5.6.2
SimuLTE	Simu5G 1.2.1	1.1.0
Veins	5.2	5.1
INET	4.4.0	3.7.0
SUMO	1.17.0	1.11.0
CloudSim	5.0.0	5.0.0
Ubuntu OS	20.0.4	20.0.4

In order to measure the access network delay that occurs in the 5G RAN in OMNeT++ simulations, as this is one of the most important parameters when it comes to edge computing performances, we observe a statistic measurement obtained from the Radio Link Control layer named rlcDelay. This measurement provides the communication delay that occurs between the user equipment and the serving base station. To obtain the end-to-end latency that describes the total delay between the user/vehicle and the edge service, to rlcDelay, we add the delay that is produced in the edge network simulated in CloudSim, that is, the delay between the base station and the host that runs the edge service that also takes into account the process of placement and/or migration of the edge services.

3.1. Description of the Simulation Environment

The simulation environment for the end-to-end delay analysis is defined as a realistic smart city traffic use case. The chosen dense urban area represents the main centre area of the city of Alicante, located on Costa Blanca, Spain. The detailed traffic-related description of this area which is roughly $2.1\text{km}\times 1.9\text{km}$ in size is provided using OpenStreetMap including information such as one-way streets, rights of passing, traffic lights, speed limits, pedestrian crossing, and roundabouts. Within this defined area, the SUMO traffic simulation generator has been used to generate a number of vehicles that dynamically enter/leave or start and stop moving within the area. The minimum distance that each vehicle must travel is set to 800 m, and the simulator ensures that after the initial warm up period, the density of the vehicles in the area is at a given steady state. The total simulation time has been set to 1800 s. The total number of vehicles per simulation is varied from 900 to 1400 thus allowing for analysis of scenarios with light to heavy, congested traffic.

The output of SUMO in the form of a set of vehicle routes is then fed to OMNeT++ with the networking scenario for each of the investigated 5G frameworks defined using identical settings regarding the layout of the 5G base stations and their interconnections. Using the Voronoi polygon approach [38], we have purposefully defined the location of nine 5G base stations throughout the urban simulation area not only choosing popular traffic hotspots but also ensuring that the coverage area across cells is as uniform as possible. The range of the distributed nine cells using this approach has measured to maximum 300 m corresponding to small-sized cells [39]. The simulation scenario configuration has been set to dense urban outdoor scenario. The antennas are interconnected using a full mesh topology with 10 GB point-to-point links as provided in the 5G framework simulation scenarios templates. In this way, using OMNeT++, the behaviour of the 5G RAN is completely captured by simulating the continuous communication between each active vehicle and its corresponding edge service located behind the 5G base stations.

The edge computing layer with server hosts located in nine different micro data centres and the edge orchestration logic in terms of allocation and migration algorithms were modelled using CloudSim. A typical fat-tree network topology has been defined for the edge network infrastructure. Using this topology, the tree root is connected to the cloud and used to host edge services whenever the edge server infrastructure is completely overloaded. The tree leaves are the nine micro data centres, each directly connected to a particular base station and providing a given number of edge server hosts (5, 7, 9, 11, or 13 hosts per micro data centre). Each edge host has four physical CPU cores and 8 GB of RAM as per a defined host profile in CloudSim. Multiple scenarios have been investigated in order to analyse the performances of the edge orchestration layer under different edge resource capacities (provided with the total of 45, 63, 81, 99, or 117 hosts in the edge computing layer).

Once active, every vehicle in the simulation requests an edge service from the edge orchestrator. The edge orchestrator will initialise and place the edge service instance according to the monitoring information regarding the availability of virtualised edge resources and the location (connected base station) of the requester. In the case where there are no sufficient resources in the edge layer, the edge service will be instantiated in the cloud. During the vehicle active period, it will move throughout the simulation area and the connection is passed from one base station to another, triggering handoff events. These triggers that are generated by OMNeT++ are picked up by the edge orchestrator in CloudSim. Upon receiving a handoff event information, the orchestrator initiates a migration of the corresponding edge service from one host to another aiming to always provide the minimum possible service latency. If there are no available resources to make the migration to the desired micro data centre, then the migration attempt will fail. The edge orchestrator will continue to reattempt the migration until it is successful or it is no longer needed because the vehicle has exited the simulation area. Both the placement and the migration logic are based on multiobjective parametrised function that aims to choose the best micro data centre for hosting each edge service under the current capacity restrictions. Within the micro data centre, the function’s goal is to ensure edge servers consolidation that consists in using the minimum number of servers to host the edge services aiming to provide power efficiency. The details of the inner workings of the edge orchestration and its validation in an isolated edge environment are available in [33].

The list of simulation parameters is summarised in Table 3. The full set of simulation scenarios has been run with each of the chosen 5G frameworks: Simu5G and 5G-Sim-V2I/N multiple times. The results obtained with each of the 5G frameworks are presented in the following section.

Table 3

Urban vehicular simulation scenario parameters.

Parameter	Value
Simulation location	Alicante city centre
Simulation area size	$2.121\times 1.871$ km2
Average no. of vehicles	[900, 1000, 1200, 1400]
No. of 5G base stations	9
5G scenario type	Dense urban outdoor
No. of micro data centres	9
Total no. of edge servers	[45, 63, 81, 99, 117]

4. Comparative Analysis

To understand how the choice of a 5G framework might influence the edge orchestration performances, specifically the end-to-end latency, we present a series of side-by-side comparisons of the results obtained while running the set of simulation scenarios described. We first focus on the macro results obtained, such as the average and median end-to-end latency and then attempt to investigate in depth why there are significant discrepancies in the obtained results. It is important to emphasise again that the results are obtained running identical simulation scenario setup in terms of input parameters as provided in Table 3 with the corresponding framework parameters for 5G dense urban outdoor scenarios detailed in Table 1. While examining the side-by-side results, please pay attention to the, sometimes, significant difference in the $y$-axis.

In Figure 4, the obtained average values for the end-to-end latency are presented as a function of the number of edge hosts. Shaded areas represent the 95% confidence intervals. The different data series represent the increasing number of vehicles in the simulation. It is evident that the use of the Simu5G framework yields to a decrease in the latency from 2 to 21 times in the most dense example. While both frameworks generally show the expected increase in delay as the density of vehicles rises, the 5G-Sim-V2I/N framework shows almost no dependency on the number of edge hosts, while Simu5G clearly demonstrates how the increasing resource capacity in the edge infrastructure influences the decrease in latency as the edge services can more easily be optimally allocated and migrated. To further investigate the origin of the gross discrepancy in the end-to-end delay, in Figure 5, only the average 5G access network delays are presented as a function of the number of vehicles. Again, the use of 5G-Sim-V2I/N framework leads to several times higher average delay, which increases to a staggering 23 times higher delay for the case of 1400 vehicles. This behaviour might even suggest an inherent instability of the 5G framework for high density scenarios, which is also observed in one of the simulations of the 900-car set and exhibits an extremely high access delay for the considered traffic intensity. Given the results, it can be concluded that the 5G-Sim-V2I/N results are far from the expectations given with the 5G specification that should support ultralow latency for high density scenarios. The Simu5G framework, on the other hand, handles the increasing traffic density very well with a relatively constant latency in the range between 5.9 and 6.5 ms with a maximum of 10% difference, which is well within the boundaries of the expected performance according to the 5G specification.

[figure(s) omitted; refer to PDF]

To understand better why there is such a significant difference in the average delay, we also compared the median end-to-end latency, as presented in Figure 6. Shaded areas are the 25th and 75th percentiles in this case. Cross-comparing the median values, the differences between the two 5G frameworks are now much smaller (max 6 ms vs. max 10 ms for 1400 vehicles) with Simu5G exhibiting a higher median delay for a smaller number of vehicles. We have summarised the relative differences of the performance metrics obtained by both extensions in Table 4. This leads to the conclusion that in the case of using the 5GSim-V2I/N framework, the 5G communication seems to exhibit some inherent problems when handling a number of extreme outliers that lead to the over 20 times increase of average delay.

[figure(s) omitted; refer to PDF]

Table 4

Relative differences of average performance metrics of (a) 5G-Sim-V2I/N versus (b) Simu5G obtained as $a-b/a$.

Parameter	Average	Median	Count
End to end delay	91.56%	−98.48%
Access network delay	91.89%	−46.38%
Edge network delay	−8.53%	−1.44%
Number of successful migrations			18.72%
Total number of migrations			−14.80%

Figure 7 provides more insight into the large discrepancy between the average and the median values. As it can be noticed (pay attention to the scale on the $y$-axis), the 5G-Sim-V2I/N framework sometimes tends to have very long delays peaking to almost 10,000 ms (10 s) for the particular use case. The presence of these extreme communications makes the framework “unstable” for use in highly mobile dense environments. The obtained edge performance results using this framework need to be carefully observed for the occurrence of such conditions. When using the Simu5G framework, although the 5G access network delay is still the most prominent, it does not reach over 40 ms for the chosen example, thus fully complying with the 5G specification. Further examination has led us to conclude that within the 5G-Sim-V2I/N framework, nodes tend to “prefer” a particular base station compared to others. This leads to overwhelming of the base station and extreme delays. This is again an unexpected behaviour of the framework as the base stations have been located so that they cover uniformly sized coverage areas which should effectively avoid overwhelming any of them, as is the case with Simu5G.

[figure(s) omitted; refer to PDF]

How the choice of 5G framework affects the specific events in the edge layer orchestration is presented in Figures 8 and 9 where the number of optimal and nonoptimal initial allocations and then subsequent migrations of edge services are depicted. It can be concluded that, as expected, the 5G network simulation model does not affect considerably the initial allocation of edge resources for the placement of edge services. However, it has an effect on the migration activities, as the use of the Simu5G framework reduces the number of migrations caused by handoff events by 25%. This leads to the conclusion that, although we have used the same access network setup for both OMNeT frameworks, there is a significant difference in the coverage area of the 5G base stations used in the two frameworks. This directly influences the number of handoffs that will be made during the simulation. Based on the obtained results, the Simu5G framework works with base stations with smaller coverage area and thus smaller overlaps between the 5G cells, enabling a lower number of handoffs and an increasingly uniform distribution of the vehicles connected per base station.

[figure(s) omitted; refer to PDF]

Overall, the side-by-side comparison leads to the conclusion that the choice of a 5G framework needs to be made very carefully with an in-depth understanding of the influence of all related variables. Our results show that in the case of this cross-comparison, Simu5G exhibits the expected behaviour much closer to the 5G specification, which is most probably due to the fact that it is more recent, follows a newer release, and it is done from scratch by replacing the SimuLTE OMNeT++ module instead of building on top of it. Our recommendations regarding the choice of a 5G framework that should be used for the purposes of edge orchestration performance analysis are as follows:

• Choose a 5G framework that adheres to a recent release of the 5G specification.

• Ensure that the 5G framework has been validated and verified for the particular use case scenarios, especially when it comes to device mobility and density.

• Follow the recommendations on setting up the 5G framework parameters for the particular use case scenario.

• Test the potential 5G framework and check if the access network performances (both average and median) are well within the expected ranges according to the 5G specification.

• Check the default parameters of the 5G scenario in order to ensure the most common setup wherein the edge orchestration is considered an extension of the service portfolio.

5. Conclusion

The development of full stack simulation environments to investigate all aspects and dependencies of different components that make up the 5G+edge computing ecosystem is of great importance for performance analysis. More complex, realistic simulation scenarios of dense urban environments can provide valuable insight into the strengths and weaknesses in proposed optimisation approaches while allowing insight into the way different aspects such as mobility and mobile communication affect the workings of the edge service management components. However, careful choice of simulation tools and frameworks must be made as the results obtained are heavily dependent on their completeness and accuracy.

In this paper, we have analysed the influence of two different 5G frameworks built as extensions of OMNeT++ and used for the analysis of urban dense vehicular scenarios. The macro scale obtained results in terms of end-to-end latency, and the more detailed investigation of the 5G RAN delay suggests that one must take great care when choosing a 5G framework and tune the parameters by following the recommended scenario setup. Our side-by-side comparison shows that the macro results may change from 2 to 21 times on average with the change of the chosen 5G framework, reaching even 23 times difference when comparing specific outputs. These changes are not isolated only in the RAN part, but they also significantly change the workload on the edge orchestrator by increasing the number of migration requests by 25%. In the 5G-Sim-V2I/N versus Simu5G cross-comparison, the Simu5G framework turns out to be the clear winner exhibiting behaviour and results that are well within the expected ranges according to the 5G specifications. Although the 5G-Sim-V2I/N framework is a specialised 5G framework developed purposefully for vehicular use cases, it has proven to demonstrate instability in highly dense urban scenarios with increased mobility.

Disclosure

A preprint of this paper has been previously published [40].

Author Contributions

Conceptualisation: S.F., C.B., K.G., and S.A.; investigation: S.F., C.B., and K.G.; methodology: S.F., K.G., and S.A.; software: S.F., K.G., P.J.R., and C.B.; validation: S.A., N.T., and C.B.; visualisations: K.G., P.J.R., and C.B.; writing (original draft preparation): C.B. and S.F.; writing (review and editing): K.G., N.T., S.A., and P.J.R.

Funding

The authors declare that there is no specific funding for this research.

References

[1] J. Pulkkinen, J. Jussila, A. Partanen, I. Trotskii, A. Laiho, "Smart mobility: services, platforms and ecosystems," Management Review, vol. 9 no. 9, pp. 15-24, DOI: 10.22215/timreview/1265, 2019.

[2] S. Sureshkumar, C. Agash, S. Ramya, R. Kaviyaraj, S. Elanchezhiyan, "Augmented reality with internet of things," 2021 International Conference on Artificial Intelligence and Smart Systems (ICAIS), pp. 1426-1430, DOI: 10.1109/ICAIS50930.2021.9395941, .

[3] K. Cao, Y. Liu, G. Meng, Q. Sun, "An overview on edge computing research," IEEE Access, vol. 8, pp. 85714-85728, DOI: 10.1109/ACCESS.2020.2991734, 2020.

[4] Q. V. Pham, F. Fang, V. N. Ha, M. J. Piran, M. le, L. B. le, W. J. Hwang, Z. Ding, "A survey of multi-access edge computing in 5G and beyond: fundamentals, technology integration, and state-of-the-art," IEEE Access, vol. 8, pp. 116974-117017, DOI: 10.1109/ACCESS.2020.3001277, 2020.

[5] B. Bajic, A. Rikalovic, N. Suzic, V. Piuri, "Industry 4.0 implementation challenges and opportunities: a managerial perspective," IEEE Systems Journal, vol. 15 no. 1, pp. 546-559, DOI: 10.1109/JSYST.2020.3023041, 2021.

[6] S. Deng, H. Zhao, W. Fang, J. Yin, S. Dustdar, A. Y. Zomaya, "Edge intelligence: the confluence of edge computing and artificial intelligence," IEEE Internet of Things Journal, vol. 7 no. 8, pp. 7457-7469, DOI: 10.1109/JIOT.2020.2984887, 2020.

[7] M. Liyanage, P. Porambage, A. Y. Ding, A. Kalla, "Driving forces for multi-access edge computing (MEC) IoT integration in 5G," ICT Express, vol. 7 no. 2, pp. 127-137, DOI: 10.1016/j.icte.2021.05.007, 2021.

[8] Z. Sharif, L. T. Jung, M. Ayaz, M. Yahya, S. Pitafi, "A taxonomy for resource management in edge computing, applications and future realms," 2022 International Conference on Digital Transformation and Intelligence (ICDI), pp. 46-52, DOI: 10.1109/ICDI57181.2022.10007397, .

[9] K. Velasquez, D. P. Abreu, M. Curado, E. Monteiro, "Resource orchestration in 5G and beyond: challenges and opportunities," Computer Communications, vol. 192, pp. 311-315, DOI: 10.1016/j.comcom.2022.06.019, 2022.

[10] H. Rahimi, Y. Picaud, K. D. Singh, G. Madhusudan, S. Costanzo, O. Boissier, "Design and simulation of a hybrid architecture for edge computing in 5G and beyond," IEEE Transactions on Computers, vol. 70 no. 8, pp. 1213-1224, DOI: 10.1109/TC.2021.3066579, 2021.

[11] A. Varga, "OMNeT++," Modeling and tools for network simulation, pp. 35-59, DOI: 10.1007/978-3-642-12331-3_3, 2010.

[12] G. Nardini, D. Sabella, G. Stea, P. Thakkar, A. Virdis, "Simu5G–an OMNeT++ library for end-to-end performance evaluation of 5G networks," IEEE Access, vol. 8, pp. 181176-181191, DOI: 10.1109/ACCESS.2020.3028550, 2020.

[13] T. Deinlein, R. German, A. Djanatliev, "5G-Sim-V2I/N: towards a simulation framework for the evaluation of 5G V2I/V2N use cases," 2020 European Conference on Networks and Communications (EuCNC),DOI: 10.1109/EuCNC48522.2020.9200949, .

[14] J. Nakazato, M. Nakamura, T. Yu, Z. Li, K. Maruta, G. K. Tran, K. Sakaguchi, "Market analysis of MEC-assisted beyond 5G ecosystem," IEEE Access, vol. 9,DOI: 10.1109/ACCESS.2021.3068839, 2021.

[15] Y. Wang, W. Ning, S. Zhang, H. Yu, H. Cen, S. Wang, "Architecture and key terminal technologies of 5G-based internet of vehicles," Computers and Electrical Engineering, vol. 95,DOI: 10.1016/j.compeleceng.2021.107430, 2021.

[16] F. Giust, V. Sciancalepore, D. Sabella, M. C. Filippou, S. Mangiante, W. Featherstone, D. Munaretto, "Multi-access edge computing: the driver behind the wheel of 5G-connected cars," IEEE Communications Standards Magazine, vol. 2 no. 3, pp. 66-73, DOI: 10.1109/MCOMSTD.2018.1800013, 2018.

[17] S. A. Abdel Hakeem, A. A. Hady, H. Kim, "5G-V2X: standardization, architecture, use cases, network-slicing, and edge-computing," Wireless Networks, vol. 26 no. 8, pp. 6015-6041, DOI: 10.1007/s11276-020-02419-8, 2020.

[18] 5GAA Automotive Association, "Technical report. C-V2X use cases and service level requirements Volume III," 2023. https://5gaa.org/c-v2x-use-cases-and-service-level-requirements-volume-iii/ https://5gaa.org/content/uploads/2023/01/5gaa-tr-c-v2x-use-cases-and-service-level-requirements-vol-iii.pdf

[19] T. Norp, "5G requirements and key performance indicators," Journal of ICT Standardization, vol. 6 no. 1, pp. 15-30, DOI: 10.13052/jicts2245-800X.612, 2018.

[20] A. Boualouache, T. Engel, "Federated learning-based scheme for detecting passive mobile attackers in 5g vehicular edge computing," Annals of Telecommunications, vol. 77 no. 3-4, pp. 201-220, DOI: 10.1007/s12243-021-00871-x, 2022.

[21] Y. Zhu, B. Mao, N. Kato, "A dynamic task scheduling strategy for multi-access edge computing in IRS-aided vehicular networks," IEEE Transactions on Emerging Topics in Computing, vol. 10 no. 4, pp. 1761-1771, DOI: 10.1109/TETC.2022.3153494, 2022.

[22] B. Li, P. Hou, K. Wang, Z. Peng, S. Jin, L. Niu, "Deployment of edge servers in 5G cellular networks," Transactions on Emerging Telecommunications Technologies, vol. 33 no. 8,DOI: 10.1002/ett.3937, 2022.

[23] K. Gilly, C. Bernad, P. J. Roig, S. Alcaraz, S. Filiposka, "End-to-end simulation environment for mobile edge computing," Simulation Modelling Practice and Theory, vol. 121,DOI: 10.1016/j.simpat.2022.102657, 2022.

[24] Y. Shu, F. Zhu, "An edge computing offloading mechanism for mobile peer sensing and network load weak balancing in 5G network," Journal of Ambient Intelligence and Humanized Computing, vol. 11 no. 2, pp. 503-510, DOI: 10.1007/s12652-018-0970-5, 2020.

[25] J. Wang, L. Wang, "A computing resource allocation optimization strategy for massive internet of health things devices considering privacy protection in cloud edge computing environment," Journal of Grid Computing, vol. 19 no. 2,DOI: 10.1007/s10723-021-09558-y, 2021.

[26] S. Yu, X. Chen, Z. Zhou, X. Gong, D. Wu, "When deep reinforcement learning meets federated learning: intelligent multitimescale resource management for multiaccess edge computing in 5G ultradense network," IEEE Internet of Things Journal, vol. 8 no. 4, pp. 2238-2251, DOI: 10.1109/JIOT.2020.3026589, 2021.

[27] E. Altman, T. Jiménez, "NS simulator for beginners," Synthesis Lectures on Communication Networks, vol. 5 no. 1,DOI: 10.1007/978-3-031-79251-9, 2012.

[28] N. R. O. Al-Rubaie, R. N. N. Kamel, R. M. Alshemari, "Simulating fog computing in OMNeT++," Bulletin of Electrical Engineering and Informatics., vol. 12 no. 2, pp. 979-986, DOI: 10.11591/eei.v12i2.4201, 2023.

[29] J. Zhang, H. Zhong, J. Cui, M. Tian, Y. Xu, L. Liu, "Edge computing-based privacy-preserving authentication framework and protocol for 5G-enabled vehicular networks," IEEE Transactions on Vehicular Technology, vol. 69 no. 7, pp. 7940-7954, DOI: 10.1109/TVT.2020.2994144, 2020.

[30] P. Barbecho Bautista, L. F. Urquiza-Aguiar, M. Aguilar Igartua, D. J. Reinoso-Chisaguano, M. C. Paredes, "An evaluation of OMNeT++-based V2X communication frameworks: on the path towards 5G-V2X simulations," Proceedings of the 24th International ACM Conference on Modeling, Analysis and Simulation of Wireless and Mobile Systems,DOI: 10.1145/3479239.3485723, .

[31] D. Krajzewicz, J. Erdmann, M. Behrisch, L. Bieker, "Recent development and applications of SUMO-simulation of urban mobility," International Journal on Advances in Systems and Measurements, vol. 5 no. 3&4, 2012.

[32] R. N. Calheiros, R. Ranjan, A. Beloglazov, C. A. De Rose, R. Buyya, "CloudSim: a toolkit for modeling and simulation of cloud computing environments and evaluation of resource provisioning algorithms," Software: Practice and Experience, vol. 41 no. 1, pp. 23-50, DOI: 10.1002/spe.995, 2011.

[33] S. Filiposka, A. Mishev, K. Gilly, "Community-based allocation and migration strategies for fog computing," 2018 IEEE Wireless Communications and Networking Conference (WCNC),DOI: 10.1109/WCNC.2018.8377095, .

[34] G. Amarasinghe, D. M. D. Assuncao, A. Harwood, S. Karunasekera, "ECSNeT++: a simulator for distributed stream processing on edge and cloud environments," Future Generation Computer Systems, vol. 111, pp. 401-418, DOI: 10.1016/j.future.2019.11.014, 2020.

[35] A. Virdis, G. Stea, G. Nardini, "SimuLTE-a modular system-level simulator for LTE/LTE-a networks based on OMNeT++," 2014 4th International Conference On Simulation And Modeling Methodologies, Technologies And Applications (SIMULTECH),DOI: 10.5220/0005040000590070, .

[36] B. McCarthy, A. O’Driscoll, "OpenCV2X mode 4: a simulation extension for cellular vehicular communication networks," 2019 IEEE 24th International Workshop on Computer Aided Modeling and Design of Communication Links and Networks (CAMAD),DOI: 10.1109/CAMAD.2019.8858436, .

[37] A. Hegde, A. Festag, "Artery-C: an OMNeT++ based discrete event simulation framework for cellular V2X," Proceedings of the 23rd International ACM Conference on Modeling, Analysis and Simulation of Wireless and Mobile Systems, pp. 47-51, DOI: 10.1145/3416010.3423240, .

[38] J. N. Portela, M. S. Alencar, "Cellular coverage map as a Voronoi diagram," The Journal of Computer Information Systems, vol. 23 no. 1, pp. 22-31, DOI: 10.14209/jcis.2008.3, 2008.

[39] J. H. Lee, J. S. Choi, S. C. Kim, "Cell coverage analysis of 28 GHz millimeter wave in urban microcell environment using 3-D ray tracing," IEEE Transactions on Antennas and Propagation, vol. 66 no. 3, pp. 1479-1487, DOI: 10.1109/TAP.2018.2797531, 2018.

[40] K. Gilly, C. Bernad, S. Filiposka, N. Thomas, P. J. Roig, S. Alcaraz, 5G Network Modelon Edge Orchestration Performance,DOI: 10.2139/ssrn.4752836, 2024.