On the Use of Containers for LoRaWAN Node Virtualization: Practice and Performance Evaluation

Abstract

This paper investigates the virtualization of LoRaWAN end nodes through Linux containers (LXCs) to improve scalability, flexibility, and resource management. By leveraging lightweight Docker-based virtualization, we break down the core functions of the LoRaWAN node, comprising the application, LoRaWAN, and LoRa layers, into modular containers. In this work, a fully virtualized end node is demonstrated. The obtainable performance is not only compared against the standard approach that leverages a LoRaWAN-compliant module but also against an emulated solution that mimics the desired functionalities purely in software. A controlled, uniform testbed, exploiting the capability of a virtual machine hypervisor to change the way the underlying hardware is abstracted to guest environments, is considered. Key metrics, including resource utilization and latency, are explicitly defined and evaluated. The results underscore the potential of container technologies to transform the deployment and management of communication solutions targeting Internet-of-Things (IoT) scenarios not only for the infrastructure but also for end devices, with implications for future advancements in wireless network virtualization.

1. Introduction

In recent years, wired networks have evolved to offer higher and higher flexibility, largely due to the shift from dedicated hardware solutions to software-driven functionalities. This trend has enhanced network performance and scalability, enabling network services to be provisioned, scaled, and optimized with remarkable efficiency. In the past, when dedicated hardware was used for implementing specific network functionalities, the update of an already deployed infrastructure required the introduction of new equipment. Supporting novel user application scenarios and/or protocols, which require disruptive network changes, resulted in a communication infrastructure redesign to comply with the new (standard) specifications. Software-Defined Networking (SDN) and Network Function Virtualization (NFV) technologies, just to mention a few, have played pivotal roles in this transformation by introducing programmability, flexibility, and cost-efficiency into the network infrastructure, for enabling the dynamic reconfiguration of available resources to meet diverse and changing demands [1].

Softwarization and virtualization of networks [2,3] have recently extended into the wireless domain, when challenges such as the efficient use of spectrum and the need for low-latency, high-throughput communication are prominent [4]. The integration of the previously mentioned SDN and NFV solutions into wireless networks has opened new possibilities for flexible network and traffic management. Among the many advancements, the Software-Defined Radio (SDR) concept has gained attention for its ability to virtualize radio functions, enabling real-time changes in radio access technology without requiring hardware modifications. In turn, these approaches permitted the so-called network disaggregation, which can be split into (i) vertical disaggregation, when the software part is decoupled from the hardware part, allowing multiple combinations to be used, and (ii) horizontal disaggregation, when network functionalities are implemented using more granular elements interconnected by explicitly designed interfaces. Such an approach is a fundamental pillar of the latest mobile communication generation, minimizing both the capital expenditure (CAPEX) in deploying new architectures and the operating expenditure (OPEX) related to the upgrade of network functionalities, which in turn results in higher revenues for operators and service providers.

However, despite leveraging different business models, IoT-like applications also demand efficient and scalable network solutions. In particular, Low-Power Wide-Area Networks (LPWANs) emerged as viable solutions that trade off coverage and battery lifetime against data rate and latency. LoRaWAN, based on the LoRa physical layer, merges these requirements with simple infrastructure devices (the gateways) and an innovative backend, where most of the infrastructure complexity is concentrated. Such a combination makes LoRaWAN ideal for a range of IoT applications, as confirmed by its wide adoption in both academia and industry. However, as the LoRaWAN ecosystem evolves, there is a growing demand for more dynamic and reconfigurable network solutions to accommodate the diverse requirements of connected devices. Traditional LoRaWAN architectures, characterized by their monolithic and hardware-dependent setups, are ripe for innovation through the application of software-centric approaches. As regards connectivity to the backend, it must be highlighted that the gateway is a relatively dumb and transparent device that is only in charge of en/decapsulating the user payload in/from a LoRa frame; i.e., it is a simple hardware component dependent on the backend software. On the other hand, virtualizing the entire functionality of the LoRaWAN end nodes could offer a promising solution that allows modular, scalable, and flexible deployment of LoRaWAN applications.

By integrating concepts from the literature and our previous research [5,6], this paper delves into the application of Linux container (LXC) [7] technology, specifically leveraging Docker lightweight virtualization, to disaggregate the LoRaWAN end node stacks into more manageable and dynamic components. Once LoRaWAN end-node functionalities are encapsulated within containers, the use of SDR allows for the complete virtualization of all the stack layers, significantly enhancing the deployment, scalability, and management of the overall application. However, despite the feasibility of this strategy having been somehow already demonstrated, a comprehensive characterization is still missing.

This work provides an answer to this research question. The novel and original contributions are as follows:

The full virtualization of a LoRaWAN end node, across all the stack layers, is proposed based on the use of LXCs and a low-cost SDR device;

A uniform environment for performance evaluation is described, leveraging virtualization of the underlying physical device to easily change memory and computational resource availability;

Multiple scenarios, based on different implementations of the same network functionality, are described and realized in real-world experiments to carry out rigorous testing and measuring and comparing key performance indicators such as latency and system resource utilization;

Analysis of the obtained results and insights about the benefits and challenges of the proposed architecture is carried out.

This paper is arranged as follows: First of all, in Section 2, the relevant literature about the topics of interest is addressed. In Section 3, the proposed approach is briefly described. In Section 4, some characteristics of the LoRaWAN technology are discussed. In Section 5, some key aspects of Docker virtualization technologies are explored. In Section 6, the experimental setup is detailed. In Section 7 and Section 8, the obtained results are discussed. For the sake of completeness, the target applications of the proposed approach, considering the obtained results, are listed in Section 9. Finally, some conclusions are drawn.

2. Related Works

In recent developments within IoT architectures, the integration of SDN and containerization technologies like Docker has shown promising results for improving network management and efficiency. As outlined in [8], SDN decouples networking control and data planes, providing a flexible management layer that enhances IoT operations. Concurrently, Docker enables portable, scalable application deployment across diverse IoT devices, streamlining network operations and facilitating rapid service deployment. This dual approach advances the scalability and flexibility required in heterogeneous IoT environments. The performance of virtual machines (VMs), containers, and containers on top of virtual machines is discussed in terms of CPU, memory, disk, and network key performance in [9], where it is reported that Docker containers on KVM-based systems are an optimal choice in terms of CPU usage.

Lightweight virtualization via Docker containers has demonstrated flexibility and efficiency and attracted attention from academics about its vast ecosystem and possible security issues, as in [10,11].

In the context of SDR for wireless networks, the potential to enhance flexibility and resource utilization is significant. In [12], the authors provide an overview of the network reconfigurability enabled by softwarization and virtualization in wireless domains. They discuss how technologies like NFV and SDN have been adapted for wireless communications, offering insights into their benefits for improved management and efficiency in network operations.

In the field of IoT communication, the utilization of SDR and LXC for the virtualization of network functionalities presents a transformative approach, which leverages container-based architectures to enhance the flexibility and scalability of network operations, allowing for the dynamic adaptation of network behaviors in real-time environments. The adoption of Docker containers facilitates the deployment and management of these network functions, simplifying the integration and scaling of LoRaWAN infrastructures across diverse application scenarios. Similar approaches are discussed in [13,14,15].

As regards virtualization techniques for Radio Access Networks (RANs), the use of both VMs and Docker containers has garnered considerable attention. Paper [16] provides a comprehensive comparison between these two approaches in the deployment of SDR functions. Their evaluation demonstrates Docker’s advantages in terms of reduced overhead, lower latency, and faster deployment times—supporting the ongoing shift toward more agile and efficient containerized network architectures.

In the domain of container migration and virtualization for wireless networks, [17] proposes a novel approach for the live migration of containerized Baseband Units (BBUs) in Cloud-RANs, focusing on ensuring network continuity, particularly in LoRaWAN and LTE networks. This work highlights the use of containerization technologies such as Docker, which enables the migration of BBUs with minimal downtime, thus preserving network stability and optimizing resource usage. Similarly, the study [18] explores the impact of Docker container virtualization on wireless mesh networks (WMNs) using SDN. Their study demonstrates how Docker containers can replace traditional, costly networking devices, optimizing WMNs by reducing operational and capital expenditures. By integrating SDN controllers like POX and Ryu, this work improves routing protocol redistribution, enhancing packet transfer, reducing latency, and improving overall network performance. Additionally, similar methodologies are explored in [19,20,21].

Particularly relevant to our study, paper [22] proposes a container-based architecture to virtualize LoRaWAN end nodes, integrating seamlessly with IP-based networks. Their architecture employs the Constrained Application Protocol (CoAP) to ensure lightweight communications, offering transparent integration with private and public LoRaWAN networks and enabling routing among end nodes, overcoming LoRaWAN’s inherent lack of IP compliance and inter-node communication. Their performance evaluations confirmed the effectiveness of deploying multiple CoAP servers per container, especially for constrained IoT devices such as Raspberry Pi. However, it must be noted that this document does not describe the entire softwarization of the end device.

Indeed, the latter concept emerged with the recent activities of some of the authors of the present work. The paper [23] discusses the potential of lightweight virtualization technologies, especially containerization, for end-to-end LoRaWAN network emulation; it highlights how containerizing both the node functionalities and backend services enables scalable and flexible deployments, facilitating realistic emulation of LoRaWAN networks with minimal hardware. As described in the germinal work [5], the integration of Docker-based microservices within the LoRaWAN architecture not only enhances operational agility but also promotes robust, scalable configurations for diverse IoT applications. This approach underpins the notion of disaggregation and decomposition of traditional hardware-centric LoRaWAN nodes, facilitating a transition to a more versatile and dynamic network infrastructure. A fully virtualized LoRaWAN end device is preliminarily discussed in [6], where the LoRa modulation is managed by a MATLAB (version R2023b) model streaming baseband samples towards/from an SDR.

In this work, the authors try to answer the open research questions about the resource requirements and scalability of full LoRaWAN node virtualization, by means of a comprehensive evaluation carried out using an explicitly designed experimental setup. In particular, the importance of an efficient implementation of software components is underlined by comparing the LoRa modulation implemented by a MATLAB-based model against a C-based one; the need for computational, storage, and communication resources is estimated in a controlled environment deploying containers within a virtual machine and changing the number of virtual processors and the memory amount accordingly. The obtainable performance for different configurations is evaluated by means of properly defined, time-related figures of merit.

3. The Proposed Approach

This paper focuses on evaluating the performance of a regular end device against a fully decomposed functionalities node. Starting from Figure 1, various disaggregation and decomposition strategies are detailed, showcasing how functionalities within end devices are segmented and containerized in different implementation scenarios. Each scenario uniquely handles the relevant stack layers, namely (a) the end-user application, (b) the LoRaWAN stack (including the management of the initial binding procedure and the payload (de)ciphering), and (c) the LoRa physical layer, utilizing either software virtualization or dedicated hardware. For the sake of clarity, a summary of the different scenarios is provided in Table 1; further details are presented in Section 4.

More specifically, based on the previous activities described in [5,6,23], the considered scenarios are as follows:

Standard: This is the reference scenario, representing a typical implementation of a LoRaWAN end node, leveraging a commercially available module. In this architecture, a clear separation between hardware-based communication functionalities and software-based application execution exists. Specifically, the LoRaWAN communication stack is fully handled by the hardware module, which also embeds the LoRa radio. Conversely, the end-user application, which is generally implemented by an external system/processor, is virtualized by means of an LXC.

Fully virtualized: This scenario represents the proposed, holistic virtualization strategy. Disaggregation and decomposition of functionalities are implemented across all the layers of the communication stack. In particular, this setup relies on an LXC per layer and takes advantage of the versatility of SDR technology to manage the LoRa radio modulation and demodulation functionalities (occurring at the baseband) through software instead of hardware. Fortunately, despite LoRa being a proprietary modulation scheme patented by Semtech, several reverse engineering (which is generally legal in the European Union if it is peformed for research and development purposes) efforts have been carried out in the past, e.g., as in [24,25].

Emulation: In this scenario, the end-node functionalities are still confined in LXCs, as in the fully virtualized case. However, the physical layer is completely abstracted out, demonstrating an implementation in which regular radio communications are omitted. Instead of utilizing LoRa radios to link the end nodes and the gateways, message forwarding towards the backend (and vice versa) is also containerized in a sort of virtual gateway.

As regards the latter scenario, its attractiveness as a (purely software) service for stress-testing real-world LoRaWAN infrastructure during the initial deployment or during normal operations must be underlined. Indeed, a variable, but controllable, load towards the backend can be easily injected, adding “emulated” devices, each one configured with a different update rate and user payload. Obviously, in this case, the impairments of the physical layer are not considered at all, since the LoRaWAN messages are directly conveyed to the backend using the virtual gateway.

4. LoRaWAN: Description and Definitions

Understanding the architecture of LoRaWAN is of paramount importance for exploring the virtualization of its network components. LoRaWAN stands for the Long-Range Wide-Area Network [26]. It is standardized by the LoRa alliance, which manages the specifications and certification procedures but relies on a proprietary radio technology, named LoRa and patented by Semtech. As suggested by the name, it is a protocol for long-range communications, but operating on low power. Accordingly, LoRaWAN is typically used for IoT-like applications, where the traffic is mainly from the field towards the users (the uplink direction), and devices have several constraints, in particular limited resources in terms of available power. This section delineates the core components of the LoRaWAN architecture: the end node, the gateway, and the backend. As a matter of fact, there is no network layer in the wireless part of a LoRaWAN infrastructure, due to the simple star-of-stars topology (i.e., there is no need for path discovery and routing). Two tiers can be identified:

The wireless tier, which includes the end devices, located a single hop away from the gateway (the center of the star);

The backend tier, further defined below, which concentrates most of the computational resources and it is often offered as a Platform as a Service (PaaS) in the cloud; it acts as a collector for the data from/to the gateways, thus forming another star consisting of wired (and wireless) links, generally based on the Internet (or Intranet for on-premise solutions).

4.1. End Node

Within the LoRaWAN architecture, the end node hosts the sensor or device endpoint that interacts with the physical world and transmits (refined) data to the final sink through one or more gateways.

Focusing on end-node protocol stack functionalities and services, it is possible to identify three primary components and layers:

Application Layer: The top layer or the end-user application, which generates traffic (uplink), handles specific user-defined tasks, and processes data before sending them over the network. It is important to stress that the LoRaWAN specifications do not define how data must be formatted, since only a generic binary user payload field is defined for the message; the implementer is in charge of this task. For this reason, real-world LoRaWAN backends support payload formatters. These code snippets interpret and process data going to and from end devices, e.g., converting binary payloads to human-readable values.

Medium Access Control Layer: LoRaWAN actually defines a Medium Access Control (MAC) protocol, which sits directly above the physical LoRa layer. LoRaWAN is responsible for coordinating the data transmission and reception processes across the wireless tier. A pure-ALOHA access scheme is adopted, a contention-based method that simplifies the transmission process; generally, robust mechanisms to handle potential collisions are not needed, because of the non-real-time nature of data transactions. The LoRaWAN layer ensures efficient use of the available communication medium by managing several critical tasks. These tasks include frame formatting, as well as adding headers and trailers to the frame for structuring the data for transmission; additionally, data rate selection is also performed, which adjusts the transmission rate to match network conditions and optimize energy consumption: the adaptive data rate (ADR) mechanism is managed by the backend. Furthermore, the LoRaWAN layer regulates the duty cycle, ensuring compliance with regional regulations on transmission times and preventing network congestion. The communication paradigm depends on the node class it belongs to, which dictates its transmission and reception patterns. In particular, (a) Class A must be supported by all nodes; communication is initiated by transmitting an uplink message to the backend after a relevant event occurs, and this is followed by two reception windows (often named RX1 and RX2) to receive a downlink message if needed, either for providing acknowledgment or additional data. (b) Class B introduces additional beacon messages for time synchronization, allowing for extra reception windows, which improves the potential throughput along the downlink direction. (c) Class C offers continuous listening for downlink messages, limiting the usage to devices that do not suffer from limited power availability (e.g., because they are main-powered or provided with some form of energy harvesting).

Physical Layer: The LoRa physical layer (PHY) is primarily responsible for the physical transmission and reception of radio signals. This layer utilizes the LoRa modulation technique, an implementation of Chirp Spread Spectrum (CSS) modulation, which is especially effective for long-range communications due to its robustness against interference and signal attenuation. Operating within the sub-GHz frequency spectrum, the LoRa PHY layer capitalizes on the properties of CSS modulation to enable communication over long distances while maintaining low power consumption. The modulation process within the LoRa PHY layer involves encoding a variable number of bits in a symbol that modulates into upchirps with a fixed bandwidth ($B_C$) and a variable time duration ($T_C$). The encoding efficiency is determined by the spreading factor (SF), which allows for $2^{SF}$ distinct initial chirp frequencies, thus accommodating the diverse number of bits per symbol. The SF can be set between 7 and 12, offering a versatile scale of data transmission rates and enhancing signal robustness against noise due to the additional processing gain.

Indeed, the relationship $2^{SF}=B_C·T_C$ must be satisfied, underpinning the balance among the communication range (the sensitivity), the data rate, and the noise immunity. This relationship highlights the flexibility of the LoRa modulation technique, which can be finely tuned to meet specific operational demands of IoT applications, ranging from throughput in the order of several $kbps$ down to hundreds of $bps$. For increasing message reliability, forward error correction mechanisms are applied; the redundant code bits are determined by the configurable coding rate parameter $CR\in \{4/5,..,4/8\}$. Additional insights into the intricacies of LoRa’s modulation strategy and its implications for network performance are discussed in further detail in [27].

Operating in unlicensed sub-GHz regions of the spectrum, regional parameters are defined that, when applied in the EU, result in $B_C=125 \mathrm{kHz}$ in the proximity of the 868 MHz region. In this case, in Europe, the duty cycle $DC$ is regulated by Section 4.3.3 of the ETSI EN300.220-2 standard [28], requiring $DC\in \{0.1,1,10\}\%$, depending on the actual channel.

As regards the practical implementation of LoRaWAN end nodes, the common approach involves the use of a microcontroller for running the upper layers of the protocol stack, including the application, which is interconnected by means of a serial link with a single-chip radio (e.g., one of the SX127X devices from Semtech, hosting a Serial Peripheral Interface—SPI—slave) [29]. Despite being effective and power-friendly, the adoption of an integrated transceiver limits configurability and customization, with limited parameter tuning (e.g., spreading factor, bandwidth, coding rate) and supporting only one channel at a time. When flexibility, experimentation, or advanced signal processing are required, an SDR becomes an intriguing choice.

4.2. Gateway

Gateways act as the communication bridge between end nodes and the backend server. They receive frames from multiple co-located end nodes and relay this information to the backend. Unlike end nodes, which operate on configurable but fixed channels and $SFs$, gateways are designed to concurrently receive traffic by leveraging different channels and $SFs$, thus minimizing the impact of the limited bandwidth and the simple MAC strategy. As regards the connectivity towards the backend, the specifications do not define how it is actually implemented; however, they are typically connected to an IP-based wired or wireless infrastructure. In particular, the $GW$ executes the so-called packet forwarder software, in charge of embedding/extracting LoRaWAN payloads into the payload of the protocol connecting with the backend. Currently, two main packet forwarders exist:

The UDP packet forwarder, also referred to as the legacy packet forwarder, is the original implementation, initially proposed by Semtech. It uses UDP as the transport protocol to transfer LoRaWAN packets. This solution is widely adopted in both industry and academia, and most manufacturers preinstall it on their gateways. The protocol is quite simple, facilitating the implementation, but also has some drawbacks. Notably, the link between the gateway and the backend is not encrypted, and the use of the unreliable UDP protocol can lead to message loss. However, these drawbacks are overcome using VPN tunneling, which ensures both privacy and reliability.

The LoRa Basics Station uses the WebSockets technology to communicate, natively supporting data encryption as it moves between the gateway and the network server. A two-step procedure is supported, in which a preliminary discovery procedure occurs before the actual connection with the gateway; additionally, a Configuration and Update Server (CUPS) is defined, which allows for easy management of device update and configuration.

It is also interesting to note that frontend solutions, like the Semtech SX125X devices, tailored for the design of LoRaWAN Gateways, allow us to lower the cost of the hardware for SDR-based LoRaWAN transceivers [30], which is relevant for medium- and large-scale deployments. Moreover, these devices also have limited supply currents, especially in standby and sleep modes, a key characteristic to limit overall power consumption.

4.3. Backend

The backend of a LoRaWAN network is the core of operations, comprising several key entities that manage the flow of data and ensure secure and efficient network functionality. These entities include the Network Server (NS), Application Server (AS), and Join Server (JS):

NS: The NS manages the routing of data across the network, ensuring efficient message delivery by deduplicating messages from multiple gateways and maintaining data integrity through network session keys. It also handles network responses such as acknowledgments and adaptive data rate commands. The NS is the endpoint of the packet forwarder connection.

AS: The AS interfaces with user applications, providing secure data delivery by decrypting messages using application-specific keys. It tailors data processing to meet the specific needs of various applications.

JS: The JS enhances network security by managing the authentication and initial device onboarding processes. It securely handles device root keys and session key generation, ensuring that each device’s identity is secure and unique.

Additional details can be found in [31].

5. Docker Virtualization

Virtualization technologies, especially lightweight ones, e.g., based on containers as for the Docker solution [32], have become central to the flexibility and scalability of IoT networks. An example is the implementation of the edge computing paradigm, as shown in [33].

Docker has become a cornerstone of modern containerization, providing a lightweight and efficient solution for deploying and managing applications across different environments. It enables operating system-level virtualization, where multiple isolated containers share the same kernel, but each has its own file system, processes, and network interfaces. This isolation allows Docker to run applications independently without the need for a fullfledged virtual machine, reducing both overhead and resource consumption. Docker also simplifies the orchestration of these containers. With Docker-Compose [34], users can define and manage multicontainer applications in a single configuration file, making deployment and scaling straightforward.

Docker includes an engine that changes an image (stored somewhere) into a running container. By leveraging an overlay filesystem, additional layers can be added on the initial image, so that the final image consists of many overimposed layers, which are efficiently stored on disk and can be easily removed if no longer needed.

Furthermore, Docker enhances the portability of applications. Containers encapsulate all dependencies required to run an application, ensuring that the application can be deployed across different environments—whether on a local machine, a cloud server, or a remote device—without worrying about compatibility issues. This portability is particularly valuable in IoT networks, where devices may run on different hardware and software configurations yet need to interact in a standardized way.

6. Experimental Setup

An explicitly designed testbed has been developed to evaluate the computational, storage, and connectivity demands of the proposed approach. Efforts have been made to ensure that experiments are carried out in well-defined conditions, to easily and effectively compare results. In particular, we have decided to leverage a virtual machine to host the containerized functionalities. Accordingly, the underlying hardware components and platform are completely abstracted, and consistency is ensured across all the testing conditions. Indeed, despite the fact that we can suffer from additional latencies due to such an additional abstraction layer, in this work, we are mainly interested in comparing the impact of different levels of memory and computational resource availability, rather than analyzing absolute performance. For this reason, a uniform, configurable environment is a main concern.

The main characteristics of the host and guest systems are summarized in Table 2. As shown in Figure 2, the setup is based on an ML350 Gen10 server from HPE; the processor is an Intel Xeon Silver 4208 with eight cores, and the system memory is 64 GB. It has to be stressed that the machine is not chosen according to the actual application requirements: as explained in the following, it is just a platform for running the VM instances to be compared under varying numbers of virtual CPUs and different amounts of available virtual memory. The host operating system is Ubuntu Server 20.04.6 LTS; as regards the hypervisor, the open-source Kernel-based Virtual Machine (KVM) has been considered, which is integrated in Linux from Kernel version 2.6.20. As a matter of fact, the KVM architecture mimics the Linux arrangement, which is based on (i) a native kernel module, switching the processor in a new guest state, executing the guest code, and leveraging the virtualization features offered by recent CPUs, and (ii) a virtual machine monitor, based on the Quick EMUlator (QEMU), which virtualizes the hardware devices and provides some virtual networking functions. An interesting feature is that the guest code is emulated in a Posix thread, which can be managed by means of regular Linux tools; when a two- or four-core guest is needed, two or four threads are created, each of them calling the KVM kernel module to start the execution. If enough real cores are available (or through scheduling otherwise), concurrent execution is managed by the normal Linux scheduler, keeping the codebase minimal. The KVM environment is configured by means of the Cockpit, a free and open-source frontend tool, furnishing an intuitive web interface that allows us to manage and modify the relevant system features and resources easily and intuitively.

In this way, the core setup remains consistent, but available (virtual) resources can be modified easily. Accordingly, different configurations can be devised to explore and evaluate significant aspects of the containerized LoRaWAN end device (see also Figure 3). The guest OS is Ubuntu 20.04.6 LTS; the VM is configured for using different numbers of virtual cores $vCPU\in \{1,3,5,7\}$ and virtual memory space $vMemory\in \{4,8,16\}$ GB; the virtual disk has a size of 215 GB.

As shown in Figure 2, the gateway used for communication is the Laird RG186 LoRaWAN gateway [35], which runs the Semtech UDP packet forwarder protocol, allowing communication between the end node and the backend services. In order to minimize impairments due to limited SDR transmitting power, noise, and interfering traffic, all the devices in the setup are positioned very close one each other. In terms of backend services, the system relies on The Things Stack (TTS) [36], a popular open-source LoRaWAN framework that provides all the necessary network management services.

Configurations

The experimental framework adopted for this study utilizes four distinct setups, as shown in Figure 3. Indeed, other than the setups mimicking the aforementioned Standard and Emulation scenarios, two other setups have been defined for implementing different “flavors” of the Fully Virtualized scenario, as better detailed in the following. In all the experiments, the message length is $L_{PL}=10$ B, and the message interval period is $T_{TX}=30$ s. The overall experiment duration is $T_{EXP}=3$ h. Other relevant settings are summarized in Table 3, resulting in the message duration of $T_{OA}=61.7$ ms.

Reference Setup: A top-level container hosts an application developed using Node-RED [37], an open-source flow-based development tool, leveraging JavaScript code for data manipulation. This Node-RED flow simulates the behavior of an end-user application that triggers the transmission of a fixed-length message based on a predefined schedule, possibly representing the uplink traffic of a sensor device in a real-world IoT-like application.

A second container serves as the bridge between the containerized Node-RED application and the LoRaWAN module, the RN2483 from Microchip, shown in Figure 2, which is connected through a virtual COM above a USB link. This container handles the serialization of the application payload into the AT-commands that are interpreted by the module.

The containers communicate publishing and subscribing MQTT messages exchanged across a (containerized) Mosquitto broker [38].

Fully Virtualized (MATLAB-based LoRa): In this setup, the application container implements the same functions as the standard setup. The user payload is processed by the next container, named “LoRaWAN”. This container runs a C script responsible for managing various LoRaWAN operations; other than adding headers and trailers, it performs tasks such as ciphering and deciphering the payload and appending the necessary headers and trailers to prepare the data for transmission using the LoRa radio.

Once the message is formatted, it is passed to the LoRa container, which hosts a MATLAB-based model of the LoRa modulation process. This model has been compiled into a standalone application and it is responsible for encoding (and decoding) symbols and performing the actual baseband (de)modulation. The MATLAB-based container outputs in-phase (I) and quadrature (Q) baseband samples, supposing a $Fs=2$ MSa/s sample rate. Each I/Q sample is a 2B-long signed integer value.

The subsequent container, named Serializer, is responsible for streaming the I/Q components to the SDR device for actual signal transmission. The low-cost ADALM-PLUTO [39] from Analog Devices is adopted, capable of operating up to 6 GHz with a maximum real-time bandwidth of 20 MHz imposed by the maximum sampling frequency of $F{s}_{MAX}=61.44$ MSa/s. It is a cost-effective experimentation platform that leverages the Analog Devices AD9363 transceiver and the Xilinx Zynq Z-7010 System On Chip (hosting both an FPGA for pre-processing and an ARM dual-core CPU for supervision) capable of streaming the I/Q samples from the ADC and towards the DAC, with a 12-bit vertical resolution. This SDR is supported by the Linux Industrial Input/Output (IIO) driver, which is part of the Linux kernel. The ADALM-PLUTO is connected to the system via a USB 2.0 interface, which imposes a throughput limit of several MSa/s, adequate for the low-bandwidth requirements of LoRa. As also described in [6], a fixed buffer of 10 MSa stores the samples (i.e., 40 MB are transferred each time to the SDR), enough to contain the actual message. Indeed, since the actual sample rate is limited to $Fs=2$ MSa/s and $T_{OA}=61.7$ ms, the number of samples per message is in the order of $N_{Sa}=FS·T_{OA}=123.4$ kSa.

All containers communicate via an MQTT broker, except for the Serializer, which sends the I/Q streams to the SDR using a TCP/IP client–server flow.

Fully Virtualized (C-based LoRa): In this setup, our proposed method adopts a virtualization strategy similar to that of the MATLAB-based SDR driven scenario, with a distinct implementation approach for the LoRa layer. In this case, the LoRa model is developed using the C programming language, known for its efficiency and control over system resources, and it is executed by its own dedicated container.

Emulation: This setup demonstrates the flexibility of the proposed approach. As already described in Section 3, this configuration does not involve any actual LoRa transmissions. Instead, communication is emulated through a virtualized component known as the Virtual Gateway (VGW). The VGW collects the Application container output, which is immediately forwarded to the backend by means of the implemented packet forwarder functionality.

Table 4 shows the network traffic for each container in the different experimental scenarios, with values for incoming (I) and outgoing (O) traffic, obtained by means of the docker stats –no-stream command. These values represent the amount of data the container has sent and received over its network interface, calculated as the incremental changes in the actual Net I and Net O metrics over time. This information provides a clear picture of data movement in kilobytes (kB) and megabytes (MB), highlighting the network load and communication patterns within the virtualized environments used in this study. These statistics are consistent with the expected traffic previously described. As expected, there is no difference in the Application traffic in all cases; similarly, both the MATLAB- and the C-based LoRa containers for the Fully Virtualized configurations provide the same traffic levels.

7. Results—Resource-Related Metrics

This section presents and discusses the results of the experiments carried out using the previously introduced Fully Virtualized configurations; the focus is on resource-related metrics.

To compare the resource usage across containers, we employed a monitoring stack comprising Prometheus and Grafana [40], well-established tools for collecting and visualizing time series. This setup provides detailed insights into the resource consumption of different system components in real time. For containerlevel resource monitoring, we used cAdvisor (Container Advisor) [41] to expose container-specific metrics, while node-exporter [42] offered system-level metrics.

In addition to the monitoring stack, we also used the docker stats –no-stream commandline tool, which provides real-time performance data for running containers. This tool was particularly useful for tracking the Process IDs (PIDs) and Network Traffic I/O for each container.

Resource Usage Comparison: MATLAB- vs. C-Based LoRa

Both the C-based and MATLAB-based LoRa containers are the most demanding, since algorithms for the baseband (de)modulation process must be executed. Figure 4 illustrates the CPU usage of the LoRa containers as a percentage of the total computational capacity of the host machine. To monitor the container’s CPU usage, cAdvisor was employed to expose container-level metrics, container_cpu_usage_seconds_total, while node-exporter provided system-level metrics, machine_cpu_cores. These metrics were scraped by Prometheus every 15 s and queried using the following PromQL expression:

The numerator (rate(container_cpu_usage_seconds_total[5m])) calculates the rate of change in CPU time consumed by the LoRa containers, averaged over a 5 min long window. The denominator (avg(machine_cpu_cores)) represents the total number of vCPU cores available on the host machine; the result is multiplied by 100 to express the CPU usage as a percentage.

It is interesting to highlight the relevant overhead introduced by the MATLAB runtime environment, which, differently from the C-based implementation, exhibits a huge vCPU usage in the startup phase and is an order of magnitude more demanding than the plain C counterpart. As regards the number of the vCPUs, a relevant impact moving from one to three vCPUs (the usage is almost divided by a factor of two) can be shown, which is less evident when moving from five to seven vCPUs.

Table 5 presents the distribution of PIDs between different vCPU cores for each container in the various configurations. A significant observation is the contrast in PID utilization within the LoRa containers in different scenarios. In the Fully Virtualized (C-LoRa) scenario, where the LoRa container is implemented using C, the container, as expected, consistently uses only a single process. In contrast, in the Fully Virtualized (M-LoRa) configuration, where the LoRa container is developed using MATLAB, the number of processes involved is substantially higher and it increases as more vCPU cores are allocated to the system. This behavior reflects the usage of the MATLAB native signal processing functions for implementing the algorithms of interest.

Figure 5 illustrates the memory usage of the LoRa containers, represented as a percentage of the total memory available on the guest machine (changing in the set {4,8,16}GB), when the number of available vCPU cores is limited to one core. These metrics were scraped by Prometheus every 15 s and queried using the following PromQL query:

As expected, memory usage scales almost linearly with the overall amount. Also, in this case, the C-based LoRa container generally uses less memory and shows more stable usage compared to the MATLAB-based one.

As a concluding remark, it is possible to state that these stress the importance of an effective implementation and allow us to figure out the feasibility of this fully virtualized LoRaWAN node on an embedded platform.

8. Results—Time-Related Metrics

After highlighting the need for resources, in this section, the results of the experiments carried out using the previously introduced configurations are presented and discussed, focusing on time-related metrics.

8.1. Time Synchronization

Given the distributed nature of the system, both the local devices and the backend need to be synchronized to a common reference time, such as UTC, to assess time-related performance. The LoRaWAN backend is inherently synchronized with UTC. Therefore, the local PC, which hosts the containers, runs the Network Time Protocol (NTP) service [43], managed by chrony [44], to periodically adjust the local time, based on measurements of offset and delay from previous steps.

The synchronization uncertainty is calculated using the formula $u_{sn}=\sqrt{{\mu }_{sn}^2+{\sigma }_{sn}^2}$, where ${\mu }_{sn}$ is the average uncompensated systematic error introduced by the operating system (i.e., the average offset), and ${\sigma }_{sn}$ represents the random error (i.e., the standard deviation of the offset). The system logs were collected over a three-hour period to measure the residual time offset after compensation, yielding $u_{sn}\approx 3.47$ ms.

It is important to note that when calculating statistics based on differences in timestamps from the same clock, the systematic errors tend to cancel out. As a result, $u_{sn}={\sigma }_{sn}\approx 0.5$ ms. A similar level of uncertainty can be attributed to timestamping accuracy, as discussed in prior studies [45]. It is also expected that the synchronization and timestamping accuracy of the backend servers do not exceed these values.

8.2. Time-Related Metrics

To evaluate the performance of each scenario from the time domain point of view, we tracked the uplink message path by capturing a series of timestamps at each stage. Within the virtualized environment, each container recorded the time at which a message entered and exited. The relevant figures of merit for direct comparison were obtained by computing the time taken at each step for the different configurations. An additional container, named Logger and specifically tasked with this function, managed the publication of these timestamps on the MQTT database. In the backend sections, timestamp collection was facilitated by the TTS platform, which automatically timestamps the reception of the uplink message at the GW, NS, and AS. Therefore, other figures of merit can be evaluated for the characterization of the complete path from the field to the end user, e.g., for direct comparison with other setups discussed in the literature.

Figure 6 illustrates the duration of each process in the different experimental configurations, identified by the subscript $Conf\in \{S,FM,FC,E\}$ for Standard, Fully Virtualized (M-LoRa), Fully Virtualized (C-LoRa), and Emulation, respectively. The timestamps $T_{Confi}$, where the subscript i is nothing but the progressive index, as shown in Figure 6, permit the computation of duration intervals $D_{Confi}$, better detailed in the following for the different configurations. In particular, in all configurations, the Node Delay $D_{ConfND}$, representing the time spent within the virtualized environment, and the End-to-End Delay $D_{ConfEE}$, representing the overall time needed to complete a roundtrip LoRaWAN transaction, are computed as well.

Standard Configuration:

$D_{S1}=T_{S2}-T_{S1}$, representing the processing time taken by the Serializer service;

–. $T_{S1}$ marks the moment when the Serializer service first receives a request for an uplink communication.

–. $T_{S2}$ corresponds to the instant when the Serializer service has completed processing the request and forwards the uplink payload to the RN2483 module.

$D_{S2}=T_{S3}-T_{S2}$, representing the time taken for the message to be gathered by the gateway;

–. $T_{S3}$ is defined as the instant when the physical gateway (GW) acknowledges the reception of the new uplink communication originating from the module.

$D_{S3}=T_{S4}-T_{S3}$, representing the time taken for the message to travel from the gateway to the NS;

–. $T_{S4}$ identifies the moment the NS is notified of the new uplink communication.

$D_{S4}=T_{S5}-T_{S4}$, representing the time taken for the message to travel from the NS to the AS;

–. $T_{S5}$ identifies the moment the AS is notified of the new uplink communication.

$D_{SEE}=T_{S5}-T_{S1}$ is the End-to-End delay;

$D_{SND}=T_{S2}-T_{S1}$ is the Node delay;

Fully Virtualized (M-LoRa) Configuration:

$D_{FM1}=T_{FM2}-T_{FM1}$, representing the time spent within the containerized service for managing the LoRaWAN layer, from receiving the uplink request to relaying the processed payload to the next layer.

–. $T_{FM1}$ marks the moment when the LoRaWAN service initially receives a request for a new uplink communication.

–. $T_{FM2}$ corresponds to the instant when the LoRaWAN service has processed the request and relays the uplink payload to the subsequent lower layer.

$D_{FM2}=T_{FM3}-T_{FM2}$, representing the time spent in the containerized service for managing the LoRa layer, from receiving the payload to generating the I/Q samples for transmission.

–. $T_{FM3}$ is the instant when the LoRa service has completed the LoRa encoding and baseband modulation, meaning the actual I/Q discrete samples have been generated.

$D_{FM3}=T_{FM4}-T_{FM3}$, representing the time spent in the containerized service for setting up the link between the Serializer service and the SDR, ensuring the system is ready to transmit the samples.

–. $T_{FM4}$ denotes the point at which the Serializer service has successfully established the connection with the SDR and is ready to stream the samples.

$D_{FM4}=T_{FM5}-T_{FM4}$, representing the time spent in the containerized service for streaming the I/Q samples to the SDR and propagating the uplink signal through the air until it is received by the gateway.

–. $T_{FM5}$ is defined as the instant when the physical GW acknowledges the reception of the new uplink communication originating from the SDR.

$D_{FM5}=T_{FM6}-T_{FM5}$, representing the additional time required for the uplink communication to reach the NS, capturing the time taken for the message to travel from the gateway to the NS.

–. $T_{FM6}$ identifies the moment the NS is notified of the new uplink communication.

$D_{FM6}=T_{FM7}-T_{FM6}$, representing the additional time required for the uplink communication to travel from the NS to the AS.

–. $T_{FM7}$ identifies the moment the AS is notified of the new uplink communication.

$D_{FMEE}=T_{FM7}-T_{S1}$ is the End-to-End delay;

$D_{FMND}=T_{FM4}-T_{S1}$ is the Node delay;

Fully Virtualized (C-LoRa) Configuration:

Durations, $D_{FCi}$, $D_{FCEE}$, $D_{FCND}$, and time points, $T_i$, in this configuration are analogous to those in the previous one, the Fully Virtualized (M-LoRa) configuration, with the only difference being their naming convention.

Emulation Configuration:

$D_{E1}=T_{E2}-T_{E1}$, representing the time spent within the containerized service for managing the LoRaWAN layer, from receiving the uplink request to relaying the processed payload to the next layer.

–. $T_{E1}$ marks the moment when the LoRaWAN service initially receives a request for a new uplink communication.

–. $T_{E2}$ corresponds to the instant when the LoRaWAN service has processed the request and relays the uplink payload to the subsequent lower layer.

$D_{E2}=T_{E3}-T_{E2}$, representing the time spent within the VGW container, from receiving the uplink payload to preparing it for transmission to the backend.

–. $T_{E3}$ is the instant when the VGW has processed the payload and is ready to send it to the backend.

$D_{E3}=T_{E4}-T_{E3}$, representing the time taken for the uplink communication to travel from the VGW to the backend.

–. $T_{E4}$ is the instant when the backend acknowledges the reception of the new uplink communication originating from the VGW.

$D_{E4}=T_{E5}-T_{E4}$, representing the time taken for the uplink communication to forward from the backend to the NS.

–. $T_{E5}$ identifies the instant when the NS is notified of the new uplink communication.

$D_{E5}=T_{E6}-T_{E5}$, representing the additional time required for the uplink communication to travel from the NS to the AS.

–. $T_{E6}$ identifies the instant when the AS is notified of the new uplink communication.

$D_{EEE}=T_{E6}-T_{E1}$ is the End-to-End delay;

$D_{END}=T_{E3}-T_{E1}$ is the Node delay;

8.3. Performance Overview

Table 6, Table 7, Table 8 and Table 9 provide a summary of statistics for the time-related metrics, organized by different CPU (virtual) cores, with 8 GiB of memory allocated in all cases. Each table presents an overview of the performance metrics for specific intervals measured in milliseconds, allowing for an assessment of efficiency at each stage of the uplink message path across the virtualized environments and the overall path.

Additionally, Figure 7, Figure 8, Figure 9 and Figure 10 illustrate the End-to-End and the Node uplink delays for different vCPU cores for all the configurations. All results presented in these figures are filtered to the 95th percentile in order to better highlight variations imputable to an increasing amount of computational resources, excluding upper outliers.

Obviously, metrics depending on the cloud-hosted backend are not affected by the changing number of vCPUs. On the other hand, it is also interesting to highlight that the time spent for processing the (de)modulation is not affected as well.

9. Discussion

Any form of virtualization adds an additional software layer that translates into overhead in terms of computational and storage requirements. For this reason, the proposed approach is not intended to substitute the use of full custom implementation of a LoRaWAN end device, which leverages the use of components explicitly designed to minimize the software and hardware footprints. On the other hand, these solutions are rigid and do not allow us to fully exploit the possibilities offered by the “white spaces” of LoRaWAN specifications. Radio virtualization enhances flexibility by enabling multiple (maybe heterogeneous) virtual radios to coexist on the same hardware. A well-accepted method for virtualizing radio devices involves the adoption of virtual machines and containers to multiplex software radio implementations over generic multipurpose radio hardware, as reported in [16].

For instance, regular LoRa radio transceivers, as the ones in the Semtech SX126X family, only permit the use of a single channel and SF at a time, preventing the implementation of interference/noise countermeasures based on the retransmission of the same message on multiple channels and/or multiple SFs. Indeed, efforts have been made to exploit SDR to support multi-user detection in LoRa radios based on successive interference cancellation and for integrating LoRa with ARQ error-control protocols and rate-less packet-level coding as in [24]. Software-defined transceivers also pave the way for an improved version of the LoRa scheme, e.g., as suggested in [46]. Another possible target application scenario is the addition of communication interfaces in systems where computational and storage resources are already available (as edge devices); in this case, the use of a full virtual approach would allow us to easily switch from one protocol to another without the need of changing the underlying hardware [47].

Another point that must be stressed is that recent research has focused on porting the containerization paradigm into constrained devices as well, allowing the integration of low-power IoT devices in DevOps workflows [48]. In this case, the efficient management of containers is also a main concern. For instance, the automotive industry is moving towards a software-defined model, which requires cars to host and execute a large number of applications with limited resources (in terms of limited memory, CPU, and energy); the solution could be the containerization of the applications, and energy and cost-efficiency could be reached exploiting the smart selection of containers to turn off based on container priorities and available resources, as suggested in [49].

A final comment is about the power consumption. The additional overhead introduced by virtualization not only requires more powerful hardware but also results in higher power consumption, if compared against a customized embedded solution. Additionally, SDRs are generally more power-hungry than regular hardwired transceivers, since the flexibility they offer also depends on higher bandwidth and sample rates, which adversely affect the supply current. For all these reasons, the proposed approach is suitable for applications where power supply is not an issue.

10. Conclusions

Our findings underline the substantial potential of container technologies to change the way nodes of wireless networks are implemented, even for relatively simple protocol stacks such as LoRaWAN. The transition to a fully virtualized design—where not only all layers of the LoRaWAN stack are disaggregated into containers, but also the radio is implemented by an SDR—proves particularly effective in enabling dynamic reconfiguration and efficient resource management, making it a promising solution for large-scale and flexible IoT deployments. This work not only confirms the feasibility of the proposed approach but also objectively quantifies the importance of an effective implementation, comparing the MATLAB- and C-based versions of the LoRa baseband. As expected, the C-based implementation provides better resource utilization, stability, and scalability, features of paramount importance in real-world applications.

While the results from this study are promising, the proposed architecture would benefit from further validation in field-based, dynamic environments to assess its performance when embedded platforms are considered. Additionally, integrating the containerized approach with other wireless communication technologies could provide insights into the broader applicability of this method in diverse IoT applications. This work paves the way for future innovations in wireless network virtualization, demonstrating the critical role of containerization in transforming IoT infrastructure management. Additional experiments have been already planned for a large-scale deployment to evaluate the impact of wireless channel impairments, which we currently intentionally limit by operating in a lab-grade, controlled environment.

Author Contributions

Conceptualization and methodology, H.K. and E.S.; software, H.K.; formal analysis, E.S. and P.F.; investigation and validation, H.K.; data curation, H.K. and E.S.; writing, all the authors contributed to writing—review and editing; visualization, H.K; supervision, A.F.; project administration and funding acquisition, A.F. All authors have read and agreed to the published version of the manuscript.

Data Availability Statement

The data presented in this study are available in the article.

Conflicts of Interest

The authors declare no conflicts of interest.

Abbreviations

The following abbreviations are used in this manuscript:

IoT	Internet of Things
LPWANs	Low-Power Wide-Area Networks
LXC	Linux Container
VMs	Virtual Machines
RAN	Radio Access Network
SDR	Software-Defined Radio
SDN	Software-Defined Networking
NFV	Network Function Virtualization
PaaS	Platform as a Service
CSS	Chirp Spread Spectrum
PHY	Physical Layer
MAC	Medium Access Control
ADR	Adaptive Data Rate
NS	Network Server
AS	Application Server
JS	Join Server
KVM	Kernel-based Virtual Machine
QEMU	Quick Emulator
IIO	Industrial Input/Output
GW	Gateway
VGW	Virtual Gateway
NTP	Network Time Protocol
PromQL	The Prometheus Query Language
PIDs	Process IDs

Footnotes

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Figures and Tables

Figure 1 Overview of functional split scenarios in LoRaWAN node implementations.

Figure 2 The experimental setup including the LoRaWAN GW, the SDR, the module, and the host.

Figure 3 Experimental setups for virtualized LoRaWAN environments, showcasing different communication and network configurations across scenarios.

Figure 4 CPU usage of the C- and MATLAB-based LoRa containers as a percentage of the total capacity.

Figure 5 Memory usage of the LoRa containers as a percentage of total guest memory capacity.

Figure 6 Time-related metrics for each stage of the uplink message path across different experimental configurations.

Figure 7 End-to-End and Node delays for different vCPU cores in the Standard configuration.

Figure 8 End-to-End and Node delays for different vCPU cores in the Fully Virtualized (M-LoRa) configuration.

Figure 9 End-to-End and Node delays for different vCPU cores in the Fully Virtualized (C-LoRa) configuration.

Figure 10 End-to-End and Node delays for different vCPU cores in the Emulation configuration.

Table 1

Overview of the different experimental scenarios.

Scenario	Application Layer	LoRaWAN/DLL Layer	Physical Layer	Gateway	NodeDelay
Reference	Containerized	HW Module	HW Module	Real	+++
Fully Virtualized	Containerized	Containerized	SDR	Real	+
Emulation	Containerized	Containerized	Absent	Virtual	++

Note: The symbols (+, ++, +++) represent qualitative assessments of the NodeDelay impact, from largest (+, the worst) to shortest (+++, the best) delay values.

Table 2

Hardware and platform specification of the experimental setup.

Host Processor	Intel(R) Xeon(R) Silver 4208 CPU @ 2.10 GHz (Cascadelake)
Host OS	Ubuntu Server 20.04.6 LTS
Guest vCPU Core (s)	{1, 3, 5, 7} Cores
Guest OS	Ubuntu 20.04.6 LTS
Guest vMemory	{4, 8, 16} GB
Guest vHardDisk	215 GB
Platform	Docker 27.4.0, Docker compose 2.20.0, MATLAB R2023b
Monitoring Tools	cAdvisor v0.39.3, node-exporter v1.8.2, Prometheus v3.1.0, Grafana v11.4.0

Table 3

Configuration parameters for LoRaWAN end-node setup.

Stack	Parameter	Assumption
Application	Payload Size	$L_{PL}=10$ B
	Message Interval	$T_{TX}=30$ s
	Experimental Period	$T_{EXP}=3$ h
LoRaWAN	Version	LoRaWAN Specification 1.0.4
	Class	A
LoRa	Spreading Factor	$SF=SF7$
	Coding Rate	$CR=4/5$
	Band	868 MHz
	Bandwidth	$B_C=125$ kHz
	Airtime	$T_{OA}=61.7$ ms

Table 4

Network traffic.

Scenarios	Container	$\Delta$ Network Traffic
		I	O
Standard	Application	0	0.1 kB
	Serializer	0.4 kB	1.2 kB
Fully Virtualized (M-LoRa)	Application	0	0.1 kB
	LoRaWAN	0.4 kB	0.6 kB
	LoRa	1.7 kB	0.5 MB
	Serializer	1.3 MB	40 MB
Fully Virtualized (C-LoRa)	Application	0	0.1 kB
	LoRaWAN	0.4 kB	0.6 kB
	LoRa	1.7 kB	0.5 MB
	Serializer	1.3 MB	40 MB
Emulation	Application	0	0.1 kB
	LoRaWAN	0.4 kB	0.6 kB
	Virtual GW	0.6 kB	1.7 kB

Table 5

Distribution of PIDs across CPU cores for containers in different experimental scenarios.

Scenarios	Container	CPU Cores
		# 1	# 3	# 5	# 7
Standard	Application	21	21	21	21
	Serializer	1	1	1	1
Emulation	Application	21	21	21	21
	LoRaWAN	3	3	3	3
	Virtual GW	5	5	5	5
Fully Virtualized (C-LoRa)	Application	21	21	21	21
	LoRaWAN	3	3	3	3
	LoRa	1	1	1	1
	Serializer	5	5	5	5
Fully Virtualized (M-LoRa)	Application	21	21	21	21
	LoRaWAN	3	3	3	3
	LoRa	25	31	36	39
	Serializer	5	5	5	5

Table 6

Summary of time-related statistics for the Standard configuration.

CPU#	Statistic (ms)	${\mathit{D}}_{\mathit{S}1}$	${\mathit{D}}_{\mathit{S}2}$	${\mathit{D}}_{\mathit{S}3}$	${\mathit{D}}_{\mathit{S}4}$	End-to-End
CPU1	Mean	1.103	138.496	12.461	219.366	371.425
	Std	0.400	10.478	9.271	22.767	23.785
	Median	1.071	137.309	19.168	208.269	363.636
	Min	0.304	120.324	0.556	202.886	345.685
	Max	6.838	159.625	30.921	359.219	516.022
	5th Pctl	0.484	123.095	0.642	203.646	347.992
	95th Pctl	1.497	156.833	20.509	260.058	416.620
CPU3	Mean	0.376	138.174	12.210	216.399	367.159
	Std	0.092	10.574	8.907	18.713	19.706
	Median	0.362	137.456	18.889	207.070	361.409
	Min	0.140	120.250	0.560	203.083	343.897
	Max	1.295	161.385	26.186	298.126	451.522
	5th Pctl	0.316	122.813	0.633	203.580	347.655
	95th Pctl	0.467	156.853	19.678	260.683	410.096
CPU5	Mean	0.357	132.085	18.177	206.253	356.872
	Std	0.050	6.810	3.325	4.435	7.414
	Median	0.352	132.091	18.861	205.137	357.266
	Min	0.135	120.232	0.758	202.522	343.874
	Max	0.710	157.389	29.595	245.883	396.458
	5th Pctl	0.305	122.248	17.918	203.417	345.756
	95th Pctl	0.429	141.138	19.618	212.657	366.738
CPU7	Mean	0.365	136.863	12.528	206.501	356.256
	Std	0.049	11.034	9.340	4.184	7.567
	Median	0.366	135.949	19.508	205.521	356.014
	Min	0.186	119.126	0.636	202.592	343.174
	Max	0.744	159.278	41.206	231.535	406.053
	5th Pctl	0.318	120.957	0.693	203.455	345.833
	95th Pctl	0.446	155.653	20.201	215.862	366.599

Table 7

Summary of time-related statistics for the Fully Virtualized (M-LoRa) configuration.

CPU#	Statistic (ms)	${\mathit{D}}_{\mathbf{FM}1}$	${\mathit{D}}_{\mathbf{FM}2}$	${\mathit{D}}_{\mathbf{FM}3}$	${\mathit{D}}_{\mathbf{FM}4}$	${\mathit{D}}_{\mathbf{FM}5}$	${\mathit{D}}_{\mathbf{FM}6}$	End-to-End
CPU1	Mean	3.077	63.747	28.435	1254.330	19.093	221.506	1590.188
	Std	0.757	22.839	7.938	66.981	1.008	42.424	84.692
	Median	2.977	57.090	29.614	1271.438	18.945	209.555	1604.391
	Min	0.157	49.896	16.147	1136.795	18.080	203.078	1448.641
	Max	6.942	338.310	109.459	1367.985	35.139	889.988	2254.066
	5th Pctl	2.691	54.845	17.019	1153.186	18.782	203.842	1468.904
	95th Pctl	3.809	102.914	36.966	1336.185	19.833	272.145	1694.090
CPU3	Mean	0.499	65.664	18.728	1258.625	19.037	212.172	1574.725
	Std	0.418	16.297	3.671	62.712	0.413	13.728	67.151
	Median	0.270	64.332	18.102	1275.234	18.930	206.596	1591.873
	Min	0.117	53.600	14.368	1142.156	18.729	203.102	1440.253
	Max	2.103	300.902	67.459	1347.932	23.384	290.774	1903.769
	5th Pctl	0.213	55.861	15.310	1156.576	18.797	203.727	1466.188
	95th Pctl	1.411	76.755	23.123	1332.375	19.744	244.304	1656.257
CPU5	Mean	0.273	65.453	17.390	1245.473	18.198	206.604	1553.391
	Std	0.126	13.885	2.904	73.764	0.453	3.778	75.676
	Median	0.219	65.186	17.367	1288.709	18.022	205.442	1592.710
	Min	0.089	55.081	13.886	1134.827	17.733	203.003	1434.621
	Max	0.894	248.150	62.538	1349.601	20.639	231.533	1842.127
	5th Pctl	0.199	56.918	14.865	1146.110	17.866	203.730	1450.479
	95th Pctl	0.602	77.618	19.945	1333.669	19.169	212.677	1642.595
CPU7	Mean	0.244	64.854	17.202	1238.718	18.332	207.290	1546.640
	Std	0.088	13.308	2.502	74.188	0.724	4.642	74.625
	Median	0.218	65.012	17.176	1271.380	18.352	206.605	1577.603
	Min	0.105	52.474	13.570	1138.849	17.443	203.579	1441.945
	Max	0.949	258.073	57.379	1355.924	21.150	258.361	1703.294
	5th Pctl	0.203	55.797	14.786	1147.882	17.519	203.972	1451.935
	95th Pctl	0.317	77.103	19.044	1331.395	19.560	212.428	1641.435

Table 8

Summary of time-related statistics for the Fully Virtualized (C-LoRa) configuration.

CPU#	Statistic (ms)	${\mathit{D}}_{\mathbf{FC}1}$	${\mathit{D}}_{\mathbf{FC}2}$	${\mathit{D}}_{\mathbf{FC}3}$	${\mathit{D}}_{\mathbf{FC}4}$	${\mathit{D}}_{\mathbf{FC}5}$	${\mathit{D}}_{\mathbf{FC}6}$	End-to-End
CPU1	Mean	2.975	24.172	13.162	1261.525	19.994	215.693	1537.520
	Std	0.547	1.702	1.448	70.653	1.259	17.722	75.449
	Median	2.963	23.926	13.000	1289.530	19.791	207.085	1565.277
	Min	0.133	18.140	8.000	1132.531	19.565	203.063	1396.831
	Max	5.558	33.081	19.000	1362.278	35.340	296.815	1691.212
	5th Pctl	2.437	22.210	11.000	1152.206	19.648	203.912	1419.026
	95th Pctl	3.593	26.648	16.000	1348.793	20.702	257.668	1637.980
CPU3	Mean	0.400	26.451	7.673	1267.809	19.689	212.490	1534.513
	Std	0.368	2.963	2.558	68.490	1.187	14.757	71.085
	Median	0.251	25.908	8.000	1297.608	19.607	206.513	1561.665
	Min	0.083	15.525	3.000	1136.675	18.626	203.088	1395.617
	Max	2.428	58.778	18.000	1364.697	33.113	306.447	1668.700
	5th Pctl	0.211	23.987	4.000	1155.515	18.752	203.759	1417.508
	95th Pctl	1.228	31.007	11.150	1346.367	20.388	241.613	1624.121
CPU5	Mean	0.290	22.851	5.905	1259.138	19.360	206.594	1514.137
	Std	0.195	1.418	1.469	72.263	1.767	4.180	72.185
	Median	0.219	22.568	6.000	1299.792	19.178	205.624	1553.839
	Min	0.190	18.939	2.000	1138.660	17.921	203.061	1393.222
	Max	1.601	31.693	13.000	1352.913	49.466	244.842	1609.577
	5th Pctl	0.200	21.139	3.000	1152.890	18.062	203.780	1408.988
	95th Pctl	0.696	25.231	8.000	1339.763	20.401	212.012	1595.753
CPU7	Mean	0.236	22.656	6.043	1247.001	19.356	206.102	1501.395
	Std	0.059	1.193	1.235	72.868	0.838	3.073	72.973
	Median	0.221	22.410	6.000	1288.602	19.171	205.388	1541.454
	Min	0.142	20.725	2.000	1130.179	18.938	202.792	1387.329
	Max	0.744	30.936	10.000	1346.013	30.304	237.397	1598.844
	5th Pctl	0.205	21.167	4.000	1149.004	18.999	203.656	1402.240
	95th Pctl	0.288	24.467	8.000	1334.168	20.216	209.989	1588.620

Table 9

Summary of time-related statistics for the Emulation configuration.

CPU#	Statistic (ms)	${\mathit{D}}_{\mathit{E}1}$	${\mathit{D}}_{\mathit{E}2}$	${\mathit{D}}_{\mathit{E}3}$	${\mathit{D}}_{\mathit{E}4}$	${\mathit{D}}_{\mathit{E}5}$	End-to-End
CPU1	Mean	2.988	4.310	17.661	1.041	206.644	232.644
	Std	0.973	0.933	0.706	0.862	2.615	3.100
	Median	2.911	4.182	17.950	0.834	206.471	232.328
	Min	0.298	1.981	14.607	0.584	203.115	226.830
	Max	7.941	9.060	20.237	11.394	225.908	252.066
	5th Pctl	1.346	3.391	16.714	0.666	203.524	228.716
	95th Pctl	5.301	6.313	18.305	2.114	211.176	238.089
CPU3	Mean	0.516	6.950	18.090	1.382	207.329	234.266
	Std	0.351	3.087	1.705	4.954	4.189	7.333
	Median	0.347	6.900	17.687	0.821	206.565	233.118
	Min	0.156	1.137	15.566	0.591	202.669	223.405
	Max	2.243	13.362	25.468	85.180	239.199	313.516
	5th Pctl	0.296	2.033	16.048	0.652	203.606	227.222
	95th Pctl	1.334	11.380	19.917	2.047	216.235	244.661
CPU5	Mean	0.378	6.871	18.553	0.985	206.696	233.484
	Std	0.136	3.051	0.187	0.942	3.073	4.410
	Median	0.336	7.039	18.567	0.831	206.303	233.127
	Min	0.170	1.122	17.536	0.574	202.859	224.395
	Max	1.486	11.861	19.649	16.581	226.068	256.010
	5th Pctl	0.307	1.709	18.138	0.652	203.285	227.087
	95th Pctl	0.645	11.170	18.761	1.674	211.360	240.102
CPU7	Mean	0.370	6.990	19.715	0.989	206.958	235.021
	Std	0.105	3.011	0.351	0.665	4.592	5.544
	Median	0.333	7.185	19.679	0.834	206.496	234.724
	Min	0.199	1.052	19.016	0.599	202.755	225.999
	Max	1.201	11.653	23.148	9.318	265.534	296.767
	5th Pctl	0.310	1.908	19.440	0.655	203.519	228.480
	95th Pctl	0.601	11.018	19.920	1.892	211.362	240.759