Federated Learning System for Dynamic Radio/MEC Resource Allocation and Slicing Control in Open Radio Access Network

Abstract

The evolution of cellular networks from fifth-generation (5G) architectures to beyond 5G (B5G) and sixth-generation (6G) systems necessitates innovative solutions to overcome the limitations of traditional Radio Access Network (RAN) infrastructures. Existing monolithic and proprietary RAN components restrict adaptability, interoperability, and optimal resource utilization, posing challenges in meeting the stringent requirements of next-generation applications. The Open Radio Access Network (O-RAN) and Multi-Access Edge Computing (MEC) have emerged as transformative paradigms, enabling disaggregation, virtualization, and real-time adaptability—which are key to achieving ultra-low latency, enhanced bandwidth efficiency, and intelligent resource management in future cellular systems. This paper presents a Federated Deep Reinforcement Learning (FDRL) framework for dynamic radio and edge computing resource allocation and slicing management in O-RAN environments. An Integer Linear Programming (ILP) model has also been developed, resulting in the proposed FDRL solution drastically reducing the system response time. On the other hand, unlike centralized Reinforcement Learning (RL) approaches, the proposed FDRL solution leverages Federated Learning (FL) to optimize performance while preserving data privacy and reducing communication overhead. Comparative evaluations against centralized models demonstrate that the federated approach improves learning efficiency and reduces bandwidth consumption. The system has been rigorously tested across multiple scenarios, including multi-client O-RAN environments and loss-of-synchronization conditions, confirming its resilience in distributed deployments. Additionally, a case study simulating realistic traffic profiles validates the proposed framework’s ability to dynamically manage radio and computational resources, ensuring efficient and adaptive O-RAN slicing for diverse and high-mobility scenarios.

1. Introduction

The ongoing evolution of beyond 5G (B5G) technologies networks and into sixth-generation (6G) mobile technology systems introduces unprecedented demands for flexibility, scalability, and efficiency. These emerging architectures must address increasingly complex service requirements, including ultra-low latency, enhanced bandwidth efficiency, and seamless adaptability to diverse deployment scenarios. However, current fifth-generation (5G) mobile technology architectures exhibit significant limitations, particularly in their monolithic and proprietary Radio Access Network (RAN) components, which restrict reconfigurability, interoperability, and optimal resource utilization.

To overcome these challenges, the Open Radio Access Network (O-RAN) and Multi-Access Edge Computing (MEC) have emerged as transformative paradigms. The O-RAN promotes disaggregation and virtualization, enabling modular, interoperable, software-based RAN components, while MEC moves compute and application hosting to the network edge to support latency-sensitive services. This integration enables the utilization of the entire O-RAN ecosystem, encompassing its defined functions and open interfaces, to establish more direct and efficient automated control over RAN operations. This integration also ensures the availability of resources to meet demanding requirements.

Although these advances offer promising solutions, the heterogeneity of B5G environments uncovers new challenges related to scalability, data security, and the diversity of Machine Learning (ML) models needed to automate RAN control. Federated learning (FL) has emerged as a revolutionary approach to tackling these challenges. Unlike traditional centralized ML systems, FL enables distributed model training by retaining data on local devices and only sharing model updates. This ensures data privacy while maintaining scalability, making it particularly suited for the O-RAN and MEC environments where sensitive, localized data are generated at the network edge. By integrating FL into the O-RAN, the flexibility and scalability needed to dynamically allocate resources, manage slicing admission control, and maintain Quality of Service (QoS) can be achieved across diverse and high-mobility deployment scenarios.

This article proposes the implementation of multiple Deep Reinforcement Learning (DRL) systems with the objective of fostering federative collaboration across these systems to generate a unified global model. This model is designed to facilitate effective responses to heterogeneous environments, such as high-mobility scenarios. As a result, each O-RAN network instance managed by the models deployed as Non-RT RIC Applications (rApps) and Extended Applications (xApps) will have the capability to regulate the admission of RAN-level slices and ensure the maintenance of QoS in environments characterized by diverse traffic profiles. This approach is intended to safeguard the security of user data while ensuring operational efficiency.

The key contributions of this paper are the following:

• Integration of FL with the O-RAN: The design and implementation of an FL system leveraging the deployment of rApps and xApps aim to address dynamic resource allocation in O-RAN. This approach ensures data privacy and minimizes communication overhead by exchanging model updates instead of raw data.

• Adaptability and Scalability: The proposed system enables slicing admission control and QoS maintenance across diverse O-RAN instances with varying resource request distributions, demonstrating flexibility in handling heterogeneous network conditions.

• Performance Evaluation and Comparative Analysis: The Federated Deep Reinforcement Learning (FDRL) framework has been rigorously evaluated against centralized systems, demonstrating superior learning efficiency, bandwidth consumption, and resiliency in distributed environments, including multi-client systems and loss-of-synchronization scenarios. In this case, it is understood that each client in the federated system corresponds to an O-RAN instance.

• Case Study Validation: A case study simulating realistic traffic profiles validates the efficacy of the proposed framework in dynamically managing radio and computational resources, highlighting its superiority over centralized approaches.

• Theoretical and Empirical Contribution: This work bridges the gap between theoretical and empirical studies by proposing an FDRL framework that is both conceptually grounded in the limitations of centralized ML for O-RAN and experimentally validated through performance evaluation in diverse network scenarios. Unlike prior studies that either focused on theoretical models or purely centralized learning approaches, our framework demonstrates practical feasibility and improvements in learning efficiency, bandwidth consumption, and resilience in real-world-inspired conditions. Furthermore, we have developed a theoretical model based on an Integer Linear Programming (ILP) solution to compare and validate the goodness of the proposed FDRL solution.

This article builds on these ideas to explore the role of FL in improving the intelligence and adaptability of future cellular networks. It continues with Section 2 dedicated to provide an overview of existing research on the 5G RAN and MEC ML-based resource allocation related to the O-RAN and federated learning. Section 3 describes the system model of the proposed solution, including the implementation scenario where its foundational technologies are integrated and the Deep Q-Network (DQN) models deployed as xApps within the O-RAN framework. In addition, this section details the configuration selected to carry out the simulation stage and the results of the tests performed under various scenarios to validate the correct operation. Section 4 shows the results of applying the proposed system in a realistic smart city use case, where different RAN instances are deployed throughout the city and face various simulated traffic profiles. Finally, the paper concludes by summarizing the key findings and outlining future research directions.

2. Related Work

The development of a next-generation RAN, especially within the O-RAN framework, is marked by a strong focus on intelligence and openness. A key goal of these efforts is to create RANs that are self-configuring and self-optimizing, thereby being capable of adapting to complex and diverse environments that integrate various platforms, technologies, and vendor solutions. Research has extensively explored critical challenges such as RAN resource allocation and fragmentation in the context of the O-RAN and MEC, striving to optimize performance while maintaining adherence to stringent QoS requirements. However, existing approaches often focus on individual aspects of network intelligence—such as O-RAN-based ML optimization, MEC-aware resource management, or FL for decentralized learning—without offering a unified framework that holistically integrates these elements.

For instance, Mollahasani et al. [1] employed actor-critic learning to optimize the resource allocation of Network Functions (NFs) between Centralized Units (CUs) and Distributed Units (DUs) in the O-RAN. This approach has been shown to lead to significant improvements in latency, throughput, and packet delivery ratios across a variety of traffic types and priorities. Similarly, the authors in [2] explored service slicing and baseband resource allocation tailored to distinct 5G service categories: Enhanced Mobile Broadband (eMBB), Ultra-Reliable Low Latency Communications (URLLCs), and Massive Machine-Type Communications (mMTCs). While these works offer valuable insights into optimizing O-RAN performance, they remain constrained to localized or single-layer optimizations without fully integrating MEC-based resource management or decentralized learning strategies.

Furthermore, an xApp-based approach presented in [3] utilizes ML algorithms for dynamic resource allocation. By overcoming the limitations of static resource allocation, this method adjusts Physical Resource Blocks (PRBs) in real time to optimize network performance and reduce costs. Likewise, Reinforcement Learning (RL)-based controllers such as those in [4] adapt modulation and coding schemes based on traffic flow and channel quality, enhancing throughput. However, these approaches rely on centralized decision making or independent RL agents, which limits their ability to generalize across heterogeneous O-RAN deployments, particularly in dynamic environments with fluctuating traffic and network conditions.

Despite these advancements, many studies focus solely on the O-RAN layer without fully integrating MEC, resulting in a gap in the management of both radio and edge computing resources in a unified manner. To address this gap, ref. [5] formulated a stochastic optimization problem for task offloading and resource allocation in MEC-enabled Cloud RAN environments. This semi-distributed framework balances energy efficiency and service delay, offering a balanced perspective on network performance trade-offs. Additionally, ref. [6] highlighted the role of Deep Learning (DL) in MEC resource allocation, covering task offloading, scheduling, and joint resource management. While these studies contribute valuable insights into MEC optimizations, they lack the integration of federated learning, a critical component for enhancing scalability and privacy in multi-client O-RAN deployments.

Building on these foundations, ref. [7] proposed a comprehensive but centralized solution that bridges radio-level resource allocation and MEC management using two DRL models based on DQN. The first model optimizes PRB and time slot allocation, while the second ensures QoS compliance across network slices. While effective in centralized environments, this approach suffers from a lack of federated collaboration, relying solely on the individual experiences of each system instance for training. Consequently, it is prone to suboptimal learning when encountering unexpected network conditions and remains vulnerable to data privacy risks and scalability limitations. The solution proposed in this paper overcomes these challenges by integrating FL with DRL, allowing for collaborative learning among multiple O-RAN instances while preserving privacy and improving adaptability to dynamic environments.

FL has emerged as a promising paradigm to address these challenges by enabling decentralized training of ML models while preserving data privacy. Recent studies emphasize the integration of FL in the O-RAN to enhance network intelligence and support advanced use cases. For instance, the paper [8] reviewed FL’s potential to improve data security and privacy in Next-Generation Wireless Networks (NextGs). The study discusses FL’s cooperative learning approach, which enables secure model training without sharing raw data, making it particularly suited for disaggregated architectures like the O-RAN.

To improve the efficiency of FL in resource-constrained environments, the authors in [9] introduced O-RANFed, a framework for client selection and resource allocation in FL tasks. By combining slicing-based client selection with deadline-aware resource optimization, this study demonstrated improvements in convergence speed and learning efficiency compared to traditional FL methods like FedAvg and FedProx. However, this approach does not fully integrate DRL-based decision-making mechanisms into the federated training process, which limits its ability to dynamically adapt to varying network conditions in real time.

Another notable approach is the integration of FL with DRL for enhanced network performance. H. Zhang et al. [10] introduced an FL-based coordination mechanism for multiple xApps, optimizing power control and slice-based resource allocation in the O-RAN. Their study demonstrated a 25% improvement in throughput for eMBB slices and a 30% reduction in latency for URLLCs slices when compared to traditional centralized RL methods. However, their approach is primarily focused on power allocation and does not comprehensively address the joint optimization of MEC and RAN resources within O-RAN environments. In contrast, the proposed framework integrates MEC-aware slicing decisions along with FL-enhanced DRL, ensuring end-to-end resource optimization in heterogeneous O-RAN networks.

Similarly, the study in [11] investigated the integration of FL and DRL for joint task offloading and fronthaul segment routing. Their work proposes a reward-based optimization framework to minimize the Cost of Delay (CoD) in task execution. Their results indicate that FL-assisted DRL reduces the average delay by approximately 40% compared to nonfederated RL models while ensuring task offloading efficiency. However, this study is tailored toward task scheduling and routing rather than dynamic slice admission and resource allocation in the O-RAN. The approach presented in our work extends FL-assisted DRL to slice admission control, PRB allocation, and MEC resource distribution, optimizing the QoS experience for diverse traffic profiles.

Despite these promising contributions, existing works have either focused on FL at a high level without integrating DRL for adaptive decision making or applied DRL without leveraging the collaborative learning benefits of FL. This work addresses these gaps by unifying FL, DRL, the O-RAN, and MEC into a single scalable framework, allowing multiple distributed DRL agents to collaborate via federated learning, thereby improving generalization, adaptability, and privacy preservation in heterogeneous RAN environments.

A comparative summary of these approaches is provided in Table 1, which highlights key performance metrics and the distinct advantages of the proposed framework.

By leveraging FL’s decentralized and privacy-preserving capabilities in combination with DRL’s real-time adaptability, this study presents a unified and comprehensive approach that advances the state of the art in intelligent O-RAN slicing and resource allocation.

3. System Model Description and Problem Statement

3.1. Integration Scenario

The implemented scenario operates within the O-RAN architecture, facilitating automated and direct control over the disaggregated modules advocated by this framework. This set of modules, referred to as E2 Nodes, comprises the Open Radio Unit (O-RU), Open Distributed Unit (O-DU), and Open Centralized Unit (O-CU), with their functions managed by the xApps housed within the Near-real-time RAN Intelligent Controller (near-RT RIC).

In this environment, the capabilities of the E2 interface are leveraged to gather information from the O-RU, O-DU, and O-CU modules of the Next-Generation Node B (gNB) and the MEC platform. Figure 1 illustrates this scenario, highlighting the aforementioned disaggregated modules following the 7.2x functional split, the interfaces defined for their interconnection, and their integration with the MEC system across both the data and control planes, with the latter being facilitated through the 5G Core Network (CN).

This scenario has been implemented to ensure that the near-RT RIC agents running in the xApps utilize the E2 interface with O-CU for the purpose of collecting RAN parameters. Additionally, they will leverage the N2 interface to access the 5G core network through the Access and Mobility Management Function (AMF). This access facilitates communication with the Network Exposure Function (NEF), a crucial component for disseminating information regarding the capabilities and services of 5G CN functions to external entities.

The MEC host, in contrast, is deployed at the edge of the 5G RAN, providing direct exposure to the O-CU. Its interconnection with the User Plane Function (UPF) is established via the N6 reference point [12]. The MEC orchestrator, functioning as the 5G Application Function (AF), facilitates efficient coordination with other NFs to support monitoring, provisioning, and policy enforcement.

3.2. Resource-Aware Slice Decision Problem and DQN Validation of Resource Allocation

The primary challenge addressed by the model deployed in the xApp dedicated to slice admission control is the selection of the most suitable slice to accommodate a specific resource demand. This selection process can be optimized using a slice-type decision algorithm, which is modeled as an ILP. The ILP determines the optimal slice type for each request by evaluating the demand alongside the available resources. These resources include MEC resources such as Central Processing Unit (CPU) cores, Random Access Memory (RAM), storage, and bandwidth, as well as RAN resources, where the associated bandwidth is evaluated based on the request’s transmission rate and latency requirements.

Let us consider an MEC system with CPU cores, RAM, storage, and Bandwidth (BW). Therefore, $MEC=[ME{C}^{CPU},ME{C}^{RAM},ME{C}^{BW},ME{C}^{ST}]$ represents the state of the MEC resources. Let us consider an O-RAN instance characterized by the BW and the latency (L) guaranteed to each connected UE. In this way, $RAN=\left[RA{N}^{BW}\right]$ represents the RAN resources corresponding to the available transmission rate status. Furthermore, a user request is defined as $[D^{CPU},D^{RAM},D^{ST},D^{BW},R,L]$ requesting $D^{\mathrm{CPU}}$ CPU cores, $D^{\mathrm{RAM}}$ RAM memory, $D^{\mathrm{ST}}$ storage, $D^{\mathrm{BW}}$ BW, R transfer rate, and L latency, respectively. It is assumed that, as a strategy for reserving resources for latency-critical services, the required latency solely determines the Bandwidth Part (BWP) to be selected for accommodating the request.

The Resource-aware Slice Decision Problem (One-RSDP) can be formulated to ensure that each request is assigned to the most suitable slice type. The objective is to minimize the unused capacity of the relevant set of available resources while ensuring that sufficient resources are allocated to meet each request’s requirements.

To formulate the One-RSDP problem, we define a binary variable $x_s$ to indicate whether one user resource request is to be assigned to the defined slice $s,s\in \mathcal{S}$:

(1)$\begin{array}{c}\hfill x_s=\left\{\begin{array}{cc}1\hfill & ,\mathrm{if}\mathrm{slice}\mathrm{type}s\mathrm{is}\mathrm{decided}\mathrm{for}\mathrm{the}\mathrm{current}\mathrm{request}\hfill \\ 0\hfill & ,\mathrm{otherwise}.\hfill \end{array}\right.\end{array}$

In addition, a binary variable z represents whether the new slice request is allocated in the system. Taking these considerations into account, the ILP formulation for the One-RSDP is as follows:

(2)$\begin{array}{ccc}\hfill min\sum _{\forall s\in \mathcal{S}}[& w_1·\left({\mathrm{MEC}}^{\mathrm{CPU}}-x_s·D^{\mathrm{CPU}}\right)+w_2·\left({\mathrm{MEC}}^{\mathrm{RAM}}-x_s·D^{\mathrm{RAM}}\right)+\hfill & \hfill \\ \hfill & w_3·\left({\mathrm{MEC}}^{\mathrm{ST}}-x_s·D^{\mathrm{ST}}\right)+w_4·\left({\mathrm{MEC}}^{\mathrm{BW}}-x_s·D^{\mathrm{BW}}\right)]-z\hfill \end{array}$

(3)$\begin{array}{cc}\hfill \mathrm{such} \mathrm{that}& x_s·D^{\mathrm{CPU}}\le {\mathrm{MEC}}^{\mathrm{CPU}}\forall s\in S\hfill \end{array}$

(4)$\begin{array}{cc}\hfill & x_s·D^{\mathrm{RAM}}\le {\mathrm{MEC}}^{\mathrm{RAM}}\forall s\in S\hfill \end{array}$

(5)$\begin{array}{cc}\hfill & x_s·D^{\mathrm{ST}}\le {\mathrm{MEC}}^{\mathrm{ST}}\forall s\in S\hfill \end{array}$

(6)$\begin{array}{cc}\hfill & x_s·D^{\mathrm{BW}}\le {\mathrm{MEC}}^{\mathrm{BW}}\forall s\in S\hfill \end{array}$

(7)$\begin{array}{ccc}\hfill & \sum _{\forall s\in S}{x}_s\le 1\hfill & \hfill \end{array}$

(8)$\begin{array}{ccc}\hfill & \mathrm{if} \mathrm{not} \mathrm{enough} \mathrm{RAN} \mathrm{capacity} \mathrm{for} \mathrm{current} \mathrm{request}\Rightarrow \sum _{\forall s\in S}{x}_s\le 0\hfill & \hfill \end{array}$

(9)$\begin{array}{cc}\hfill & \mathrm{if}{x}_s=1\Rightarrow z=1\forall s\in S\hfill \end{array}$

(10)$\begin{array}{ccc}\hfill & \mathrm{if}\sum _{\forall s\in S}{x}_s=0\Rightarrow z=0\hfill & \hfill \end{array}$

(11)$\begin{array}{cc}\hfill & x_s,z\mathrm{binary}\forall s\in S\hfill \end{array}$

The objective (2) is to minimize the unused capacity of the relevant set of available resources while maximizing the probability of assigning the current slice request. The formalization of the One-RSDP problem led to a multi-objective model incorporating several objectives. To simplify it into a single-objective model, the weighted sum method was applied. The relative importance of each component in the objective function was determined by assigning weight factors $w_i,i=\{1,\dots ,4\}$, where ${\sum }_{i=1}^4{w}_i=1$. In our scenario, all weight values are equal, indicating that all components of this function are equally important. In addition, to prevent any term from dominating the others, their magnitudes were normalized.

Constraints (3), (4), (5), and (6) ensure that the sum of the MEC resources demanded by the current slice request and all the previous allocated slices do not exceed its available capacity in terms of CPU, RAM, storage, and BW resources.

Constraint (7) guarantees that the current request is assigned to only one slice type, which will be the most suitable. Moreover, constraint (8) uses an external function to determine if there are sufficient RAN resources to assign the current slice request using the RAN resource allocation strategy explained in Section 3.3. If no such resources exist, the request will not be allocated even if the MEC has enough capacity.

The nonlinear constraints (9) and (10) are implications that define the dependency between the variables $x_s$ and z, establishing that z is equal to 1 if the current slice request is allocated, and it is equal to 0 if insufficient availability exists to place the request. Using linearization techniques, both nonlinear constraints can be linearized. Thus, constraint (9) is equivalent to

(12)$z\ge x_s\forall s\in \mathcal{S}$

and constraint (10) is linearized with the following expression:

(13)$z\le \sum _{\forall s\in \mathcal{S}}{x}_s$

Finally, all variables are defined as binary in (11).

The iterative algorithm in Figure 2 utilizes a One-RSDP ILP to determine the outcome of a slice request. This process is continuously iterative until all demands are satisfied. The system evolves iteratively for each request received, following a defined probability distribution, such as the Poisson distribution, to determine the optimal slice type. Each time one slice is assigned, the corresponding resources are reserved during a specified period. When finishing, these resources are released.

The ILP model was implemented using the Python-based package Pyomo [13,14] and Gurobi [15] as its underlying solver, whereas the rest of the iterative algorithm was coded in the Python 3.10 programming language. For each experiment, we ran the solutions 100 times, which enabled us to present results with confidence intervals of 95%. The simulations were performed on a workstation with a 3.30 GHz Intel Core-i9 processor and 64 GB of RAM.

Using an ILP is costly in terms of the time required to provide a solution, which makes it unfeasible to use in real time. For instance, in a scenario with the input data defined in Section 4.2, the average computation time required to obtain a result with the defined ILP is 22.32 ms in average, while the time spent on the proposed solution is about 395.4 μs. An approximately 56 times faster system response time is achieved compared to the ILP, running both systems on machines with equal computational power.

Consequently, the mathematical model is evaluated in this study solely as a performance benchmark. For this reason, approximation functions such as neural networks with methods based on DRL were selected in this work. These methods enable the identification of a satisfactory solution that is close to optimal, with processing speeds suitable for real-time applications, once the trained model is obtained.

3.3. Resource Allocation Strategy

The objective of the model implemented in the xApps is to efficiently allocate user demands to the appropriate RAN-level slice. These demands include bandwidth and latency from the radio perspective, as well as MEC resources. To achieve this, it continuously evaluates the available resources in terms of PRBs and slots, as well as CPU cores, storage, RAM, and MEC bandwidth. By analyzing this information, the model dynamically implements the allocation strategy, ensuring the creation of slices that match user demands and network policies.

The allocation of radio resources, in terms of the transmission rate, is achieved through the allocation of PRBs according to the Shannon–Hartley channel capacity model. This model incorporates various factors, including received signal power, noise power, interference power, spectral efficiency, and interference from adjacent channels. The following expression delineates the model utilized:

(14)$R=W\ast {log}_2(1+SiNR)$

In this equation, R denotes the transmission rate (in bits per second), W represents the effective bandwidth (in hertz), and $SiNR$ is the signal-to-interference-plus-noise ratio (in decibels). Furthermore, the subsequent expression is employed to evaluate the effective bandwidth.

(15)$W=B{W}_{RB}\ast \eta \ast N_{OFDM}$

The bandwidth assigned to a PRB, denoted as $B{W}_{RB}$, is determined by the Subcarrier Spacing (SCS), as expressed by the following equation: $B{W}_{RB}=SCS\ast 12subcarriers$. The spectral efficiency (as indicated by the corresponding MCS index table) is represented by $\eta$, and the number of Orthogonal Frequency Division Multiplexing (OFDM) symbols required to meet the slice request is denoted by $N_{OFDM}$. The allocation of RAN resources to ensure a specified transmission rate is determined by the number of OFDM symbols necessary to meet this demand.

To address the latency requirements of user demand, a mixed-numbering scenario is under consideration. This approach aligns with the implementation by L. Marijanovic et al [16]. The available transmission bandwidth is to be divided into multiple BWPs, each with a distinct SCS. Slice requests from time-sensitive services will be allocated in the BWP configured to provide lower latency [17].

Taking into account the bandwidth of each PRB in its respective BWP, a resource allocation matrix is created. This matrix consists of 14 OFDM symbols for each PRB, which can be defined from a frequency point of view. This approach provides a limited amount of resources to allocate in terms of PRBs at the RAN level. The following expression can be used to calculate this:

(16)$N_{PRBs}=14\ast (BW{P}_{BW}-GB)/B{W}_{RB}$

The number of allocable PRBs for a specific BWP is denoted by $N_{PRBs}$, the bandwidth defined for the BWP is $BW{P}_{BW}$, and the guard band associated with the selected numerology is denoted by $GB$.

As illustrated in the time/frequency grid of Figure 3, the channel bandwidth is apportioned into two BWPs, each configured with a distinct numerology. This configuration is used to allocate guaranteed resources to latency-critical and noncritical requests, respectively. It is important to note that each BWP consists of a finite amount of time–frequency resources, the size of which depends on the selected numerology.

This configuration utilizes a BWP dedicated to latency-critical requests, reducing the period between radio subframe slots. This contributes to enhanced latency reduction in comparison to BWPs employing higher numerologies. As the figure above illustrates, a subframe radio slot is composed of 14 OFDM symbols. The selection of a higher numerology for BWP2 reduces the duration of the slots by half compared to BWP1, decreasing the period between two symbols of the same position within the slot and contributing to reduced latency in services assigned to this BWP.

3.4. DQN Model for RAN-Level Slice Admission Control

As depicted in Figure 4, the system employs a DQN-based resource allocation algorithm to handle incoming slice requests while ensuring optimal network utilization. The DQN agent is deployed as an xApp within the near-RT RIC. This strategic placement allows the agent to receive up-to-date network data—such as resource availability, active slice requests, and QoS metrics—enabling it to make prompt and informed decisions regarding slice admission control.

Upon receiving a request, the DQN agent assigns it to the appropriate slice type by analyzing available network resources and QoS constraints. To maintain efficiency, the system dynamically updates the list of processed requests, releasing resources previously occupied by completed tasks before handling new slice requests. If sufficient resources are available, the request is admitted, and the required PRBs are allocated within the corresponding BWP1 or BWP2. The selection between BWP1 or BWP2 depends on the latency sensitivity of the service, which is based on a predefined threshold.

Conversely, if resources are insufficient and the request is not latency-sensitive, the xApp issues a slice admission rejection report to the non-RT RIC, ensuring that rejected slices are properly logged.

3.4.1. State Space

At predetermined time intervals t, the DQN agent observes the system state by constructing a state vector

$s_t=(\mathrm{SR},\mathrm{RF}),$

where the following are defined:

• SR contains the resource requirements at both the RAN and MEC.

• RF is a binary indicator denoting whether the system has sufficient resources to fulfill the request (1 if sufficient; it is 0 if otherwise).

The state vector includes the parameters of Table 2:

This formulation ensures that the DQN agent is fully informed of the current resource availability and network demands, allowing it to make precise allocation decisions.

3.4.2. Action Space

At each decision step, the agent selects an action from the following set:

$A_t=\{a_1,a_2,\dots ,a_7\},$

where $a_1,a_2,\dots ,a_6$ correspond to accepting the slice request and assigning it to a specific slice type, while $a_7$ represents rejecting the request due to insufficient resources. The action space is further categorized as follows:

• Slice Admission (${\mathit{a}}_{\mathbf{1}}-{\mathit{a}}_{\mathbf{6}}$): The agent assigns a slice request to its corresponding slice type based on available resources.

• Slice Rejection (${\mathit{a}}_{\mathbf{7}}$): If resources are insufficient, the request is rejected, and a blocking report is sent to the non-RT RIC.

3.4.3. Reward Function

To ensure optimal decision making, the reward function is designed to maximize efficient resource utilization while maintaining QoS guarantees. At each time step t, the agent receives a positive reward only when a request is correctly processed:

$R_t=\left\{\begin{array}{cc}1,\hfill & \mathrm{if}\mathrm{the}\mathrm{slice}\mathrm{request}\mathrm{is}\mathrm{correctly}\mathrm{processed}\hfill \\ 0,\hfill & \mathrm{otherwise}\hfill \end{array}\right..$

A request is considered correctly processed if the following are met:

• The assigned PRBs match the requested bandwidth segment.

• The latency and transmission rate requirements are satisfied.

• The slice life cycle constraints are met.

• The system either assigns or rejects the request correctly based on resource availability.

In this context, “correctly processed” implies that the agent successfully allocates resources while meeting QoS constraints or appropriately rejects the request when resources are insufficient.

3.4.4. DQN Learning Process

The DQN learning process refines the agent’s policy for resource allocation by iteratively updating a deep neural network that approximates the optimal action-value function. At each time step t, the agent interacts with the environment—whose state is continuously updated by the near-RT RIC—and collects experiences in the form of tuples $(s_t,a_t,R_t,s_{t+1})$. The RIC aggregates real-time data on resource availability, active slice requests, and QoS metrics, which are used to construct the state vector $s_t=(\mathrm{SR},\mathrm{RF})$.

Upon observing $s_t$, the agent selects an action $a_t$ via an $ϵ$-greedy policy: with probability $ϵ$, a random action is chosen (exploration), and with probability $1-ϵ$, the action with the highest predicted Q-value is selected (exploitation). The chosen action—whether corresponding to slice admission or rejection—is executed, resulting in a transition to a new state $s_{t+1}$ and yielding a reward $R_t$ that reflects whether the slice request was correctly processed.

Each experience tuple is stored in a replay memory $\mathcal{D}$, facilitating the random sampling of mini-batches during training. This mechanism decouples the learning process from the sequential correlations of real-time network events, thereby enhancing stability.

The agent employs a deep neural network parameterized by $\theta$ to approximate the Q-function, $Q(s,a;\theta )$. A separate target network, with parameters ${\theta }^{-}$, is maintained to generate stable target Q-values. The learning objective is to minimize the temporal difference (TD) error, as quantified by the following loss function:

(17)$L\left(\theta \right)={\mathbb{E}}_{(s_t,a_t,R_t,s_{t+1})\sim \mathcal{D}}\left[{\left(R_t+\gamma \underset{a^{\prime }}{max}Q(s_{t+1},a^{\prime };{\theta }^{-})-Q(s_t,a_t;\theta )\right)}^2\right],$

where $\gamma$ is the discount factor. Minimization of this loss via a stochastic gradient descent iteratively updates the network parameters $\theta$ to better approximate the true action-value function.

To further enhance training stability, the target network parameters ${\theta }^{-}$ are periodically synchronized with the current network parameters $\theta$. This decoupling from the rapidly fluctuating estimates of the primary network mitigates the risk of divergence during learning.

By continuously sampling from the replay memory and updating the network parameters, the DQN agent incrementally improves its approximation of the optimal Q-function. Convergence is typically indicated by a stabilization of the loss function and improved resource allocation performance. Consequently, the agent’s decisions increasingly satisfy QoS requirements while optimizing network resource utilization. The integration of real-time network data provided by the RIC ensures that the learning process remains responsive to dynamic conditions, enabling adaptive and efficient resource allocation.

4. Federated System

The training and evaluation stage of the centralized DRL system, including an individual instance of this model, was concluded with positive results in terms of the functionality of the implemented models in [7]. However, this centralized approach has a clear disadvantage: the agents only learn based on their own experience. The agents learn exclusively from their own experiences, which can result in these models responding inaccurately to scenarios where the network behaves in an atypical or unexpected manner.

In order to solve this problem and provide the proposed system with a greater capacity for effective management in different traffic environments, a FDRL system was implemented, extending the principles of Horizontal Federated Learning (HFL) to RL [18,19]. In this way, multiple agents (located in different devices or environments) collaborate to train a shared RL model without centralizing their interaction data.

This approach provides the system with important advantages:

• Scalability: Multiple agents can learn in parallel in different environments.

• Data privacy: Since agents do not share raw data (environment states or actions), the system maintains privacy.

• Exploration diversity: Since agents operate in different environments, where data have the same format characteristics but different arrival and life cycle distributions, the aggregated model benefits from a diverse set of experiences, which can lead to better generalization.

4.1. Implementation Scenario

The proposed federated approach involves the deployment of multiple RAN instances and agents of the described model in each, with each agent functioning as an xApp in the near-RT RIC, similar to its role in the centralized approach. The new feature of this federated stage is the implementation of an rApp that acts as an aggregation server to form a global federated model from the local training parameters of the clients deployed in the different near-RT RICs.

Figure 5 shows the proposed scenario for this federated system. Each RAN instance has a dedicated MEC system to provide compute resources at the edge of the network and manage latency-critical services. Taking advantage of the virtualization and disaggregation benefits of the O-RAN architecture, each RAN instance has a dedicated near-RT RIC that implements the DQN models as part of the DRL system. On the other hand, a non-RT RIC is implemented, connected to each near-RT RIC, where the FL aggregation server is deployed as an rApp. This rApp is responsible for aggregating parameters from the various clients.

4.2. Simulation Setup

To carry out the implementation of the proposed FDRL system, the open source framework Flower has been used.

Flower is a novel FL framework that supports large-scale training and evaluation, both on real edge devices and on single or multi-node computational clusters. This enables scalable algorithmic investigation of real-world system conditions [20]. One of the main advantages provided by Flower, in addition to being language and framework independent of ML by design, is that it is fully extensible and can incorporate emerging algorithms, training strategies, and communication protocols. This makes it highly flexible and allows it to effectively incorporate the Stable Baselines3 DQN algorithms implemented to develop the centralized stage models [21]. To carry out this integration, it was necessary to study the data structures used by each of these frameworks to represent the neural network parameters (the only information present in the communication between clients and server) in order to create an effective compatibility. This enabled leveraging Flower’s functionalities to manage the federation and execute commands on clients deploying DQN models built using the Stable Baselines3 library.

In preparation for the simulation phase, a set of slice requirements was defined, as outlined in Table 3. These requirements associate two distinct slice types with each typical use case in the 5G landscape (eMBB, URLLCs, and mMTC). Each slice type specifies the required resources in terms of MEC parameters (CPU, RAM, storage, and bandwidth) and RAN parameters (minimum transmission rate and maximum latency, denoted as R and L, respectively, in Table 3). To generate the slice requests that form the synthetic dataset used for model training, two types of slice requests were defined for each slice type, resulting in 12 unique demand types. Each slice request was generated using an arrival rate that determines the time interval between successive requests, following an exponential distribution, while the request life cycle adhered to a Poisson distribution. The distribution parameters were kept constant for each instance of the implemented O-RAN system.

During the development of the training phase, various hyperparameter combinations were tested to identify the most effective configuration in terms of convergence. The number of neurons in the hidden layer was determined after evaluating models trained with 4, 8, 16, and 32 neurons.The exploration rate was adjusted between 0.25 and 0.6 to optimize the training phase dedicated to exploitation. Additionally, the learning rate was evaluated over the range of 0.01 to 0.0001, utilizing discrete values corresponding to negative powers of 10 within this interval.

In the $ϵ$-greedy strategy, the balance between exploration and exploitation is controlled by the $ϵ$ parameter, which determines the probability of selecting a random action versus choosing the action with the highest estimated Q-value. To analyze the impact of different exploration–exploitation trade-offs, a series of tests were conducted with initial exploration values of ${ϵ}_0=0.25,0.5,$ and $0.75$, with each defining the proportion of the training phase allocated to exploration.

The training process was divided into two main phases:

• Exploration Phase:

  • −. The agent randomly explores different actions to gather diverse experiences.

  • −. The $ϵ$-value follows an exponential decay function to gradually shift from exploration to exploitation.

• Exploitation Phase:

  • −. Once $ϵ$ reaches zero, the agent exclusively selects actions with the highest Q-values, leveraging the knowledge accumulated during exploration.

The $ϵ$-decay function is mathematically defined as follows:

(18)${ϵ}_t={ϵ}_0·e^{-\lambda t}$

where the following are defined:

• ${ϵ}_t$ is the exploration rate at timestep t.

• ${ϵ}_0$ is the initial exploration rate ($0.25,0.5,$ or $0.75$).

• $\lambda$ is the decay rate, determining how fast exploration decreases.

• t is the current training timestep.

This approach ensures that exploration gradually reduces over time, transitioning smoothly into a purely exploitative phase. The best-performing configuration was found with ${ϵ}_0=0.25$, dedicating 25% of the training period to exploration before switching to full exploitation. Table 4 summarizes the configuration parameters that yielded optimal performance during the centralized training stage of the proposed model.

The scenario setup was divided into two parts, with the first part focusing on configuring the radio resources and the second part focusing on configuring the MEC resources. To conduct the RAN simulation, a channel bandwidth of 100 MHz was assumed, resulting in two BWPs of 50 MHz each, with different numerologies, to allocate PRBs to slice requests requiring low latency or not, respectively. BWP1 was configured with a 15 kHz SCS, while BWP2 used a 30 kHz SCS. The guardband applied to each side of the carrier corresponds to the numerology immediately adjacent to the guardband, resulting in a 692.5 kHz guard for BWP1 and a 1045 kHz guard for BWP2. The spectral efficiency, denoted by $\eta$, was assumed to be that corresponding to 64-QAM modulation with a code rate of 873/1024. The threshold value used to determine whether a request is latency sensitive was set to 10 ms, so slice request radio resources whose latency demand is below this value will be allocated in BWP2.

Regarding the resource configuration of the MEC platform, reserved resources are allocated for each slice defined in the configuration, as outlined in Table 5. This approach ensures isolation between different types of slices in terms of available resources.

For the FL system, the participating clients were configured to receive commands from the server in the format expected by Stable Baselines3 implementations and respond with the requested parameters correctly. On the other hand, the server was configured to use the FedAvg aggregation strategy, which was selected because of its favorable behavior in environments such as the proposed system [22].

4.3. Simulation Results

To evaluate the effectiveness of the FDRL system, a comprehensive simulation was conducted that consisted of three main tests. Each test was designed to analyze different aspects of the federated learning process in a multi-client O-RAN environment, ensuring that the proposed solution would be robust, scalable, and adaptable to various real-world constraints.

The goal of these simulations is to validate the ability of the federated learning-based slice admission control to accomplish the following:

1. Ensure convergence of the global model across different training conditions.

2. Demonstrate how clients benefit from collaboration in the federated setting.

3. Evaluate the system’s response to asynchronous participation of clients, which reflects real-world scenarios where different O-RAN instances may have variable availability.

Each simulation test represents a unique network condition, focusing on federation dynamics, client participation variability, and system resilience against synchronism loss. The training process involves multiple clients acting as local learners under their respective environments, while a federated aggregation server updates a global model to optimize decision making across all participants.

In these tests, the reward function was designed to maximize the number of correctly processed slice admission requests within the simulated network, where an ideal outcome would be a reward equivalent to the number of slice requests in an episode, that is, a value of 100. The DQN agents, deployed as xApps in near-RT RICs, continuously improve their policies through reinforcement learning, learning from past decisions and optimizing their slice selection strategies in a collaborative manner via federated learning.

The simulation framework was structured as follows:

• Each client undergoes local training rounds where it updates its DQN-based slice admission control policy based on locally collected data.

• After a defined number of local training steps, the clients transmit their learned models to the aggregation server, which aggregates the model parameters and distributes a new global model to all participating clients.

• The training process is iterative, with the global model progressively improving through multiple federation rounds.

• The simulations examine the effects of late client arrivals and synchronization loss, assessing the system’s adaptability in real-world conditions.

4.3.1. Test 1—Single System

The first test was focused on testing the federation process. The scenario implemented in this test is shown in Figure 6, where two clients participate during all aggregation rounds. Two hundred thousand training timesteps were implemented in each of the five rounds of local training managed by the aggregation server.

During the five rounds of individual local training for each client, the behavior of the reward obtained was monitored, observing its relationship with the exploration ratio. Figure 7 shows the results for each client during the individual local training. It can be seen that each client, given that it experienced a different environment, presented a different behavior during the first round of training. After aggregating the parameters from both clients, the server constructed a global model, which it sent to each near-RT RIC before initiating the next round of local training. The results indicate that, starting from round 3, with the selected hyperparameters and slice request generation configuration, two rounds of aggregation combined with local training of 200,000 timesteps were sufficient to achieve convergence in the global model.

Across all simulation rounds, each training episode was capped at 100 slice requests, meaning that the maximum cumulative reward achievable per episode is 100, as illustrated in Figure 7a,b. When the model reached the maximum reward by accurately processing 100 demands, it is assumed that this behavior validates the efficacy of the slice admission control.

4.3.2. Test 2—Three-Client System with Late Arrival

After verifying the correct functioning of the federation process in Test 1, the second test focused on the implementation of a system with three clients, as shown in Figure 8. Two of these three clients join the system late: Client 1 arrives at the end of Round 1 and participates in the remaining aggregation rounds, while Client 3 does not participate in the aggregation rounds but arrives at the beginning of the last local training round and trains that model with the global model that the server sends at that time.

This test was set up with 200 thousand local training steps for each of the five aggregation rounds implemented. Only Client 2 participated in all rounds and, as the results in Figure 9 show, it was not until after the third round that the model reached the desired convergence.

Client 1 entered the system in Round 2 and started training with the model obtained by Client 1 after Round 1. Then, it started participating in the aggregation and, after two rounds, it managed to form a global model that obtained the maximum number of rewards.

In the case of Client 3, it joined the system only at the beginning of the final round, during which it received the global model from the server and performed local training in Round 5. The results demonstrate that, after the global model had converged, Client 3 achieved the maximum reward during its training round.

4.3.3. Test 3—Three-Client System with Synchronism Loss

The third evaluation test of the proposed FDRL system involves a scenario where one of the participating clients loses synchrony with the system. As shown in Figure 10, the system consists of three clients, with two of them participating in the aggregation process throughout all the implemented rounds. Client 1, however, joins the system at the beginning of Round 2 but subsequently loses synchrony and does not rejoin until the end of Round 4. At that point, the server transmits the global model, formed through the aggregation of the remaining clients, to Client 1.

After confirming in the previous tests that convergence in the global model could be achieved after three rounds of local training with two clients, the number of training steps was halved in this third test. Figure 11 presents the results obtained in terms of the reward achieved.

As observed in the previous tests, Clients 1 and 2, which participated in all rounds, achieved a global model that converged after the third round. In contrast, the results for Client 3 indicate that when it lost synchrony, the global model had not yet reached convergence. Despite obtaining the maximum reward at the end of the second round, this model is not considered to meet the convergence criterion, as the reward was achieved within less than 10% of the total training period. However, once Client 3 reconnected to the system, it received the global model from the server and trained further to achieve the maximum reward.

Another indication of the nonconvergence of the global model after the second round of training is that only Client 3 achieved the maximum reward. This occurred because the global model formed at this stage was better aligned with the characteristics of Client 3’s environment but remained suboptimal for the other clients.

5. Use Case: Dynamic Radio/MEC Resource Allocation and Slicing Management in High Mobility Scenarios

After confirming the effectiveness of the federated approach in the proposed solution and its advantages in terms of scalability and training process efficiency, this section introduces the application of the system in a use case with a realistic scenario.

5.1. Setting up the Case Study Scenario

The use case scenario simulates a smart city comprising four nodes, each representing an instance of a near-RT RIC (along with their associated RAN and MEC systems) that manages distinct traffic profiles specific to the area of the city it serves. Each of these four nodes hosts two models deployed as xApps within the near-RT RICs:

• Model 1: Manages the admission control of the radio and MEC resource slices.

• Model 2: Guarantees the quality of service of the processed slices through the execution of configuration actions, as described in [7].

Additionally, a separate node is defined to implement a non-RT RIC. This node hosts an aggregation server deployed as an rApp, which is responsible for managing the aggregation of DQN models from client nodes using FL. The FL framework is utilized to collaboratively train the Model 1 instances of each client node while preserving data privacy.

5.1.1. Traffic Profile Definition

High mobility scenarios in the O-RAN represent some of the most challenging environments for resource allocation, where efficient management can significantly affect network performance. Such scenarios inherently introduce dynamic and heterogeneous traffic conditions, making an approach of establishing specific traffic profiles for each environment practical and enlightening.

Each defined profile represents a realistic high mobility scenario, highlighting the diverse and dynamic nature of traffic patterns observed in modern cellular networks. These profiles allow for a comprehensive evaluation of the adaptability, scalability, and performance of the FDRL system under different conditions:

1. Dense urban mobility

   Urban areas often experience the following:

   • High user density, resulting in frequent reservation of resources.

   • Dynamic traffic patterns due to fluctuating user activity (e.g., commuting, leisure, emergencies, etc.).

Moderate suburban mobility

Suburban regions, with lower user density and more stable traffic, offer the opposite scenario to urban areas. They test the system’s ability to effectively manage less dynamic but resource-intensive traffic.

Mobility on highways

Highways pose unique challenges:

• Users (e.g., vehicles) move quickly through the coverage areas.

• Requests are less frequent but may require greater resources for real-time applications such as autonomous driving, streaming, and security systems.

This scenario evaluates system performance under low but critical demand.

Event-based mobility

Temporary events such as concerts or sports matches introduce the following:

• Extremely high and localized traffic surges.

• Brief but intense demands for radio and MEC resources.

This scenario tests the system’s ability to allocate resources efficiently during peak loads.

5.1.2. Implementation Scenario

The scenario designed to implement this use case envisions a smart city with diverse traffic-generating environments, such as those found in the city of Barcelona, as illustrated in Figure 12.

In this scenario, the four near-RT RIC nodes are placed at different points in the city to provide the system with a wide range of traffic conditions. The configuration of the slice request generation environment is adapted to the distribution of each traffic profile. The selected values are presented in Table 6.

For the dense urban mobility profile, a node is placed in the Example district, which is an urban area characterized by high population density and intense user activity, including video streaming, online gaming, and real-time conferencing. The high arrival rate reflects the continuous and relentless demand for resources in such environments. Additionally, the life cycle is relatively short, as urban traffic often involves brief and rapid interactions, such as short video streams or voice calls.

For the moderate suburban mobility profile, a node is located in the Sarrià-Sant Gervasi district, which is characterized by moderate user density and more stable traffic patterns compared to urban environments. The arrival rate reflects the predictable and less dynamic nature of traffic in these areas. Requests in this profile typically have a longer duration, such as streaming or browsing, representing sustained interactions with moderate resource requirements.

The mobility profile for highways is applied by positioning a node near the Ronda de Dalt. Highways are characterized by low but critical traffic generated by moving vehicles, including autonomous driving updates, infotainment systems, and safety alerts. The low arrival rate reflects the reduced number of interactions typically expected in such scenarios. Requests are generally of long duration due to resource-intensive applications, such as navigation systems or continuous vehicle monitoring. This ensures that the system is tested to manage prolonged demands with high reliability.

For the eventual mobility profile, a node is placed in the district of Les Corts near the Camp Nou stadium, where events such as sports matches generate highly localized and intense bursts of traffic. The very high arrival rate captures the sudden surges in demand that occur during these events. Requests are short-lived, reflecting the rapid turnover in resource allocation, as users frequently switch activities such as posting on social media or streaming content.

5.2. System Evaluation

The proposed system for this use case was implemented in the described realistic scenario and evaluated in two stages. In the initial stage, the FL framework was executed to establish a global model, enabling the Model 1 instances at each node to acquire knowledge about the traffic environments present in the other nodes and to perform slice admission actions efficiently. The second stage involved evaluating the system’s operation, which included executing both Model 1 and Model 2 at each node, verifying their proper functionality and ensuring their interaction with the implemented database. This process follows the communication channel between xApps as described in [7], where both models interact with a global database to ensure data consistency.

5.2.1. FL Evaluation

To evaluate the FL framework in this use case, the environments for the Model 1 instances were configured on each deployed node to align with the parameters defined by their respective traffic profiles. The aggregation server was then set up to ensure all nodes were available after each round of local training, facilitating the construction of the global model.

In this stage, five rounds of training were conducted, with each round consisting of 100,000 timesteps. The results of the FL process are presented in Figure 13 for the nodes corresponding to the different traffic profiles: dense urban mobility (Figure 13a), moderate suburban mobility (Figure 13b), highway mobility (Figure 13c), and event-based mobility (Figure 13d).

The results indicate that, after two rounds of training, the clients were unable to achieve the maximum number of rewards. However, by the third round, the global model created by the server through the aggregation of parameters from all clients demonstrated the ability to process all types of slice requests. This is confirmed by observing that the same global model achieved the maximum number of rewards across all defined traffic environments.

This federated approach used approximately one-third of the training time required to achieve convergence in the centralized approach, thus justifying the advantages it brings to the proposed solution.

5.2.2. Joint Evaluation

To conduct this evaluation, one of the nodes defined in the use case was selected, and an evaluation period of 100 timesteps was executed. During this period, the two DQN models deployed on this node were run simultaneously, mimicking their operation in a real-world scenario. Both models interacted with their respective databases to prevent inconsistencies in the current status of processed requests and available resources, as outlined in [7] for the centralized approach. The results of this evaluation period are presented in Figure 14, showing that both models achieved the maximum number of rewards, thereby validating the system’s effectiveness.

5.2.3. Analysis of the Advantages Versus Centralized Approach

In implementing this use case, it is crucial to consider the impact of, both centralized and federated approaches on bandwidth utilization and convergence speed. To facilitate this comparison, it is important to examine the information transmitted in each approach. The centralized ML approach involves transmitting sensitive information within the observation and action vectors, whereas the FL system transmits only the neural network parameters of the client model to the aggregation server, as illustrated in Figure 15. Similarly, for the system outlined in this paper, the aggregation cloud would correspond to the aggregation server based on the Flower framework that is deployed as an rApp in the non-RT RIC.

To illustrate this point, consider the use case outlined in this section for the centralized approach. Assuming each agent transmits 500,000 samples (comprising observations and actions, as determined from the centralized stage training results presented in [7]) to the server for training, we can calculate the bandwidth utilization. Table 7 presents the bandwidth utilization for this scenario, including the size of the observation vector (assuming the observation vector consists of seven elements of type float32, each occupying 4 bytes) and the action vector, for one and four clients.

Conversely, the federated approach requires each client to transmit only the neural network parameters to the aggregation server. Table 8 presents the bandwidth utilization of the federated system for this specific use case, assuming each client sends 100,000 samples (model parameters, with a worst-case size of approximately 1 MB) per round to the aggregation server. It is further assumed that three rounds are needed to achieve convergence based on the results from the federated stage. To enhance security, Transport Layer Security (TLS) was enabled for client–server communications. This included the TLS header in the transmitted data, increasing the total data size by 10%.

In summary, for this specific use case, the centralized ML approach would transmit 64 MB, whereas the FL approach would transmit only 26.4 MB while also incorporating an additional security layer. Moreover, the convergence rate of the learning process highlights improved efficiency in the federated approach. Figure 16 compares the global model formed after the third round in the use case presented with the centrally trained model from [7]. The early convergence of the federated system is clearly evident, emphasizing the effectiveness of this method in such scenarios.

5.2.4. Scalability Analysis

The system has been thoroughly evaluated in tests involving two, three, and four clients. Its scalability is determined not by the number of clients but by the capacity of each client to process multiple simultaneous slice requests efficiently. This feature is essential for ensuring the system’s adaptability to dynamic and high-demand environments, as outlined in the use case traffic profiles.

Each client in the system is designed to manage multiple slice requests, thereby ensuring effective scaling in the presence of higher traffic loads. The scalability of the system is governed by the number of slice requests received per second per client. This number is determined by the characteristics of the traffic profile, which in turn affect the arrival rates and fluctuations in resource demand.

To assess this, we have analyzed the total number of requests processed over a one-hour period for each traffic profile based on the request arrival rates defined in Table 6. Table 9 presents the estimated minimum and maximum number of requests the system can process in one hour (3600 s) for each scenario.

In order to make a more precise assessment of the system’s capacity, an estimate has been made of the number of active slice requests at any given moment within the system. This estimate considered both the arrival rate of requests and the average life cycle duration before resources were released. The estimated active requests are presented in Table 10.

These values provide a more detailed perspective on the system’s operational scalability:

• In dense urban mobility scenarios, the system actively manages between 54 and 135 slice requests at any given time.

• In moderate suburban areas, the system handles between 72 and 180 active requests, adapting to a more stable but resource-intensive workload.

• In highway mobility environments, the system processes between 90 and 225 simultaneous active requests, ensuring sufficient resources for vehicular applications.

• For event-based mobility, where extreme demand surges occur, the system must scale to manage up to 480 simultaneous active slice requests at peak load.

The findings corroborate the system’s capacity for adaptability and scalability, as evidenced by its effective management of thousands of slice requests per hour under diverse high-mobility conditions. The efficacy of the system in handling substantial request volumes, as evidenced by its ability to support real-world O-RAN deployments with varying traffic demands, underscores its superiority over approaches that merely increase the number of clients.

6. Limitations and Future Work

While the proposed FDRL system demonstrates significant improvements in O-RAN slicing and resource allocation, certain limitations should be acknowledged. These limitations highlight potential areas for future research and enhancements.

The efficiency of MEC resource allocation is highly dependent on the initial configuration parameters, such as slice definitions, total resources reserved per slice, and the load balancing mechanism. While the system can dynamically adapt through deep reinforcement learning, the initial parameterization plays a crucial role in achieving optimal allocation. In highly dynamic scenarios where MEC resource availability fluctuates unpredictably, performance could be further improved by integrating additional adaptive mechanisms.

Although the FL-based approach enhances privacy and scalability, it introduces additional processing overhead due to periodic aggregation rounds. One key factor is the added latency introduced by TLS encryption, which secures communications between clients and the aggregation server. While the impact of this latency remains manageable in the current setup, real-time applications with strict latency constraints, such as URLLCs services, might require further optimizations, such as compression techniques or differential updates for reducing model exchange overhead.

The effectiveness of the FL framework is contingent upon active participation from a diverse set of clients. Insufficient client involvement can result in a global model that lacks generalizability due to limited data representation. To address this, it is crucial to implement robust incentive mechanisms that encourage clients to contribute their local models during aggregation rounds. Future efforts should explore adaptive incentive strategies, such as dynamic aggregation weight adjustments and customized pricing models, to enhance client engagement and ensure the development of a comprehensive global model.

While FL significantly reduces bandwidth usage compared to centralized training by transmitting model parameters instead of raw data, its application to DRL in the O-RAN presents additional challenges beyond computational costs. Although local training introduces computational overhead, modern AI hardware has mitigated this constraint in many scenarios. However, the real challenges in the O-RAN arise from energy efficiency, synchronization, and real-time latency requirements, which can impact large-scale FL deployments. Future optimizations should consider lightweight RL models and hybrid training strategies that offload intensive computations to more capable network elements.

These limitations serve as potential avenues for future research, focusing on improving the efficiency, adaptability, and scalability of FDRL-based resource allocation in O-RAN environments.

7. Conclusions

This paper proposed and evaluated an FDRL system for dynamic radio and MEC resource allocation and slicing management in O-RAN environments. By addressing the limitations of centralized DRL systems, the FDRL framework demonstrated its capability to enhance scalability, data privacy, and adaptability across diverse high-mobility scenarios, such as smart cities.

The key contributions of this research are as follows:

1. Improved scalability and adaptability: The federated approach allowed multiple agents to collaboratively train a shared RL model without centralizing interaction data. This not only ensured robust performance in varied traffic conditions but also enabled parallel learning across heterogeneous environments.

2. Enhanced privacy and security: By decentralizing the learning process, the system preserved user data privacy, which is critical in sensitive applications involving edge devices.

3. Superior performance over centralized models: The proposed framework achieved faster convergence compared to centralized DRL models and reduced bandwidth consumption by 58%. Additionally, the system effectively handled dynamic resource allocation and complex slice management scenarios, maintaining high-quality service under varying network conditions.

4. Practical application for next-generation O-RAN deployments: The system was tested under high-mobility conditions, demonstrating stable performance in smart city scenarios and event-based mobility environments. These results underline the feasibility of deploying FDRL-based slicing management in real-world O-RAN infrastructures.

Although the proposed system presents significant improvements, certain challenges remain. The efficiency of MEC resource allocation is dependent on initial configuration parameters, and federated learning introduces additional processing overhead due to aggregation rounds and security mechanisms such as TLS encryption. Future work will explore optimization techniques to mitigate these challenges, including lightweight DRL models and adaptive learning strategies.

The findings from this study highlight the potential of FL frameworks in transforming resource management for next-generation cellular networks. Future research can further extend this approach by incorporating hybrid learning models and optimizing energy efficiency in federated deployments.

Author Contributions

Conceptualization, M.M.-M., C.R.d.M., C.C.-P., and S.S.-R.; methodology, M.M.-M. and C.C.-P.; software, M.M.-M. and C.R.d.M.; validation, C.R.d.M., C.C.-P., and S.S.-R.; formal analysis, M.M.-M. and C.C.-P.; investigation, M.M.-M.; resources, C.C.-P. and S.S.-R.; data curation, M.M.-M. and C.R.d.M.; writing—original draft preparation, M.M.-M.; writing—review and editing, M.M.-M., C.R.d.M., C.C.-P., and S.S.-R.; visualization, M.M.-M. and C.C.-P.; supervision, C.C.-P. and S.S.-R.; project administration, C.C.-P. and S.S.-R.; funding acquisition, C.C.-P. and S.S.-R. All authors have read and agreed to the published version of the manuscript.

Data Availability Statement

The software presented in this study is publicly available in a GitHub repository located at https://github.com/mmmorfa/FL_Model1 (accessed on 10 January 2025).

Conflicts of Interest

The authors declare no conflicts of interest.

Abbreviations

The following abbreviations are used in this manuscript:

5G	Fifth-Generation Mobile Technology
6G	Sixth-Generation Mobile Technology
AF	Application Function
AMF	Access and Mobility Management Function
B5G	Beyond 5G Technologies
BWP	Bandwidth Part
CN	Core Network
CPU	Central Processing Unit
CU	Centralized Unit
DL	Deep Learning
DQN	Deep Q-Network
DRL	Deep Reinforcement Learning
DU	Distributed Unit
eMBB	Enhanced Mobile Broadband
FDRL	Federated Deep Reinforcement Learning
FL	Federated Learning
gNB	Next Generation Node B
HFL	Horizontal Federated Learning
ILP	Integer Linear Programming
MCS	Modulation and Coding Scheme
MEC	Multi-Access Edge Computing
ML	Machine Learning
mMTC	Massive Machine-Type Communications
near-RT RIC	Near-real-time RAN Intelligent Controller
NEF	Network Exposure Function
NF	Network Function
O-CU	Open Centralized Unit
O-DU	Open Distributed Unit
OFDM	Orthogonal Frequency Division Multi-plexing
O-RAN	Open Radio Access Network
O-RU	Open Radio Unit
PRB	Physical Resource Block
QoS	Quality of Service
RAM	Random Access Memory
RAN	Radio Access Network
rApps	Non-RT RIC Applications
RB	Resource Block
RF	Radio Frequency
RIC	RAN Intelligent Controller
RL	Reinforcement Learning
RU	Radio Unit
SCS	Subcarrier Spacing
TLS	Transport Layer Security
UE	User Equipment
UPF	User Plane Function
URLLCs	Ultra-Reliable Low Latency Communications
xApps	Extended Applications

Footnotes

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Figures and Tables

Table 1

Comparison of FL–DRL approaches in O-RAN slicing and resource allocation.

Work	Main Focus	Performance Gain	Limitations
[10]	Power control, slice allocation	+25% throughput, −30% latency	No MEC integration
[11]	Task offloading, routing	−40% delay, optimized scheduling	No slicing control, PRB allocation, nor MEC integration

Table 2

State vector of model 1.

SR						RF
RAN		MEC
BW	L	CPU	ST	RAM	B	$0\mid 1$

Table 3

Slice requirements.

Slice Type	MEC CPU	MEC RAM	MEC Storage	MEC BW	RAN R	RAN L
1	4 cores	16 GB	100 GB	40 Mbps	50 Mbps	20 ms
2	4 cores	32 GB	100 GB	100 Mbps	30 Mbps	30 ms
3	8 cores	16 GB	32 GB	80 Mbps	20 Mbps	1 ms
4	4 cores	8 GB	16 GB	50 Mbps	25 Mbps	5 ms
5	2 cores	8 GB	32 GB	40 Mbps	10 Mbps	10 ms
6	2 cores	8 GB	32 GB	40 Mbps	5 Mbps	40 ms

Table 4

Simulation training hyperparameters.

Parameters	Value
Hidden neurons	32
Buffer size	150,000
Learning rate	0.01
Batch size	64
Gamma	0.99
Training timesteps	500,000

Table 5

Maximum resources reserved per slice.

Slice Type	CPU Cores	RAM (GB)	Storage (GB)	BW (Mbps)
1	128	512	5000	2000
2	128	512	5000	2000
3	128	512	5000	2000
4	128	512	5000	2000
5	64	256	2000	1000
6	64	256	2000	1000

Table 6

Selected values for request generation.

Traffic Profile	Arrival Rate (Requests/Second)	Life Cycle Rate (Minutes)
Dense urban mobility	4–6	2–5
Moderate suburban mobility	2–3	6–10
Mobility on highways	1–2	10–15
Event-based mobility	8–12	1–4

Table 7

Bandwidth utilization in the Centralized approach.

	Per Client			For 4 Clients
Centralized system	Size Observation vector	Size Action vector	Total	Total
	14 MB	2 MB	16 MB	64 MB

Table 8

Bandwidth utilization in the Federated approach.

	Per Client		For 4 Clients
Federated system	Size Parameters (Bidirectional)	Size after 3 rounds	Total	Total + TLS
	2 MB	6	24 MB	26.4 MB

Table 9

Scalability analysis based on the system’s request-handling capacity.

Traffic Profile	Min Requests in 1 H	Max Requests in 1 H
Dense urban mobility	14,400	21,600
Moderate suburban	7200	10,800
Mobility on highways	3600	7200
Event-based mobility	28,800	43,200

Table 10

Estimated number of active requests in the system.

Traffic Profile	Estimated Active Requests
Dense urban mobility	54–135
Moderate suburban	72–180
Mobility on highways	90–225
Event-based mobility	240–480

References

1. Mollahasani, S.; Erol-Kantarci, M.; Wilson, R. Dynamic CU-DU Selection for Resource Allocation in O-RAN Using Actor-Critic Learning. arXiv; 2021; arXiv: 2110.00492

2. Motalleb, M.K.; Shah-Mansouri, V.; Parsaeefard, S.; López, O.L.A. Resource Allocation in an Open RAN System Using Network Slicing. IEEE Trans. Netw. Serv. Manag.; 2023; 20, pp. 471-485. [DOI: https://dx.doi.org/10.1109/TNSM.2022.3205415]

3. Qazzaz, M.M.H.; Kułacz, Ł.; Kliks, A.; Zaidi, S.A.; Dryjanski, M.; McLernon, D. Machine Learning-based xApp for Dynamic Resource Allocation in O-RAN Networks. arXiv; 2024; arXiv: 2401.07643

4. Mungari, F. An RL Approach for Radio Resource Management in the O-RAN Architecture. Proceedings of the 2021 18th Annual IEEE International Conference on Sensing, Communication, and Networking (SECON); Rome, Italy, 6–9 July 2021; pp. 1-2. [DOI: https://dx.doi.org/10.1109/SECON52354.2021.9491579]

5. Zhang, Q.; Gui, L.; Hou, F.; Chen, J.; Zhu, S.; Tian, F. Dynamic Task Offloading and Resource Allocation for Mobile-Edge Computing in Dense Cloud RAN. IEEE Internet Things J.; 2020; 7, pp. 3282-3299. [DOI: https://dx.doi.org/10.1109/JIOT.2020.2967502]

6. Djigal, H.; Xu, J.; Liu, L.; Zhang, Y. Machine and Deep Learning for Resource Allocation in Multi-Access Edge Computing: A Survey. IEEE Commun. Surv. Tutor.; 2022; 24, pp. 2449-2494. [DOI: https://dx.doi.org/10.1109/COMST.2022.3199544]

7. Martínez-Morfa, M.; Mendoza, C.R.D.; Cervelló-Pastor, C.; Sallent, S. DRL-based xApps for Dynamic RAN and MEC Resource Allocation and Slicing in O-RAN. Proceedings of the 2024 15th International Conference on Network of the Future (NoF); Castelldefels, Spain, 2–4 October 2024; pp. 106-114. [DOI: https://dx.doi.org/10.1109/NoF62948.2024.10741435]

8. Islam, N.; Monir, M.F.; Syeed, M.; Hasan, M.; Uddin, F. Federated Learning Integration in O- RAN: A Concise Review. Proceedings of the 2023 33rd International Telecommunication Networks and Applications Conference; Melbourne, Australia, 29 November–1 December 2023; pp. 283-288. [DOI: https://dx.doi.org/10.1109/ITNAC59571.2023.10368561]

9. Singh, A.K.; Nguyen, K.K. Joint selection of local trainers and resource allocation for federated learning in open RAN intelligent controllers. Proceedings of the 2022 IEEE Wireless Communications and Networking Conference (WCNC); Austin, TX, USA, 10–13 April 2022; pp. 1874-1879.

10. Zhang, H.; Zhou, H.; Erol-Kantarci, M. Federated Deep Reinforcement Learning for Resource Allocation in O-RAN Slicing. arXiv; 2022; arXiv: 2208.01736

11. Ndikumana, A.; Nguyen, K.K.; Cheriet, M. Federated Learning Assisted Deep Q-Learning for Joint Task Offloading and Fronthaul Segment Routing in Open RAN. arXiv; 2023; arXiv: 2308.07258[DOI: https://dx.doi.org/10.1109/TNSM.2023.3245544]

12. Kekki, S.; Featherstone, W.; Fang, Y.; Kuure, P.; Li, A.; Ranjan, A.; Scarpina, S. MEC in 5G networks. ETSI White Pap.; 2018; 28, pp. 1-28.

13. Bynum, M.L.; Hackebeil, G.A.; Hart, W.E.; Laird, C.D.; Nicholson, B.L.; Siirola, J.D.; Watson, J.; Woodruff, D.L. Pyomo–Optimization Modeling in Python; 3rd ed. Springer Science & Business Media: Cham, Switzerland, 2021; Volume 67.

14. Hart, W.E.; Watson, J.-P.; Woodruff, D.L. Pyomo: Modeling and solving mathematical programs in Python. Math. Program. Comput.; 2011; 3, pp. 219-260. [DOI: https://dx.doi.org/10.1007/s12532-011-0026-8]

15. Gurobi Optimization, LLC. Gurobi Optimizer. 2025; Available online: http://www.gurobi.com (accessed on 10 January 2025).

16. Marijanovic, L.; Schwarz, S.; Rupp, M. Optimal Resource Allocation with Flexible Numerology. Proceedings of the 2018 IEEE International Conference on Communication Systems (ICCS); Chengdu, China, 19–21 December 2018; pp. 136-141. [DOI: https://dx.doi.org/10.1109/ICCS.2018.8689253]

17. Yazar, A.; Pekoz, B.; Arslan, H. Fundamentals of Multi-Numerology 5G New Radio. arXiv; 2019; arXiv: 1805.02842

18. Wen, J.; Zhang, Z.; Lan, Y.; Cui, Z.; Cai, J.; Zhang, W. A survey on federated learning: Challenges and applications. Int. J. Mach. Learn. Cybern.; 2023; 14, pp. 513-535. [DOI: https://dx.doi.org/10.1007/s13042-022-01647-y] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/36407495]

19. Mammen, P.M. Federated Learning: Opportunities and Challenges. arXiv; 2021; arXiv: 2101.05428

20. Beutel, D.J.; Topal, T.; Mathur, A.; Qiu, X.; Fernandez-Marques, J.; Gao, Y.; Sani, L.; Li, K.H.; Parcollet, T.; de Gusmão, P.P.B. et al. Flower: A Friendly Federated Learning Research Framework. arXiv; 2022; arXiv: 2007.14390

21. Raffin, A.; Hill, A.; Gleave, A.; Kanervisto, A.; Ernestus, M.; Dormann, N. Stable-Baselines3: Reliable Reinforcement Learning Implementations. J. Mach. Learn. Res.; 2021; 22, pp. 1-8.

22. Qi, P.; Chiaro, D.; Guzzo, A.; Ianni, M.; Fortino, G.; Piccialli, F. Model aggregation techniques in federated learning: A comprehensive survey. Future Gener. Comput. Syst.; 2024; 150, pp. 272-293. [DOI: https://dx.doi.org/10.1016/j.future.2023.09.008]