Toward 6G: Latency-Optimized MEC Systems with UAV and RIS Integration

Abstract

Multi-access edge computing (MEC) has emerged as a cornerstone technology for deploying 6G network services, offering efficient computation and ultra-low-latency communication. The integration of unmanned aerial vehicles (UAVs) and reconfigurable intelligent surfaces (RISs) further enhances wireless propagation, capacity, and coverage, presenting a transformative paradigm for next-generation networks. This paper addresses the critical challenge of task offloading and resource allocation in an MEC-based system, where a massive MIMO base station, serving multiple macro-cells, hosts the MEC server with support from a UAV-equipped RIS. We propose an optimization framework to minimize task execution latency for user equipment (UE) by jointly optimizing task offloading and communication resource allocation within this UAV-assisted, RIS-aided network. By modeling this problem as a Markov decision process (MDP) with a discrete-continuous hybrid action space, we develop a deep reinforcement learning (DRL) algorithm leveraging a hybrid space representation to solve it effectively. Extensive simulations validate the superiority of the proposed method, demonstrating significant latency reductions compared to state-of-the-art approaches, thereby advancing the feasibility of MEC in 6G networks.

1. Introduction

The rapid advancement of wireless technologies has driven the proliferation of portable intelligent devices, marking the dawn of the Internet of Things (IoT) era. According to the International Telecommunication Union (ITU), global mobile subscribers and data traffic are projected to grow substantially in the coming years, with mobile data traffic expected to reach approximately 5 zettabytes per month by 2030 [1]. This explosive growth, combined with the rising demand for data-intensive applications such as extended reality (XR), autonomous vehicles, and industrial automation, poses significant challenges to the capacity and sustainability of current 5G networks. To address the requirements of these emerging applications, future networks must achieve ultra-low latency and unparalleled reliability. Consequently, the development of 6G systems has become imperative to ensure robust, seamless, and efficient connectivity for the next generation of communication technologies (Ahmed et al. [2]).

The evolution toward 6G networks introduces significant advancements in communication efficiency and intelligent data processing, enabling seamless support for emerging technologies and applications. A crucial component of this vision is multi-access edge computing (MEC), which offloads computationally intensive tasks from resource-constrained Internet of Things (IoT) devices to edge servers. By processing data closer to the source, MEC effectively minimizes latency and reduces energy consumption, addressing two critical challenges faced by modern IoT ecosystems. As next-generation networks aim to deliver ultra-reliable, low-latency, and energy-efficient communication, MEC stands out as a foundational technology (Tariq et al. [3]). Its integration with advanced 6G capabilities ensures robust performance for diverse applications, ranging from real-time analytics to autonomous operations.

The deployment of 6G networks demands innovations to improve wireless channel propagation, particularly in overcoming high penetration losses associated with THz frequencies in complex environments. To address these challenges, solutions such as massive multiple-input multiple-output (mMIMO) and hybrid analog-digital beamforming have been proposed, ensuring efficient and reliable communication (Ahmed et al. [4]). However, designing efficient multi-antenna transceivers for THz bandwidth introduces significant challenges due to the complexity of beamforming in such high-frequency domains. To tackle these issues, UAV-based communications, where unmanned aerial vehicles act as flying base stations to establish line-of-sight (LoS) links and reconfigurable intelligent surface (RIS)-assisted environments, which dynamically control signal propagation to enhance physical layer communication, have been proposed. These innovations aim to overcome the limitations of THz communication while enhancing network performance. Moreover, UAVs improve signal strength and network efficiency by providing flexible and dynamic communication links, while RIS, with its programmable metasurfaces, mitigates path loss and channel sparsity by redirecting signals to strengthen connections between base stations and users (Mehrabian and Wong [5]). By combining these technologies, RIS-assisted UAV communications effectively leverage the strengths of both approaches, offering a powerful solution for addressing key challenges in next-generation networks and garnering significant research attention (Xu et al. [6]).

However, despite these advantages, the practical deployment of RIS and UAVs introduces several challenges. UAV-based communication is constrained by high power consumption, limited battery life, and stringent airspace regulations, which impact operational feasibility (Banafaa et al. [7]). Additionally, RIS requires real-time reconfigurability and advanced control mechanisms to dynamically adjust signal propagation, posing implementation difficulties. Efficient energy management strategies, regulatory compliance frameworks, and intelligent control algorithms are essential to overcoming these obstacles and fully realizing the potential of RIS-UAV-assisted networks.

The integration of MEC solutions with RIS-assisted UAV communications holds immense potential for next-generation wireless networks. This approach combines the computational efficiency of MEC with the enhanced signal propagation and flexibility of RIS and UAV technologies, paving the way for innovative solutions to meet the stringent demands of future wireless systems.

1.1. Contributions

The implementation of MEC systems assisted by both UAV and RIS technologies has garnered significant attention from the research community due to their potential to enhance network performance. Recent studies (Xu et al. [8]), Hu et al. [9] have demonstrated that the complete integration of these technologies offers substantial advantages compared to the exclusive use of either UAV-based MEC systems or RIS-assisted MEC systems (Wang et al. [10]). However, most existing works that integrate UAVs and RIS have primarily considered RIS mounted on the facades of buildings. This limitation underscores the need for exploring alternative RIS deployment strategies, such as UAV-assisted RIS, to fully realize the potential of these combined technologies.

Under these perspectives, this paper addresses the optimization problem of task offloading and resource allocation in MEC systems supported by a RIS-equipped UAV. The proposed optimization problem considers practical constraints, assuming perfect CSI at the MIMO base station (MBS), UAV energy limitations affecting its movement and service time, and configurable RIS phase shifts without additional delay overhead. These constraints ensure feasibility in real-world deployments.

The key contributions of this work to the current state of the art are as follows:

• We propose a novel optimization framework for a MEC system hosted within a massive MBS and assisted by a RIS-equipped UAV capable of flying within the coverage area. For this system, we formulate an optimization problem aimed at minimizing system latency by jointly optimizing the power allocation for each user, user association, phase shift configuration of RIS reflecting elements, and computing resource allocation at the MBS, all subject to the MBS’s computing resource constraints and QoS requirements. Moreover, this optimization problem is formulated into a Markov decision process (MDP).

• Addressing the challenge of discrete-continuous hybrid action spaces, the paper proposes a unique solution that integrates a mechanism to represent discrete actions using an embedding table and leverages a conditional variational autoencoder to handle continuous actions. By integrating these into a unified hybrid action space, the framework leverages the twin delayed deep deterministic policy gradient (TD3) algorithm to solve the joint optimization problem effectively.

• Extensive simulation results reveal that the proposed algorithm, when compared with benchmark approaches, effectively minimizes the total latency for executing tasks across all users.

1.2. Paper Organization

The remainder of this paper is structured as follows: Section 2 provides a review of related work. Section 3 describes the system model under consideration, followed by the formulation of the optimization problem for latency minimization in Section 4. The proposed methodology is detailed in Section 5, while Section 6 evaluates the effectiveness of the proposed algorithm in minimizing total system latency. Finally, Section 7 presents the conclusion and discusses potential future research directions.

2. Related Work

The integration of line-of-sight (LoS) transmissions enabled by UAVs with the smart and controllable propagation environment created by reconfigurable intelligent surfaces (RIS) is a growing research focus for next-generation wireless networks. UAV-assisted MEC systems have demonstrated significant potential to enhance network computing capacity dynamically and support emergency scenarios. Task offloading in such systems has become a critical research area, addressing challenges in resource allocation optimization and ensuring network efficiency.

Recent advancements have leveraged UAVs in 5G networks to enhance task offloading, addressing critical challenges like resource constraints and latency. Ning et al. [11] proposed a 5G-enabled task offloading network that clusters users geographically and models the offloading problem as a mixed-integer non-linear programming (MINLP) problem to maximize throughput. Similarly, Song et al. [12] introduced an evolutionary multi-objective RL algorithm to optimize UAV trajectory and task offloading, with objectives to minimize task delay, reduce UAV energy consumption, and maximize task collection efficiency. These works highlight the growing focus on integrating UAVs for efficient task management in next-generation networks.

Task dependencies have gained significant attention in UAV-assisted MEC systems due to their impact on resource allocation and system efficiency. Xu et al. [13] addressed this by formulating a joint optimization problem for resource allocation and UAV trajectory design, aiming to minimize system energy consumption while meeting delay and dependency constraints. Their approach combined dynamic programming and optimization algorithms to provide efficient solutions. However, their work was limited to a single UAV with constrained power and computational resources, underscoring the need for scalable and resource-efficient alternatives to support more complex and demanding scenarios.

Multi-UAV MEC systems address these limitations by enabling resource sharing and improving task offloading efficiency. For example, Goudarzi et al. [14] proposed a two-layer framework incorporating a queue-based algorithm to minimize delay, while Bai et al. [15] tackled non-convex load balancing challenges using Lyapunov optimization for online decision-making. Furthermore, Zhao et al. [16] introduced a multi-agent DRL approach based on the twin delayed deep deterministic policy gradient (TD3) algorithm to optimize UAV trajectories, power allocation, and task offloading, achieving significant cost reductions and scalability.

RIS technology has been explored to enhance MEC systems. Wang et al. [10] proposed a RIS-aided MEC system for heterogeneous networks (HetNet), focusing on the joint optimization of caching, task offloading, computing resources, and RIS/base station resource allocation. To address the NP-hard nature of this problem, they developed a two-stage optimization algorithm. Their findings indicate that integrating RIS into MEC HetNet systems can significantly reduce task computing delays, highlighting the potential of RIS to improve system performance.

The integration of RIS and UAV technologies has proven to significantly enhance MEC system performance through joint optimization. Xu et al. [8] proposed a RIS-enhanced, UAV-assisted MEC framework utilizing NOMA communication, focusing on optimizing RIS phase shifts, resource allocation, decoding order, and UAV deployment. Building on this, Hu et al. [9] incorporated dynamic UAV trajectories into the framework, further optimizing computation/offloading bits, RIS phase shifts, bandwidth allocation, and UAV trajectories. These advancements have demonstrated improved computation capacity, underscoring the effectiveness of combining RIS and UAV technologies in MEC systems for superior performance and resource efficiency.

3. System Model

We consider a mMIMO communication system that extends coverage between a MIMO base station (MBS) and distributed users, such as mobile users, vehicles, and IoT sensors, with the assistance of a RIS-equipped UAV. This integration significantly enhances network performance and quality of service (QoS), especially in areas affected by propagation blockages, improving reliability for UEs beyond direct MBS coverage (Zhai et al. [17]). In this system, the MBS is equipped with a large antenna array of L elements to serve K single-antenna users, which are grouped into M clusters represented by ${\mathbf{K}}_{\mathbf{U}}=\{{\mathcal{K}}_1,{\mathcal{K}}_2,\dots ,{\mathcal{K}}_M\}$. Each cluster has a varying number of users. The RIS-equipped UAV functions as a small-cell base station, with an RIS panel of N discrete elements reflecting signals from UEs to the MBS. The (m, k)-th UE denotes the k-th UE in the m-th cluster, where $u_{m,k}$ is the user association indicator, determining whether the UE offloads its computing task to the MEC server ($u_{m,k}=1$ for offloading, $u_{m,k}=0$ otherwise). User associations are represented by the vector $u_m={\left[u_{m,k}\right]}_{k=1}^{K_m}$ for cluster m and the matrix $\mathbf{u}={\left[u_m\right]}_{m=1}^M$ for all clusters, it is expressed as follows:

(1)$\begin{array}{c}\hfill u_{m,k}=\left\{\begin{array}{cc}{\displaystyle 1}\hfill & \mathrm{if}\mathrm{computing}\mathrm{task}\mathrm{offloading}\mathrm{is}\mathrm{needed},\hfill \\ {\displaystyle 0}\hfill & \mathrm{otherwise}.\hfill \end{array}\right.\end{array}$

A time division multiple access (TDMA) scheme is employed when the UAV serves the m-th cluster, allowing users to offload tasks to the MBS. The RIS plays a critical role by steering beams to reflect only the desired signal toward the MBS, minimizing interference from other users and ensuring efficient communication (Di Renzo et al. [18]).

3.1. Channel Model

We consider a 3D Cartesian coordinate system to represent the spatial positions of the MBS, UAV-RIS, and users. The MBS is located at $(x_0,y_0,H_0)$, where $H_0$ denotes the antenna height, while the UAV-RIS is positioned at $(x_U,y_U,H_U)$, with $H_U$ being the UAV’s altitude. The users (UEs) are distributed at ground level, with their positions represented as $(x_k,y_k,0)$, $k=1,2,\dots ,K$. The positions of all entities, including the MBS, UAV-RIS, and UEs, are assumed to be accurately tracked using the GPS and stored locally at the MBS. Communication between the MBS and the UAV-RIS is modeled under a LoS assumption, ensuring high-quality signal propagation. For the channel between the MBS and UAV-RIS, we adopt a free-space path loss model to describe signal attenuation. This model accounts for the LoS condition, and the specific mathematical expression for the free-space path loss model is expressed as follows:

(2)${\beta }_{m,0}={\displaystyle \frac{{\beta }_0}{d_{m,0}^2+{(H_0-H_m)}^2}},m=1,\dots ,M,$

where $d_{m,0}=\sqrt{{(x_0-x_m)}^2+{(y_0-y_m)}^2}$, $d_0$ is defined as the distance, and ${\beta }_0$ signifies the power gain associated with the wireless channel. Additionally, $H_m$ refers to the altitude of the UAV as it hovers above the m-th cluster.

The communication channel between the UAV-RIS and the $(m,k)$-th UE, classified as an air-to-ground channel, is more complex due to the effects of propagation blockages. To address this, the path-loss formulation incorporates both the air-to-air and air-to-ground links, represented as follows:

(3)${\beta }_{m,k}=P{L}_{m,k}+{\eta }_{LoS}{P}_{m,k}^{LoS}+{\eta }_{NLoS}{P}_{m,k}^{NLoS},$

We define the average additional losses for the LoS and NLoS paths as ${\eta }_{\mathrm{LoS}}$ and ${\eta }_{\mathrm{NLoS}}$, respectively. Based on these, the distance-based path loss can be expressed as follows:

(4)$P{L}_{m,k}=10log{\left({\displaystyle \frac{4\pi f_c{D}_{m,k}}{c}}\right)}^{\alpha },$

Here, $\alpha$ is the path loss exponent with $\alpha \ge 2$, while c represents the speed of light (in meters per second) and $f_c$ denotes the carrier frequency (in Hertz). The Euclidean distance from the UAV-RIS to the $(m,k)$-th UE is represented as $d_{m,k}$, and the total Euclidean distance, considering the UAV height $H_m$, is given by $D_{m,k}=\sqrt{d_{m,k}^2+H_m^2}$. As a result, the probability of LoS given as follows (Al-Hourani et al. [19]):

(5)$P_{m,k}^{LoS}={\displaystyle \frac{1}{1+aexp\left[-b\left(\arctan \left(\frac{H_m}{d_{m,k}}\right)-a\right)\right]}}$

where the values of the constants a and b are environment-dependent. Consequently, the probability of an NLoS connection can be expressed as $P_{m,k}^{\mathrm{NLoS}}=1-P_{m,k}^{\mathrm{LoS}}$. The free-space path loss model is chosen for the MBS-to-UAV-RIS link due to the highly probable LoS between the two, given the UAV-RIS’s elevated positioning and the absence of significant obstacles. This assumption aligns with established aerial communication models and ensures accurate signal attenuation estimation. In contrast, the UAV-to-UE link is more susceptible to environmental factors such as urban obstructions and terrain variations. To account for these uncertainties, a probabilistic LoS/NLoS model is employed, effectively capturing the dynamic nature of air-to-ground signal propagation in UAV-assisted networks.

For UEs requiring assistance from the RIS to communicate with the MBS, the small-scale fading coefficients for the channels are represented as $h_{m,k}\in {\mathbb{C}}^{N\times 1}$ for the link from the $(m,k)$-th UE to the UAV-RIS, and $h_{m,0}^H\in {\mathbb{C}}^{N\times L}$ for the link from the UAV-RIS to the MBS, where $m\in M$ and $k\in K_m$. These coefficients are assumed to be independent and identically distributed (i.i.d.) random variables with zero mean and unit variance. The superscript H denotes the conjugate transpose operation. Furthermore, we define $H_{m,k}\in {\mathbb{C}}^{L\times N}$ and $H_{m,0}^H\in {\mathbb{C}}^{L\times N}$ as the channel matrices for the $(m,k)$-th UE to UAV-RIS link and the UAV-RIS to MBS link, respectively, where these matrices describe the small-scale fading effects over the respective multi-antenna channels are as follows:

(6)$H_{m,k}=\sqrt{{\beta }_{m,k}}{h}_{m,k},\mathrm{and}{H}_{m,0}^H=\sqrt{{\beta }_{m,0}}{h}_{m,0}^H.$

Hence, the cascaded channel matrix of the link from the $(m,k)$-th UE to the MBS via the UAV-RIS, $G_{m,k}\in {\mathbb{C}}^{L\times 1}$, was described by Wu and Zhang [20]:

(7)${\mathbf{G}}_{m,k}={\mathbf{H}}_{m,0}^H{\mathbf{\Phi }}_m{\mathbf{H}}_{m,k},m\in \mathcal{M},$

where ${\mathbf{\Phi }}_m=\mathrm{diag}[{\varphi }_{1m},{\varphi }_{2m},\dots ,{\varphi }_{Nm}]$ represents the phase shift matrix at the UAV-RIS. Each element ${\varphi }_{nm}={\alpha }_{nm}{e}^{j{\theta }_{nm}}$ characterizes the reflection properties of the n-th reflecting element, where ${\alpha }_{nm}\in [0,1]$ denotes the reflection amplitude, and ${\theta }_{nm}\in [0,2\pi ]$ represents the phase shift. Here, $n=1,2,\dots ,N$ and $m\in M$. Assuming that the RIS reflecting elements modify only the phase of the reflected signals, we set ${\alpha }_{nm}=1$, ensuring maximum signal reflection without amplitude attenuation.

3.2. Transmission Scheme

Given that the $(m,k)$-th UE in the m-th group lacks a direct link to the MBS due to propagation blockages like large buildings, it offloads its computing task to the MBS by transmitting its signal via the RIS-equipped UAV. Consequently, the signal received at the MBS from the $(m,k)$-th UE is represented as follows:

(8)$\begin{array}{cc}\hfill y_{m,k}& =\sqrt{P_{m,k}}{\mathbf{G}}_{m,k}^H{\mathit{f}}_{m,k}{s}_{m,k}\hfill \\ & +\sum _{l=1,l\ne k}^{K_m}\sqrt{P_{l,m}}{\mathbf{G}}_{l,m}^H{\mathit{f}}_{l,m}{s}_{l,m}+n_0,\hfill \end{array}$

here, $P_{m,k}$ represents the transmit power of the $(m,k)$-th UE, ${\mathbf{f}}_{m,k}\in {\mathbb{C}}^{L\times 1}$ is the MBS beamforming vector for the $(m,k)$-th UE, and $s_{m,k}$ is the offloading information transmitted by the UE with $|s_{m,k}{|}^2\le 1$. The AWGN at the MBS is denoted by $n_0\sim \mathcal{CN}(0,{\sigma }_0^2)$, and the maximum transmit power of the UE is $P_{m,k}^{\max }$. The first term in (8) represents the signal transmitted by the $(m,k)$-th UE via the RIS panel, while the second term captures intra-cell interference from other UEs in the m-th group transmitting simultaneously. To mitigate this interference, the zero-forcing (ZF) technique is defined as follows:

(9)${\mathbf{G}}_m=[{\mathbf{G}}_{m,1},\dots ,{\mathbf{G}}_{m,K_m}]\in {\mathbb{C}}^{L\times K_m}(m=0,1,\dots ,M).$

As the eigenvalue distribution of the square matrix ${\mathbf{G}}_m^H{\mathbf{G}}_m\in {\mathbb{C}}^{K_m\times K_m}$ becomes increasingly deterministic with larger L, we leverage this property of the mMIMO system to design the beamforming vector ${\mathbf{f}}_{m,k}$ using the zero-forcing (ZF) technique, expressed as follows:

(10)${\overline{\mathit{f}}}_m={\mathbf{G}}_m{\left({\mathbf{G}}_m^H{\mathbf{G}}_m\right)}^{-1},$

where ${\tilde{\mathbf{f}}}_m=[{\tilde{\mathbf{f}}}_{m,1},\dots ,{\tilde{\mathbf{f}}}_{m,K_m}]\in {\mathbb{C}}^{L\times K_m}$, ${\tilde{\mathbf{f}}}_{m,k}\in {\mathbb{C}}^{L\times 1}$, $m=0,1,\dots ,M$, $k\in K_m$. The matrix ${\tilde{\mathbf{f}}}_m$ is obtained from the ZF beamforming solution given in (10), where ${\overline{\mathit{f}}}_m$ represents the precoding matrix computed based on the pseudo-inverse of the channel matrix ${\mathbf{G}}_m$. Each column ${\tilde{\mathbf{f}}}_{m,k}$ corresponds to the beamforming vector for user k within cluster m. To ensure unit-norm beamforming vectors, we normalize ${\tilde{\mathbf{f}}}_{m,k}={\tilde{\mathbf{f}}}_{m,k}/\parallel {\tilde{\mathbf{f}}}_{m,k}\parallel$ and then compute the final beamforming vector ${\mathbf{f}}_{m,k}$ as follows:

(11)${\mathit{f}}_{m,k}=\sqrt{p_{m,k}}{\tilde{\mathit{f}}}_{m,k},m=0,1,\cdots ,M,k\in {\mathcal{K}}_m,$

where $p_{m,k}$ represents the power control coefficient of the $(m,k)$-th UE. Consequently, Equation (8) can be rewritten as follows and the intra-cell interference has been successfully canceled.

(12)$y_{m,k}=\sqrt{P_{m,k}}\sqrt{p_{m,k}}{\mathbf{G}}_{m,k}^H{\tilde{\mathit{f}}}_{m,k}{s}_{m,k}+n_0,$

Let ${\mathbf{p}}_m={\left[p_{m,k}\right]}_{k=1}^{K_m}$ represent the power control coefficients, and $\mathbf{p}={\left[{\mathbf{p}}_m\right]}_{m=0}^M$ aggregate these coefficients across all panels. Similarly, let $\mathbf{\Phi }={\left[{\Phi }_m\right]}_{m=1}^M$ denote the phase shift configuration of the RIS panels. The achievable throughput for the transmission of the $(m,k)$-th UE to the MBS can be expressed as follows:

(13)$\begin{array}{c}\hfill R_{m,k}\left(p_{m,k},{\mathbf{\Phi }}_m\right)=W{log}_2\left(1+{\displaystyle \frac{P_{m,k}{p}_{m,k}|{\mathbf{G}}_{m,k}^H{\tilde{\mathit{f}}}_{m,k}{|}^2}{{\sigma }_0^2}}\right),\end{array}$

where W is the allocated bandwidth to the $(m,k)$-th UE.

3.3. Offloading Model

In the context of computation modeling, it is assumed that a specific task of size ${\mathcal{I}}_{m,k}$ can be processed either locally by the $(m,k)$-th UE or offloaded for execution to the MEC server housed at the MBS. Accordingly, two models are introduced to characterize the computation latency, as outlined below.

3.3.1. Local Computing

Let $F_{m,k}$ denote the number of CPU cycles required to process each bit of the task for the $(m,k)$-th UE. Consequently, the time taken to execute the task locally can be calculated as follows, according to Raza et al. [21]:

(14)$T_{m,k}^l={\displaystyle \frac{{\mathcal{I}}_{m,k}{\mathcal{F}}_{m,k}}{c_{m,k}}},m=0,1,\dots ,M,k\in {\mathcal{K}}_m,$

where $c_{m,k}$ denotes the upper limit of computational resources available to the $(m,k)$-th UE.

3.3.2. Offloading to MBS

Alternatively, when the task is offloaded from the $(m,k)$-th user equipment (UE) to the MBS, the transmission time for offloading must be considered, which is given as follows, according to Raza et al. [21]:

(15)$T_{m,k}^{tx}(p_{m,k},{\mathbf{\Phi }}_m)={\displaystyle \frac{{\mathcal{I}}_{m,k}}{R_{m,k}\left(p_{m,k},{\mathbf{\Phi }}_m\right)}},$

where $R_{m,k}\left(p_{m,k},{\Phi }_m\right)$ represents the communication rate defined in (11). After the task is successfully transmitted to the MBS, the computation time required to process the offloaded task at the MBS can be expressed as follows:

(16)$T_{m,k}^{com}\left({\zeta }_{m,k}^{bs}\right)={\displaystyle \frac{{\mathcal{I}}_{m,k}{\mathcal{F}}_{m,k}}{{\zeta }_{m,k}^{bs}}},m=0,1,\dots ,M,k\in {\mathcal{K}}_m,$

where ${\zeta }_{m,k}^{\mathrm{bs}}$ represents the computing capacity of the MBS allocated to process the task for the $(m,k)$-th UE. For simplicity, let ${\zeta }_m={\left[{\zeta }_{m,k}^{\mathrm{bs}}\right]}_{k=1}^{K_m}$ and $\zeta ={\left[{\zeta }_m\right]}_{m=0}^M$ represent the overall MBS computing capacity allocation. The total latency for executing the task of the $(m,k)$-th UE can therefore be expressed as follows:

(17)$\begin{array}{cc}& T_{Total}=T_{m,k}^{tot}(p_{m,k},u_{m,k},{\mathbf{\Phi }}_m,{\zeta }_{m,k}^{bs})=(1-u_{m,k})T_{m,k}^l+\hfill \\ & +u_{m,k}\left(T_{m,k}^{tx}(p_{m,k},{\mathbf{\Phi }}_m)+T_{m,k}^{com}\left({\zeta }_{m,k}^{bs}\right)\right).\hfill \end{array}$

The time required to transmit the computation results from the MBS back to the UEs can be neglected, as this latency is significantly smaller than the total latency incurred during task execution (Raza et al. [22,23]).

4. Problem Formulation

The UAV-RIS-assisted MEC model optimization aims to minimize total latency for task execution by jointly optimizing power allocation ($\mathbf{p}$), user association ($\mathbf{u}$), RIS phase shift matrix ($\mathbf{\Phi }$), and computing resource allocation (ı). Constraints ensure the MBS computing capacity is not exceeded and QoS requirements for all users are met. The problem is formulated to balance resources and reduce latency effectively. The detailed formulation is as follows:

$\begin{array}{}\mathrm{(18)}& {\displaystyle \underset{\mathit{p},\mathbf{u},\mathbf{\Phi },\zeta }{min}\sum _{m=0}^M\sum _{k=1}^{K_m}{T}_{Total}}\hfill \mathrm{(18a)}& \mathrm{s}.\mathrm{t}.0\le p_{m,k}\le 1,\hfill \mathrm{(18b)}& R_{m,k}\left(p_{m,k},{\mathbf{\Phi }}_m\right)\ge {\overline{r}}_0,m=0,1,\cdots ,M,k\in {\mathcal{K}}_m,\hfill \mathrm{(18c)}& 0\le {\theta }_{nm}\le 2\pi ,\forall n=1,2,\dots ,N,m\in \mathcal{M},\hfill \mathrm{(18d)}& \sum _{k=1}^{K_m}{u}_{m,k}{\zeta }_{m,k}^{bs}\le {\zeta }_{max},\hfill \end{array}$

The constraints are defined as follows: $u_{m,k}$ represents the user association coefficient as per (1). Constraint (18a) specifies the allowable power range for each user, while (18b) ensures that users meet the QoS requirement in terms of a minimum achievable uplink rate (${\overline{r}}_0$). Constraint (18c) defines the valid range for each RIS phase-shift coefficient. Finally, (18d) reflects the MEC server’s computing resource limits that can be allocated to each UE.

5. Proposed Scheme

The proposed scheme aims to enhance task offloading efficiency in a UAV-RIS-assisted MEC model by systematically addressing decision-making complexities. By reformulating the problem as an MDP (Ahmed et al. [24]), we enabled a structured framework to optimize offloading decisions, power allocation, user associations, and resource distribution. The following sections outline this MDP formulation in detail:

5.1. MDP Formulation

In the UAV-RIS-assisted MEC model, we optimize the offloading decision, the power allocated by each user ($\mathbf{p}$), user association ($\mathbf{u}$), the phase shift matrix of the RIS ($\mathbf{\Phi }$), and computing resource allocation at the MBS (ı) subject to the MBS computing resource constraints and QoS requirements. The task offloading problem is modeled as an MDP, where the system state at the next time slot is determined solely by the current state and the action taken at the present time. In each time slot, the system observes the current state, selects an action based on this observation, and generates a reward that reflects the effectiveness of the action. The primary objective is to minimize system latency by utilizing an optimization scheme that effectively maps system states to latency-reducing actions.

5.1.1. Sate Space

The state captures essential information required for decision-making while avoiding redundancy. It includes three main factors: the task size (${\mathcal{I}}_{m,k}$), CPU cycles per bit (${\mathcal{F}}_{m,k}$), and the minimum required uplink rate (${\overline{r}}_0$).

The task size (${\mathcal{I}}_{m,k}$) represents the data amount that needs to be processed, measured in bits. This directly impacts the computation and transmission latency. CPU cycles per bit (${\mathcal{F}}_{m,k}$) denote the computational effort required to process one bit of data, influencing the processing time both locally and at the MEC server. The minimum required uplink rate (${\overline{r}}_0$) serves as a QoS constraint to ensure adequate communication performance. These three factors are mathematically expressed as follows:

(19)$s\left(t\right)=\{{\mathcal{I}}_{m,k},{\mathcal{F}}_{m,k},{\overline{r}}_0\}.$

These components are sufficient to derive or calculate other interrelated variables, such as path loss, LoS probability, and computational capacity.

5.1.2. Action State

The action space includes the control variables necessary for optimizing system performance. The first component is the transmission power allocation ($p_{m,k}$), which specifies the transmission power for each user and is bounded by the user’s maximum allowable power:

(20)$0\le P_{m,k}\le P_{m,k}^{\max }.$

The second component, user association ($u_{m,k}$), determines whether the user’s task is offloaded to the MEC server or processed locally. The third component involves optimizing the phase shifts (${\Phi }_m$) of the RIS panel to improve signal quality. This is expressed in (7). Lastly, the MEC resource allocation (${\zeta }_{m,k}^{\mathrm{bs}}$) determines the computational capacity assigned to each user, constrained by the MEC server’s total capacity as expressed in Equation (18d).

Thus, the action space is represented as follows:

(21)$a\left(t\right)=\{p_{m,k},u_{m,k},{\Phi }_m,{\zeta }_{m,k}^{\mathrm{bs}}\}.$

5.1.3. Reward Function

The reward function is structured to align with the objective of minimizing the total latency for all users. The total latency for a user $(m,k)$ is defined as the sum of local computation, transmission, and MEC processing times as expressed in (17). To ensure the agent minimizes this latency, the reward function includes a latency term:

(22)$r\left(t\right)=-\sum _{m=0}^M\sum _{k=1}^{K_m}{T}_{m,k}^{\mathrm{tot}}.$

5.2. DRL-Based Algorithm

The proposed DRL-based algorithm addresses the challenges of the hybrid discrete-continuous action space in the MDP, which conventional DRL methods cannot handle effectively. Converting to a discrete space causes scalability issues, while using a continuous space increases approximation complexity, both of which degrade performance. To resolve this, a novel algorithm is designed specifically for hybrid action spaces, ensuring efficient and robust decision-making.

5.2.1. Hybrid Space Modeling in UAV-RIS-Assisted MEC

In the UAV-RIS-assisted MEC model, converting the MDP into a hybrid space representation facilitates efficient optimization of the hybrid discrete-continuous action space. This approach simplifies the complexity inherent in optimizing offloading decisions, user associations, power allocation, RIS phase shifts, and MEC resource allocations while ensuring scalability and effective policy learning.

The hybrid space is constructed by first encoding the discrete components of the action space, such as user association ($u_{m,k}$) and RIS phase shifts (${\Phi }_m$). An embedding table $G_{\omega }\in {\mathbb{R}}^{N_u\times l_1}$ is utilized to map each discrete action to a continuous vector representation, ensuring that the discrete variables are represented compactly and consistently. Each embedding $g_{\omega ,u}=G_{\omega }\left(u\right)$ captures the essential features of the discrete actions, reducing the dimensionality and enabling effective integration with continuous variables.

For the continuous components of the action space, including transmission power ($p_{m,k}$) and MEC resource allocation (${\zeta }_{m,k}^{\mathrm{bs}}$), a conditional variational autoencoder (VAE) is employed. The VAE encoder $q_{\varphi }\left(\mathbf{z}\right|\mathbf{p},\mathsf{ı},s,g_{\omega ,u})$ maps these variables into a hybrid representation $\mathbf{z}\in {\mathbb{R}}^{l_2}$, conditioned on the current state $s\left(t\right)$ and the embeddings of discrete actions $g_{\omega ,u}$. This hybrid representation encapsulates the interdependencies between discrete and continuous variables, maintaining a unified and compact action representation.

The resulting hybrid representation combines the discrete embeddings and the continuous hybrid variables into a single hybrid vector:

(23)$h=[g_{\omega ,u},\mathbf{z}]\in {\mathbb{R}}^{M{l}_1+l_2}.$

This vector serves as the input to the reinforcement learning algorithm. During execution, the hybrid vector is decoded back into the original action space. Discrete actions ($u_{m,k},{\Phi }_m$) are retrieved through a nearest neighbor search in the embedding table:

(24)$u_{m,k}=arg\underset{u}{min}\left|\right|g_{\omega ,u}-g{\left|\right|}_2,$

while continuous actions ($p_{m,k},{\zeta }_{m,k}^{\mathrm{bs}}$) are reconstructed using the VAE decoder $q_{\psi }$:

(25)$\mathbf{p},\mathsf{ı}=q_{\psi }(\mathbf{z},s,g_{\omega ,u}).$

The hybrid space is trained to accurately reconstruct the original action space through a combined loss function. The reconstruction loss ensures the decoded actions match the originals:

(26)${\mathcal{L}}_{\mathrm{recon}}=\left|\right|\mathbf{p}-\widehat{\mathbf{p}}{\left|\right|}_2^2+\left|\right|\mathsf{ı}-\widehat{\mathsf{ı}}{\left|\right|}_2^2,$

and the Kullback–Leibler (KL) divergence term regularizes the hybrid variables for smooth and meaningful representations:

(27)${\mathcal{L}}_{\mathrm{KL}}=D_{\mathrm{KL}}(q_{\varphi }\left(\mathbf{z}\right|\mathbf{p},\mathsf{ı},s,g_{\omega ,u})\left|\right|\mathcal{N}(0,I)).$

The final training loss for the hybrid space is a combination of these terms:

(28)$\mathcal{L}={\mathcal{L}}_{\mathrm{recon}}+\beta {\mathcal{L}}_{\mathrm{KL}},$

where $\beta$ is a weighting parameter to balance reconstruction and regularization.

Integrating the hybrid space with reinforcement learning algorithms like TD3 ensures that the compact representation is directly used for policy and value function approximations. The actor network outputs the hybrid representation $h=[g_{\omega ,u},\mathbf{z}]$, while the critic evaluates the Q-value for the hybrid actions. This hybrid representation enables efficient optimization of the hybrid action space while preserving the relationships between discrete and continuous components.

The hybrid space significantly reduces the dimensionality of the action space, enhancing computational efficiency and convergence stability. Furthermore, it captures the relationships between discrete and continuous actions, improving the overall performance in dynamic and large-scale UAV-RIS-assisted MEC environments. This methodology not only simplifies hybrid action optimization but also ensures scalability and adaptability in complex, resource-constrained scenarios, making it ideal for minimizing latency and optimizing system performance in UAV-RIS MEC systems.

5.2.2. Long-Term Latency Minimization Algorithm for UAV-RIS-Assisted MEC Systems

The long-term latency minimization algorithm for UAV-RIS-assisted MEC systems integrates the hybrid action representation space with the TD3 algorithm to address the offloading problem. The TD3, designed for deterministic strategy, excels in handling high-dimensional continuous action spaces. Its key components include an actor network, which maps system states to actions and drives decision-making, and a critic network, which estimates rewards for specific state-action pairs to refine decisions. This integration ensures effective task offloading and optimization, aiming to minimize long-term system latency.

The proposed approach employs a hybrid action space representation to enhance decision-making in UAV-RIS-assisted MEC systems. The actor network utilizes the system state s as input and outputs a hybrid action vector $(g,z)$, where $g\in {\mathbb{R}}^{M{l}_1}$ and $z\in {\mathbb{R}}^{l_2}$. A decoder processes $(g,z)$, mapping it to a hybrid action $a=\{i_{m,k},p_{m,k}\},\forall m,k$.

The hybrid action value is approximated using twin critic networks, $Q_{{\theta }_1}$ and $Q_{{\theta }_2}$, which utilize the hybrid action a as input and evaluate its value. The training process leverages experience tuples $(s,a,r,s^{\prime })$ stored in a replay buffer D. The critic networks are optimized through the Double Q-Learning algorithm, ensuring stability and precise value approximation.

The loss function for critic networks is defined as follows:

(29)$L_{CDQ}\left({\theta }_j\right)=\mathbb{E}\left[{(\zeta -Q_{{\theta }_j}(s,g,z))}^2\right],\forall j=1,2,$

where $\zeta =r+\gamma min{Q}_{{\widehat{\theta }}_j}(s^{\prime },{\pi }_{\zeta }\left(s^{\prime }\right))$, and ${\widehat{\theta }}_j$ represent the parameters of the target network.

The actor network, employing a hybrid policy, is updated using the deterministic policy gradient method, expressed as follows:

(30)${\nabla }_{\zeta }J\left(\zeta \right)=\mathbb{E}\left[{\nabla }_{{\pi }_{\zeta }\left(s\right)}{Q}_{{\theta }_1}(s,{\pi }_{\zeta }\left(s\right)){\nabla }_{\zeta }{\pi }_{\zeta }\left(s\right)\right].$

By integrating the hybrid representation space with the TD3, we propose a novel approach to address the optimization problem in UAV-RIS-assisted MEC systems. This approach effectively balances exploration and exploitation in hybrid action spaces, ensuring robust performance in dynamic network environments.

In Algorithm 1, the actor and critic networks, along with the discrete action table and VAE parameters, are initialized to manage the hybrid action space (lines 1–2). The initial system state and replay buffer are then prepared (lines 3–4). A warm-up phase follows, where the VAE parameters are trained using reconstruction and regularization losses (lines 5–7). During each environment step, the current state is observed, and hybrid actions are generated by the actor network with added exploration noise (lines 8–10). These hybrid actions are decoded into discrete actions (e.g., user association and RIS phase shifts) and continuous actions (e.g., power and resource allocation) for execution, producing rewards and the next state (lines 11–13). Transition tuples are stored in the replay buffer for training (line 14). Critic networks are updated using loss functions, while the actor network is refined via policy gradients (lines 15–17). Representation parameters are periodically updated for efficient action decoding (lines 18–20). The algorithm iterates until convergence or the maximum steps are reached (lines 8–22). The framework of the proposed algorithm is illustrated in Figure 1.

Algorithm 1 Proposed algorithm for the UAV-RIS-assisted MEC optimization.

1:. Initialize the actor network ${\pi }_{\zeta }$ and critic networks $Q_{{\theta }_1},Q_{{\theta }_2}$ with $\zeta ,{\theta }_1,{\theta }_2$ parameters;  2:. Initialize embedding table $G_{\omega }$ and $\varphi ,\psi ,\omega$;  3:. Initialize system state $s_1$ as in Equation (19);  4:. Initialize a replay buffer, denoted as $\mathcal{D}$;  5:. while warm-up iterations not complete do  6:.     Update $\omega ,\varphi ,\psi$ with samples in $\mathcal{D}$ as per Equation (28);  7:. end while  8:. while environment steps not complete do  9:.     Monitor the current system state s; 10:.     Choose hybrid actions by actor network 11:.     $g,z={\pi }_{\zeta }\left(s\right)+{ϵ}_g,{ϵ}_g\sim \mathcal{N}(0,\sigma )$; 12:.     Decode $u_{m,k},{\Phi }_m=f_D\left(g\right),p_{m,k},{\zeta }_{m,k}^{\mathrm{bs}}=q_{\psi }(z,s,g)$ as per (20), (21), (24), and (25); 13:.     Execute $\{u_{m,k},{\Phi }_m,p_{m,k},{\zeta }_{m,k}^{\mathrm{bs}}\}$, obtain reward r as per Equation (22) and next state $s^{\prime }$; 14:.     Store $(s,g,z,r,s^{\prime })$ in $\mathcal{D}$; 15:.     Analyze hybrid actions using the critic network. 16:.     Sample mini-batch from $\mathcal{D}$; 17:.     Update $Q_{{\theta }_1},Q_{{\theta }_2}$ as per Equation (29); 18:.     Update ${\pi }_{\zeta }$ with according to Equation (30); 19:.     while training iterations not complete do 20:.         Update $\omega ,\varphi ,\psi$ using $\mathcal{D}$ as per Equation (28); 21:.     end while 22:. end while

5.2.3. Computation Complexity

The computational complexity of the proposed algorithm is analyzed across two key phases: the warm-up phase and the environment interaction phase. The warm-up phase involves updating the parameters $\omega ,\phi ,\psi$ using stored samples from the replay buffer $\mathcal{D}$ via stochastic gradient descent (SGD). Let $N_{\mathrm{warm}}$ denote the number of warm-up iterations, and assume each neural network layer has $O\left(F\right)$ parameters with L layers. The complexity of this phase is approximately $O(N_{\mathrm{warm}}·L·F^2)$.

In the environment interaction phase, the actor network ${\pi }_{\zeta }$ selects hybrid actions with a complexity of $O\left(F\right)$. Decoding and retrieving discrete actions (e.g., user association $u_{m,k}$, RIS phase shifts ${\Phi }_m$) and reconstructing continuous actions (e.g., power allocation $p_{m,k}$, computing resource allocation ${\zeta }_{m,k}^{bs}$) require matrix operations and nonlinear activations, leading to $O\left(F^2\right)$. The reward computation in (22) incurs a negligible $O\left(1\right)$ cost.

Once the transition $(s,g,z,r,s^{\prime })$ is stored in $\mathcal{D}$, the critic networks $Q_{{\theta }_1},Q_{{\theta }_2}$ update using mini-batches of size B, following (29), with a complexity of $O(B·L·F^2)$. The actor network update, as per (30), incurs the same cost, while additional training iterations contribute $O(N_{\mathrm{train}}·L·F^2)$.

Summing all contributions, the overall complexity for T environment steps can be expressed as follows:

(31)$O(N_{\mathrm{warm}}·L·F^2)+O(T·(F+F^2+B·L·F^2+N_{\mathrm{train}}·L·F^2)).$

Since the dominant term is $O(T·B·L·F^2)$, scalability depends on batch size B, network depth L, and feature count F. Additionally, the number of user equipment (UE) units K and RIS elements M directly influence the action space complexity, increasing computational overhead in action decoding and network updates.

Moreover, in large-scale mMIMO systems, ZF beamforming introduces an additional computational burden due to the required matrix inversion, resulting in a complexity of $O\left(K^3\right)$, where K is the number of UEs. This cubic complexity can significantly impact real-time processing, especially as the number of UEs grows. Despite this, the proposed framework remains computationally feasible due to its structured hybrid action space and efficient optimization strategy, making it suitable for large-scale deployments in MEC-assisted networks.

6. Numerical Results

This section provides numerical results and a discussion of our proposed algorithm in terms of the latency minimization when compared with other state-of-the-art schemes.

The simulation scenario is designed to evaluate the performance of the proposed MEC system. A massive MBS is positioned at coordinates $(0,0,30)$ meters, providing direct service to users within a 500 m radius. To extend coverage for users located up to 2000 m away, a RIS-equipped UAV operates within the area at an altitude ranging from 50 to 150 m, with user equipment (UE) randomly distributed across the extended region (Nguyen et al. [25]). The transmission parameters include an uplink transmission power of 30 dBm, a central frequency of 2.4 GHz, a bandwidth of 1 MHz, and a noise spectral density of $-130$ dBm/Hz. To ensure quality of service (QoS), the minimum achievable uplink rate per UE is set at 1 Mbps. The computational tasks involve a task size of 100 kB with a computational complexity of 600 cycles per bit. The MEC server, hosted at the MBS, has a maximum processing capacity of 30 Giga cycles per second, while each UE is limited to a maximum of 0.5 Giga cycles per second. This setup provides a realistic and challenging environment for evaluating task offloading and resource allocation in the RIS-assisted UAV-enabled MEC framework (Liu et al. [26]). The detailed configuration of the simulation parameters is presented in Table 1.

The proposed scheme is compared with the following baseline schemes.

1. Deep Q-network (DQN) (Khan et al. [27]): DQN algorithm is designed for discrete action spaces, where a neural network approximates the Q-value function to derive optimal policies.

2. Soft actor–critic (SAC) (Heidarpour et al. [28]): SAC is a model-free, off-policy algorithm optimized for continuous action spaces. It employs a stochastic policy with an entropy-based objective to balance exploration and exploitation. While SAC performs well in purely continuous environments.

3. Multi-Agent deep deterministic policy gradient (MADDPG) (Tariq et al. [3]): MADDPG extends deterministic policy gradient methods to multi-agent systems, facilitating coordinated decision-making among RIS-equipped UAVs and MEC servers.

4. Proposed scheme: The proposed approach employs a hybrid discrete-continuous optimization framework with hybrid space representation to optimize task offloading, power allocation, and RIS phase configurations. By integrating a conditional variational autoencoder and the TD3 algorithm, it addresses hybrid action spaces.

Figure 2 illustrates the average reward progression for various DRL algorithms, i.e., the proposed scheme, DQN, SAC, and MADDPG over 2000 training episodes. The average reward reflects the capability of each method in addressing the task offloading and resource allocation challenges in a UAV-RIS-assisted MEC environment. The results demonstrate the superiority of the proposed scheme, which achieves the highest average reward by the end of the training. This performance is attributed to its hybrid discrete-continuous optimization framework, effectively utilizing hybrid representations and advanced policy learning to navigate complex decision spaces. Among the baseline methods, the MADDPG algorithm shows competitive performance due to its multi-agent coordination capabilities. However, its effectiveness is hindered by inefficiencies in handling hybrid action spaces, an area where the proposed scheme excels. SAC achieves moderate results, optimized for continuous actions but constrained by its lack of inter-agent coordination. Meanwhile, DQN, with its discrete action space, shows the weakest performance, highlighting its limitations in handling the complexity and scale of the optimization problem. For clarity, Figure 2 incorporates shaded confidence intervals that represent the variability in rewards across multiple training trials. These confidence intervals highlight the stability of each algorithm, with the proposed scheme exhibiting a narrower range of fluctuations, indicating more consistent performance. MADDPG and SAC show moderate variance, while DQN not only achieves the lowest average reward but also displays the widest confidence interval, reinforcing its instability in dynamic UAV-RIS-MEC environments. Despite varying performance levels, all algorithms demonstrate steady learning progression, with rewards increasing as training episodes advance. However, the faster convergence, higher average reward, and reduced variability of the proposed scheme underscore its robustness for latency-critical, resource-intensive applications. These findings emphasize the significance of hybrid optimization in dynamic MEC environments, where decision efficiency and stability are crucial.

In Figure 3, the outperformance of the proposed method compared with the state-of-the-art schemes is demonstrated in terms of total network latency. For instance, at $F_{m,k}=600$, the total latency with the proposed scheme and the benchmark schemes are approximately 28.8 ms, 32.2 ms, 40.8 ms, and 49.3 ms for the proposed scheme, MADDPG, SAC, and DQN, respectively. Moreover, it is evident from the figure that as $F_{m,k}$ increases, the total network latency for all schemes grows, yet the proposed scheme consistently achieves the lowest latency. This performance gap highlights the efficiency of the proposed approach in minimizing latency. To this end, it is worth mentioning that these results are valid within the range of considered CPU cycles per bit ($F_{m,k}$) values, demonstrating the robustness of the proposed method across varying computational demands.

In Figure 4, the total network latency is evaluated for the proposed scheme in comparison with other benchmark schemes across varying computing capacities of the MBS (${\zeta }_{max}$) measured in megacycles per second. The evaluation considers a fixed number of UEs ($K=100$) and CPU cycles per bit ($F_{m,k}=600$ cycles/bit). Figure 4 illustrates that increasing the computing capacity of the MBS (${\zeta }_{max}$) leads to a noticeable reduction in total network latency for all schemes. This trend underscores the critical role of computational resources at the MBS in minimizing latency. Among the evaluated schemes, the proposed method consistently outperforms the benchmark schemes (DQN, SAC, and MADDPG), achieving the lowest latency across all considered values of ${\zeta }_{max}$. Moreover, the latency reduction with the proposed scheme is more pronounced compared to the other schemes, highlighting its efficiency in leveraging enhanced computational resources.

Figure 5 demonstrates the impact of varying the number of UEs on total task delay, with user counts ranging up to a maximum of 300 and RIS $N=100$. The results reveal a clear trend: as the number of UEs increases, the total time delay rises significantly. Among the evaluated algorithms, the proposed scheme consistently achieves the lowest task delay across all scenarios, highlighting its superior ability to handle high user densities efficiently. In contrast, DQN exhibits the highest delay, reflecting its limited scalability for large-scale environments. SAC and MADDPG perform moderately, but their delays grow more sharply with increasing UEs compared to the proposed scheme. This analysis underscores the proposed scheme’s robustness and scalability in managing resource allocation and task offloading in densely populated MEC systems.

Figure 6 illustrates the task completion ratio as a function of the number of UEs in the system. The task completion ratio is defined as the percentage of tasks completed within the predefined delay threshold relative to the total number of tasks. As shown in Figure 6, the completion ratio decreases with an increasing number of UEs due to resource limitations within the UAV-RIS-assisted MEC framework. Specifically, the proposed scheme demonstrates the highest task completion ratio across all user densities, showcasing its robust capability to prioritize high-priority tasks and manage resource allocation effectively. In contrast, the benchmark algorithms—DQN, SAC, and MADDPG—exhibit significantly lower completion ratios as the number of UEs grows, highlighting their limitations in handling resource scarcity and task prioritization. The proposed scheme’s superior performance stems from its hybrid action space optimization and reward mechanism, prioritizing high-priority tasks. This scheme improves overall completion ratios by allocating more resources to critical tasks, ensuring effectiveness across varying UE densities.

7. Conclusions

In this paper, we proposed an MEC system hosted within a massive MIMO base station, serving M groups of users with the assistance of a RIS-equipped UAV to enhance system coverage and performance. To optimize this communication scenario, we formulated the problem of minimizing total task execution latency across all UEs as an MDP and developed a novel DRL algorithm, leveraging a hybrid space representation. This approach effectively reduced latency while ensuring efficient resource utilization. Extensive simulations demonstrated the effectiveness of our approach in improving system performance. Specifically, our framework achieved a latency reduction of 10.56%, 29.41%, and 41.62% compared to MADDPG, SAC, and DQN, respectively. Additionally, it improved the task completion ratio by 12.16%, 40.68%, and 56.60% over MADDPG, SAC, and DQN, respectively. These results validate the superiority of the proposed method over state-of-the-art schemes, demonstrating its potential for next-generation communication networks. Moreover, the findings highlight the scalability of the proposed solution. As the number of UEs and RIS elements increases, our algorithm maintains a practical computational complexity, as analyzed in Section 5.2.3. Despite these promising outcomes, certain limitations remain. The model assumes idealized channel conditions, which may not fully capture real-world deployments where environmental uncertainties affect performance. In future work, we aim to develop adaptive solutions that incorporate real-time feedback to handle dynamic environments and fluctuating system states. Furthermore, the framework will be extended to consider energy efficiency and security as key performance metrics, enhancing its applicability to diverse 6G use cases.

Data Availability Statement

No new data were created or analyzed in this study.

Conflicts of Interest

The author declares no conflicts of interest.

Abbreviations

The following abbreviations are used in this manuscript:

DRL	deep reinforcement learning
IoT	Internet of Things
ITU	International Telecommunication Union
LoS	line-of-sight
MEC	multi-access edge computing
MDP	Markov decision process
MBS	MIMO base station
QoS	quality of service
RISs	reconfigurable intelligent surfaces
TDMA	time division multiple access
UAVs	unmanned aerial vehicles
UE	user equipment

Footnotes

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Figures and Table

Figure 1. Framework of the proposed algorithm.

Figure 2. Average rewards vs. no. of episodes.

Figure 3. The total time delay according to different schemes vs. [Forumla omitted. See PDF.]m,k, with K = 100 and [Forumla omitted. See PDF.] = 30 Giga cycles/s.

Figure 4. The total time delay according to different schemes vs. [Forumla omitted. See PDF.], with K = 100 and [Forumla omitted. See PDF.] = 600 cycles/bit.

Figure 5. The total time delay vs. no. of UEs.

Figure 6. Task completion ratio vs. no. of UEs.

Table 1

Simulation parameters.

Parameter	Value
MBS coverage radius for direct connection	500 m
Maximum distance of users from MBS	2000 m
3D Cartesian coordinates of the MBS	$(0,0,30)$
Altitude range of RIS-equipped UAV $(H^{\min },H^{\max })$	$(50,150)$ m
Maximum transmission power ($P_{m,k}^{\max }$)	30 dBm
Central frequency ($f_c$)	2.4 GHz
Bandwidth (W)	1 MHz
White noise spectral density (${\sigma }_0^2$)	$-130$ dBm/Hz
Minimum achievable uplink rate (${\tilde{r}}_0$)	1 Mbps
Task size (${\mathcal{I}}_{m,k}$)	100 kB
Task computation complexity (${\mathcal{F}}_{m,k}$)	600 cycles/bit
Maximum MEC server computing resource (${\zeta }_{\max }$)	30 Giga cycles/s
Maximum UE computing resource ($c_{m,k}$)	0.5 Giga cycles/s

References

1. Measuring Digital Development—Facts and Figures 2023. Available online: https://www.itu.int/hub/publication/d-ind-ict_mdd-2023-1/ (accessed on 20 January 2025).

2. Ahmed, M.; Raza, S.; Soofi, A.A.; Khan, F.; Khan, W.U.; Xu, F.; Chatzinotas, S.; Dobre, O.A.; Han, Z. A survey on reconfigurable intelligent surfaces assisted multi-access edge computing networks: State of the art and future challenges. Comput. Sci. Rev.; 2024; 54, 100668. [DOI: https://dx.doi.org/10.1016/j.cosrev.2024.100668]

3. Tariq, M.N.; Wang, J.; Raza, S.; Siraj, M.; Altamimi, M.; Memon, S. Towards Optimal Resource Allocation: A Multi-agent DRL based Task Offloading Approach in Multi-UAV-Assisted MEC Networks. IEEE Access; 2024; 12, pp. 81428-81440. [DOI: https://dx.doi.org/10.1109/ACCESS.2024.3411022]

4. Ahmed, M.; Raza, S.; Soofi, A.A.; Khan, F.; Khan, W.U.; Abideen, S.Z.U.; Xu, F.; Han, Z. Active reconfigurable intelligent surfaces: Expanding the frontiers of wireless communication—A survey. IEEE Commun. Surv. Tutor.; 2024; [DOI: https://dx.doi.org/10.1109/COMST.2024.3423460]

5. Mehrabian, A.; Wong, V.W. Joint Spectrum, Precoding, and Phase Shifts Design for RIS-Aided Multiuser MIMO THz Systems. IEEE Trans. Commun.; 2024; 72, pp. 5087-5101. [DOI: https://dx.doi.org/10.1109/TCOMM.2024.3375816]

6. Xu, F.; Ahmad, S.; Ahmed, M.; Raza, S.; Khan, F.; Ma, Y.; Khan, W.U. Beyond encryption: Exploring the potential of physical layer security in UAV networks. J. King Saud Univ.-Comput. Inf. Sci.; 2023; 35, 101717. [DOI: https://dx.doi.org/10.1016/j.jksuci.2023.101717]

7. Banafaa, M.K.; Pepeoğlu, Ö.; Shayea, I.; Alhammadi, A.; Shamsan, Z.A.; Razaz, M.A.; Alsagabi, M.; Al-Sowayan, S. A comprehensive survey on 5G-and-beyond networks with UAVs: Applications, emerging technologies, regulatory aspects, research trends and challenges. IEEE Access; 2024; 12, pp. 7786-7826. [DOI: https://dx.doi.org/10.1109/ACCESS.2023.3349208]

8. Xu, Y.; Zhang, T.; Zou, Y.; Liu, Y. Reconfigurable intelligence surface aided UAV-MEC systems with NOMA. IEEE Commun. Lett.; 2022; 26, pp. 2121-2125. [DOI: https://dx.doi.org/10.1109/LCOMM.2022.3183285]

9. Hu, H.; Sheng, Z.; Nasir, A.A.; Yu, H.; Fang, Y. Computation Capacity Maximization for UAV and RIS Cooperative MEC System with NOMA. IEEE Commun. Lett.; 2024; 28, pp. 592-596. [DOI: https://dx.doi.org/10.1109/LCOMM.2024.3357752]

10. Wang, Y.; Niu, J.; Chen, G.; Zhou, X.; Li, Y.; Liu, S. RIS-Aided Latency-Efficient MEC HetNet with Wireless Backhaul. IEEE Trans. Veh. Technol.; 2024; 73, pp. 8705-8719. [DOI: https://dx.doi.org/10.1109/TVT.2024.3354371]

11. Ning, Z.; Dong, P.; Wen, M.; Wang, X.; Guo, L.; Kwok, R.Y.; Poor, H.V. 5G-enabled UAV-to-community offloading: Joint trajectory design and task scheduling. IEEE J. Sel. Areas Commun.; 2021; 39, pp. 3306-3320. [DOI: https://dx.doi.org/10.1109/JSAC.2021.3088663]

12. Song, F.; Xing, H.; Wang, X.; Luo, S.; Dai, P.; Xiao, Z.; Zhao, B. Evolutionary multi-objective reinforcement learning based trajectory control and task offloading in UAV-assisted mobile edge computing. IEEE Trans. Mob. Comput.; 2022; 22, pp. 7387-7405. [DOI: https://dx.doi.org/10.1109/TMC.2022.3208457]

13. Xu, B.; Kuang, Z.; Gao, J.; Zhao, L.; Wu, C. Joint offloading decision and trajectory design for UAV-enabled edge computing with task dependency. IEEE Trans. Wirel. Commun.; 2022; 22, pp. 5043-5055. [DOI: https://dx.doi.org/10.1109/TWC.2022.3231408]

14. Goudarzi, S.; Soleymani, S.A.; Wang, W.; Xiao, P. Uav-enabled mobile edge computing for resource allocation using cooperative evolutionary computation. IEEE Trans. Aerosp. Electron. Syst.; 2023; 59, pp. 5134-5147. [DOI: https://dx.doi.org/10.1109/TAES.2023.3251967]

15. Bai, Z.; Lin, Y.; Cao, Y.; Wang, W. Delay-Aware Cooperative Task Offloading for Multi-UAV Enabled Edge-Cloud Computing. IEEE Trans. Mob. Comput.; 2024; 23, pp. 1034-1049. [DOI: https://dx.doi.org/10.1109/TMC.2022.3232375]

16. Zhao, N.; Ye, Z.; Pei, Y.; Liang, Y.C.; Niyato, D. Multi-Agent Deep Reinforcement Learning for Task Offloading in UAV-Assisted Mobile Edge Computing. IEEE Trans. Wirel. Commun.; 2022; 21, pp. 6949-6960. [DOI: https://dx.doi.org/10.1109/TWC.2022.3153316]

17. Zhai, Z.; Dai, X.; Duo, B.; Wang, X.; Yuan, X. Energy-efficient UAV-mounted RIS assisted mobile edge computing. IEEE Wirel. Commun. Lett.; 2022; 11, pp. 2507-2511. [DOI: https://dx.doi.org/10.1109/LWC.2022.3206587]

18. Di Renzo, M.; Zappone, A.; Debbah, M.; Alouini, M.S.; Yuen, C.; De Rosny, J.; Tretyakov, S. Smart radio environments empowered by reconfigurable intelligent surfaces: How it works, state of research, and the road ahead. IEEE J. Sel. Areas Commun.; 2020; 38, pp. 2450-2525. [DOI: https://dx.doi.org/10.1109/JSAC.2020.3007211]

19. Al-Hourani, A.; Kandeepan, S.; Lardner, S. Optimal LAP altitude for maximum coverage. IEEE Wirel. Commun. Lett.; 2014; 3, pp. 569-572. [DOI: https://dx.doi.org/10.1109/LWC.2014.2342736]

20. Wu, Q.; Zhang, R. Beamforming optimization for wireless network aided by intelligent reflecting surface with discrete phase shifts. IEEE Trans. Commun.; 2019; 68, pp. 1838-1851. [DOI: https://dx.doi.org/10.1109/TCOMM.2019.2958916]

21. Raza, S.; Liu, W.; Ahmed, M.; Anwar, M.R.; Mirza, M.A.; Sun, Q.; Wang, S. An efficient task offloading scheme in vehicular edge computing. J. Cloud Comput.; 2020; 9, pp. 1-14. [DOI: https://dx.doi.org/10.1186/s13677-020-00175-w]

22. Raza, S.; Wang, S.; Ahmed, M.; Anwar, M.R.; Mirza, M.A.; Khan, W.U. Task offloading and resource allocation for IoV using 5G NR-V2X communication. IEEE Internet Things J.; 2021; 9, pp. 10397-10410. [DOI: https://dx.doi.org/10.1109/JIOT.2021.3121796]

23. Raza, S.; Ahmed, M.; Ahmad, H.; Mirza, M.A.; Habib, M.A.; Wang, S. Task offloading in mmwave based 5g vehicular cloud computing. J. Ambient Intell. Humaniz. Comput.; 2023; 14, pp. 12595-12607. [DOI: https://dx.doi.org/10.1007/s12652-022-04320-y]

24. Ahmed, M.; Raza, S.; Ahmad, H.; Khan, W.U.; Xu, F.; Rabie, K. Deep reinforcement learning approach for multi-hop task offloading in vehicular edge computing. Eng. Sci. Technol. Int. J.; 2024; 59, 101854. [DOI: https://dx.doi.org/10.1016/j.jestch.2024.101854]

25. Nguyen, L.D.; Tuan, H.D.; Duong, T.Q.; Dobre, O.A.; Poor, H.V. Downlink beamforming for energy-efficient heterogeneous networks with massive MIMO and small cells. IEEE Trans. Wirel. Commun.; 2018; 17, pp. 3386-3400. [DOI: https://dx.doi.org/10.1109/TWC.2018.2811472]

26. Liu, T.; Tang, L.; Wang, W.; Chen, Q.; Zeng, X. Digital-twin-assisted task offloading based on edge collaboration in the digital twin edge network. IEEE Internet Things J.; 2021; 9, pp. 1427-1444. [DOI: https://dx.doi.org/10.1109/JIOT.2021.3086961]

27. Khan, I.; Raza, S.; Rehman, W.u.; Khan, R.; Nahida, K.; Tao, X. A Deep Learning-Based Algorithm for Energy and Performance Optimization of Computational Offloading in Mobile Edge Computing. Wirel. Commun. Mob. Comput.; 2023; 2023, 1357343. [DOI: https://dx.doi.org/10.1155/2023/1357343]

28. Heidarpour, A.R.; Heidarpour, M.R.; Ardakani, M.; Tellambura, C.; Uysal, M. Soft actor–critic-based computation offloading in multiuser MEC-enabled IoT—A lifetime maximization perspective. IEEE Internet Things J.; 2023; 10, pp. 17571-17584. [DOI: https://dx.doi.org/10.1109/JIOT.2023.3277753]