Virtual Machine Placement in Edge Computing Based on Multi-Objective Reinforcement Learning

Abstract

With the popularization of internet of things (IoT), the energy consumption of mobile edge computing (MEC) servers is also on the rise. Some important IoT applications, such as autonomous driving, smart manufacturing, and smart wearables, have high real-time requirements, making it imperative for edge computing to reduce task response latency. Virtual machine (VM) placement can effectively reduce the response latency of VM requests and the energy consumption of MEC servers. However, the existing work does not consider the selection of weighting coefficients for the optimization objectives and the feasibility of the solution. Besides, these algorithms scalarize the objective functions without considering the order-of-magnitude difference between objectives. To overcome the above problems, the article proposes an algorithm called EVMPRL for VM placement in edge computing based on reinforcement learning (RL). Our aim is to find the Pareto approximate solution set that achieves the trade-off between the response latency of VM requests and the energy consumption of MEC servers. EVMPRL is based on the Chebyshev scalarization function, which is able to efficiently solve the problem of selecting weighting coefficients for objectives. EVMPRL can always search for solutions in the feasible domain, which can be guaranteed by selecting the servers that can satisfy the current VM request as the next action. Furthermore, EVMPRL scalarizes the Q-values instead of the objective functions, thus avoiding the problem in previous work where the order-of-magnitude difference between the optimization objectives makes the impact of an objective function on the final result too small. Finally, we conduct experiments to prove that EVMPRL is superior to the state-of-the-art algorithm in terms of objectives and the solution set quality.