Graph-based reinforcement learning for software-defined networking traffic engineering

Abstract

With the continuous expansion of global Internet infrastructure, wide area networks play a crucial role in transmitting traffic between multiple data centers and users worldwide. However, efficient traffic management has become a core challenge due to the high costs of building and maintaining these networks. Traditional traffic engineering methods based on linear programming achieve optimal solutions but suffer from exponential computational complexity growth with network size, making them impractical for real-time applications in large-scale networks. Recent machine learning approaches show promise but still face fundamental limitations in handling complex network constraints and maintaining performance across different network scales. This paper proposes GRL-TE (Graph-based Reinforcement Learning for Traffic Engineering), a novel framework that achieves near-optimal performance while maintaining computational efficiency across diverse network scales. GRL-TE introduces three key innovations: (1) TopoFlowNet, a graph neural network architecture that models WANs as bipartite graphs with edge nodes representing physical links and path nodes representing candidate paths, enabling efficient bidirectional information propagation through GINConv layers while MLP modules handle collaborative relationships among paths serving the same demand; (2) A one-step A2C mechanism specifically designed for TE with immediate reward structure, eliminating the need for future state estimation and significantly simplifying training; (3) Integration of ADMM as a post-processing step to iteratively reduce constraint violations while improving solution quality. Extensive experiments on six real-world WAN topologies ranging from 12 to 1,739 nodes demonstrate that GRL-TE achieves an overall average demand satisfaction rate of 89.36%, outperforming state-of-the-art learning-based methods (Teal: 82.04%, Figret: 82.20%) and the clustering-based NCFlow (76.48%), while providing 3-4 orders of magnitude speedup compared to LP solvers on large-scale networks. The framework maintains robust performance under link failures and meets real-time scheduling requirements for production deployment.