Improving energy autonomy of positive energy districts using multi-agent deep reinforcement learning

Abstract

In recent years, Positive Energy Districts (PEDs) have emerged at the forefront of urban innovation, rapidly transforming communities by integrating shared Energy Storage Systems (ESS) and Electric Vehicles (EVs) to redefine the future of sustainable communities. However, energy management in such communities remains extremely challenging due to the dynamic nature of EV availability, unpredictable renewable energy generation, and the necessity to maintain user comfort while optimizing energy use. Overcoming these challenges is critical for enabling PEDs to achieve carbon neutrality, reduce costs, and improve energy sharing. In addition, Vehicle-to-Grid (V2G) technology and shared ESS offer unique opportunities to optimize energy consumption and facilitate access to the open energy market, but fully exploiting their potential requires advanced strategies such as Deep Reinforcement Learning (DRL). To address these needs, this work proposes a novel Community Multi-Agent Deep Reinforcement Learning Vehicle-to-Grid (CoMAD V2G) solution based on Multi-Agent Reinforcement Learning (MARL), which enhances the utilization of community-generated energy and increases community autonomy by controlling the charging and discharging cycles of V2G-enabled EVs. Real data on household consumption, solar energy production, EV dynamics, and electricity prices are used to evaluate and verify the effectiveness of the proposed solution in a realistic environment. Under these conditions, the proposed solution achieves improved energy exchange with the external grid on an annual basis, a result not attained with comparable conventional heuristic or alternative learning-based approaches for the community under consideration. Furthermore, the solution reduces household electricity costs by up to 25%, highlighting its potential to deliver significant economic and sustainability benefits for PEDs.