Optimizing Power Grid Topologies with Reinforcement Learning: A Survey of Methods and Challenges (Foundations and Trends® in Engineering)

Electrical power grids form the backbone of modern society, being responsible for transporting electricity from producers to consumers 24 hours a day, 365 days a year. Operating these grids is a demanding control task that requires continuous monitoring and frequent interventions by skilled experts to maintain network stability, keep power flow within the thermal limits of the equipment, and ensure voltage and frequency levels are met. Power grid operation is becoming increasingly complex due to the rising integration of renewable energy sources and the need for more adaptive control strategies. Reinforcement Learning (RL) has emerged as a promising approach to power network control, offering the potential to enhance decision-making in dynamic and uncertain environments. The Learning To Run a Power Network (L2RPN) competitions have played a key role in accelerating research by providing standardized benchmarks and problem formulations, leading to rapid advancements in RL-based methods. This monograph provides a comprehensive and structured overview of RL applications for power grid topology optimization, categorizing existing techniques, highlighting key design choices, and identifying gaps in current research. Additionally, a comparative numerical study evaluating the impact of commonly applied RL-based methods is presented, offering insights into their practical effectiveness. By consolidating existing research and outlining open challenges, this work aims to provide a foundation for future advancements in RL-driven power grid optimization.