Physics-Constrained Adversarial Attack Generation and Active Defense for the Hybrid Deep Reinforcement Learning-Based Load Frequency Control

With the transition to Industry 5.0, there is a growing demand to deploy highly autonomous and resilient artificial intelligence (AI) systems in critical infrastructures such as power grids. In the field of load frequency control (LFC) in power grids, a hybrid control architecture in which deep reinforcement learning (DRL) controllers coexist with traditional proportionalintegral-derivative (PID) controllers can become a typical deployment model during this technological transition. However, the inherent vulnerability of DRL controllers to adversarial attacks introduces new security challenges in such complex environments: attacks not only affect the DRL-controlled areas but may also propagate to PID-controlled areas through inter-area power exchanges, potentially causing broader system instability. To accurately assess the cascading risks under this hybrid architecture, we propose a physics-constrained adversarial attack framework to simulate realistic threats targeting DRL controllers that can propagate across areas. First, we design and implement three typical hybrid control scenarios, i.e., single-agent DRL, partial-agent DRL, and full-agent DRL. Second, we propose a key-feature selection method based on gradient saliency, and we design an attack strategy that adheres to physical constraints while maintaining stealth and efficiency. Third, to enhance the system’s resiliency, we propose a two-stage active defense strategy highly compatible with the hybrid architecture. Finally, we conduct extensive simulation experiments under three typical hybrid control scenarios to evaluate the impact of the adversarial attack and the performance of the defense strategy.