State of charge estimation for a UAV battery via Adaptive Extended Kalman Filter and Deep Reinforcement Learning

This study presents the development of an accurate State of Charge (SoC) estimation method for lithium-ion batteries used in mission- and safety-critical applications. The proposed approach integrates an Adaptive Extended Kalman Filter (AEKF) with Deep Reinforcement Learning (DRL) in order to enhance estimation accuracy under dynamically changing operating conditions. Within this framework, AEKF implementations based on 1RC, 2RC, and 3RC battery models are examined, where the 1RC model may be insufficient to capture the battery’s fully-fledged characteristics, the 2RC model offers a balanced trade-off between accuracy and complexity, and the 3RC model enables a more comprehensive representation of the battery dynamics. A policy learned via deep reinforcement learning is used at deployment time to tune the statistics of the update and measurement models of the state estimator depending on operating conditions of interest. Unlike conventional approaches that rely on fixed or heuristically tuned noise covariance parameters, the proposed tuning policy exploits patterns learned at training time to effectively tune the noise covariance parameters to better ensure the convergence of the EKF under time-varying operating conditions for which conventional approaches may fail. In the proposed method, the Process Covariance Matrix ( Q p ) and the Measurement Covariance Matrix ( R m ) are adaptively estimated using Proximal Policy Optimization (PPO)-based policy learning within a predefined clipped range. As operating conditions change, the filter updates its covariance parameters through a learning-based adaptation mechanism, improving the robustness and accuracy of SoC estimation. The proposed method is evaluated by using real UAV flight data obtained during operational missions. The results demonstrate that the PPO-based AEKF improves SoC estimation accuracy compared with that of the traditional EKF approach. A comparative evaluation of the investigated models shows that for the 2RC case, the learning-based AEKF (RL-AEKF) configuration provides the highest SoC estimation accuracy among all three battery models and the two filtering approaches, namely EKF and AEKF.