RL4Sys: A Lightweight System-driven RL Framework for Drop-in Integration in System Optimization

Reinforcement learning (RL) has achieved notable success in complex decision-making tasks across various domains, including games, autonomous driving, and robotics. Motivated by these advances, systems researchers have explored RL for optimizing system behavior. However, practical deployment remains uncommon, as existing RL frameworks are ill-suited for system-oriented use cases, introducing substantial integration and runtime overhead in system software. To address this gap, we present RL4Sys, a lightweight RL framework designed specifically for seamless system-level integration. RL4Sys includes a minimal client that embeds easily within target systems to record trajectories and run inference from locally cached deep policies. RL4Sys’s remote RL trainers are executed asynchronously and distributed across servers, leveraging zero-copy gRPC and adaptive batching to update policies without blocking the original system. Our evaluation shows that RL4Sys matches the convergence behavior of conventional RL frameworks and achieves up to 220% higher throughput in environment-oriented settings compared to the state-of-the-art framework, RLlib, while incurring less than 6% runtime overhead relative to the original non-RL system. These results indicate that RL4Sys substantially lowers the barrier to deploying RL for real-world systems optimization.