Balancing Communication and Acceleration: Exact One-to-One Optimization for Distributed Multiagent Learning Systems

This article investigates optimization-driven learning techniques to address the critical challenge of balancing communication efficiency with convergence acceleration in distributed multiagent systems. While existing accelerated methods typically necessitate multiple internode communications per iteration, we propose two novel methods, heavy-ball exact fusion (HBEF) and Nesterov-accelerated exact fusion (NAEF), that maintain a single-communication operation while achieving enhanced convergence. By fusing momentum mechanisms with bias correction, the developed methods not only precisely preserve convergence guarantees but also demonstrate accelerated convergence compared to baseline exact diffusion and contemporary accelerated counterparts. A distinctive dual acceleration method is further proposed through momentum parameter coordination. Rigorous convergence analysis reveals the momentum parameter's critical role in acceleration behavior. Extensive numerical evaluations across representative machine learning tasks validate the proposed methods' superiority in both transient convergence speed and steady-state accuracy. Notably, they achieve state-of-the-art performance at half or one-third the communication cost, effectively bridging the long-standing gap between communication efficiency and rapid convergence in distributed learning.