Content-Aware Adaptive Dense Communication Network: Enhancing Multi-LLM Collaboration with Dynamic Information Flow

Multi-agent systems built upon large language models (LLMs) are hindered by the limitations of traditional token-based communication, which suffers from information bottlenecks, redundant processing, and a lack of end-to-end differentiability. While dense vector communication offers improvements, existing methods lack adaptivity to diverse task requirements. To address this, we propose the Content-Aware Adaptive Dense Communication Network (CADCN), a novel architecture that empowers LLM agents to communicate through dense vectors by dynamically perceiving content and adaptively selecting routing and transformation strategies. CADCN introduces the Content-Aware Communication Unit (CACU), which integrates Content-Aware Routing via a lightweight network and Adaptive Dense Transformation using a Mixture-of-Experts structure, ensuring full differentiability. Our experiments, conducted under a highly constrained training token budget, demonstrate that CADCN consistently achieves superior performance across diverse general knowledge, reasoning, mathematical, and coding benchmarks compared to prior dense communication approaches. Furthermore, CADCN significantly outperforms vanilla LLMs trained on vastly larger datasets, highlighting its remarkable data efficiency and capability expansion. Ablation studies confirm the synergistic contributions of CADCN's adaptive components, while analysis of communication dynamics reveals learned expert specialization. Our findings establish CADCN as a highly efficient and intelligent paradigm for robust multi-LLM collaboration.