A Multi-Modal Machine Learning Architecture for Resource-Efficient Sensing and Sustainable Edge Intelligence

Intelligent sensing at the network edge is a tricky issue, even though it is not an easy endeavor to try to maximize accuracy but is rather a skirmish against limited resources. Embedded systems are identifying increased sensors and are becoming omnipresent and the real-time and multi-modal interpretation is booming, rendering traditional and cloud-reliant or computationally intensive machine learning models ineffective. It thus requires the creation of architecture that will handle this wilderness of limited compute and energy in real-time, not monolithic models that have been transplanted out of data centers. The current paper constitutes a computational framework of multi-modal learning at the edge straightforwardly addressing the issue of the efficiency-accuracy trade-off. We do not consider the highly complex suite of WSM-2023 streams of benchmarks as the very classification tasks but instead, an approximation of the rough and rugged and changing sensory landscape of actual deployments. More specifically, we rely on the Controlled Optimization Procedure (COP) which specializes in a rigid comparison of three multi-modal fusion approaches, i.e. Early Fusion, Late Fusion as well as on the Adaptive Gating based Hierarchical Fusion which feature algorithmic paradigms capable of synthesizing information retrieved via various sensors without advance plan of action fusion. Using both intensive statistical and energetic analysis, we show that, though each of the fusion strategies has its strength, the final decision here is that the Adaptive Gating-based Hierarchical Fusion provides better computational efficiency and adaptive robustness and how it can be reconfigured to operate in more degraded and variable sensory situations. The work forms the original merit of adaptive and context-sensitive architecture of complex implementations of sustainable edge intelligence and provides a viable roadmap to follow when selecting perceptual system, sense and reason rather than just manipulation data through a preset and rigidly programmed algorithm.