Novel Middleware Framework for Integrating Extended Reality into Robotic Manufacturing Processes

The integration of extended reality (XR) into industrial robotics requires robust middleware solutions capable of bridging heterogeneous systems, protocols, and user interactions. This paper presents a novel middleware framework designed to connect industrial robots with XR devices such as the HoloLens. The architecture employs a hybrid communication layer that combines MQTT (Message Queuing Telemetry Transport) and ØMQ (Zero Message Queue), leveraging the Sparkplug Robotics API model for robot data and publisher–subscriber streaming for XR camera feeds. A Redis cache database is introduced to ensure efficient data handling and prevent data corruption. On the robot side, the system is built on ROS 2 (Robot Operating System) and connects to proprietary industrial protocols through dedicated bridges, enabling seamless interoperability. Spatial alignment between physical robots and XR overlays is achieved using ArUco marker-based synchronization, while real-time kinematic and process data are visualized directly in XR. The middleware further supports bidirectional interaction, allowing users to adjust parameters and issue commands through XR devices. Beyond functionality, safety considerations are incorporated by integrating human–robot interaction safeguards and ensuring compliance with industrial communication standards. The proposed solution demonstrates how middleware-driven XR integration enhances transparency, control, and safety in robotic manufacturing processes, laying the foundation for greater efficiency and adaptability in Industry 4.0 environments.