HumAIne Benchmarking Suite: Evaluation of Human–AI Collaboration

This presentation introduces the HumAIne Benchmarking Suite, a comprehensive framework designed to evaluate the quality of Human-AI Collaboration (HAIC). Moving beyond traditional model-centric metrics like accuracy and latency, the suite provides a structured methodology to assess the actual collaboration process, measuring indicators such as system-level efficiency, user trust, cognitive load, and interaction frequency. The slides detail the architecture of the platform, the standardized logging schema used across diverse domains (including healthcare and smart cities), and its specialized modules for fairness evaluation, simulation, and system usabilit