Rethink Repeatable Measures of Robot Performance With Statistical Query

A standardized robot-testing algorithm should satisfy accuracy, efficiency, and very importantly, repeatability-consistently yielding similar outcomes across multiple executions by different stakeholders. Achieving repeatability grows challenging with increasing complexity, intelligence, diversity, and inherent stochasticity in testing methods, robotic platforms, and environments. While existing efforts address repeatability through ethical, hardware, or procedural means, this study specifically targets algorithm-level repeatability, focusing on statistical query (SQ) algorithms commonly used in standardized evaluations. We propose a lightweight, adaptive modification for any SQ-based routine, including Monte Carlo, importance sampling, and adaptive sampling, guaranteeing provable repeatability with bounded accuracy and efficiency. Effectiveness is demonstrated across three cases: standardized manipulator testing, intelligent risk assessment for automated vehicles, and performance evaluation of humanoid robot locomotion.