Deploying a Low-Cost Vision-Guided Cartesian Robot for Physical Interaction in Mobile Testing: An Industrial Case Study

Automation of tests and related tasks is becoming increasingly essential, as it significantly reduces manual effort and improves efficiency. However, certain test cases cannot be fully automated due to the need for physical interaction either during or prior to their execution. In mobile testing scenarios, for instance, physical interaction may be required to enable USB debugging when it is not pre-authorized, preventing the full automation of test suites. Accordingly, the use of robotic mechanisms for software testing has emerged as a viable alternative to overcome the limitations of full automation. However, such solutions often involve high acquisition, development, and integration costs. To address these challenges, this paper proposes an alternative low-cost approach: a Cartesian robot controlled via Arduino, equipped with a 3D-printed finger. The robot’s movements and interactions with the device screen are guided by computer vision, enabling automated physical interaction during testing. This setup enables automated testing while significantly reducing development and operational costs compared to conventional robotic arms typically used for such tasks. As a preliminary evaluation, our proposed solution was deployed in an industrial setting to enable USB debugging and unblock the automation process within a software testing framework. The results indicate that the approach is highly accurate. Furthermore, when compared to the manual process, which takes approximately 120 minutes, our method reduces the time to 70 minutes, resulting in a time savings of approximately 42%.