Industrial monocular depth estimation, semantic segmentation, and 6D pose estimation using foundational depth models in a data-scarce setting

Many industrial inspection tasks often require 3D information in order to be solved properly. Some examples include object pose estimation with 6 degrees of freedom (6D), conformity checks and volumetric measurements. However, obtaining accurate 3D or depth information about a scene in an industrial environment is in itself a considerably challenging task, especially in production of mechanical parts, where many reflective metallic surfaces are encountered. Reflections make it difficult or even impossible to employ specialized depth sensors such as LiDARs and stereo 3D cameras, or traditional depth estimation and 3D reconstruction algorithms. Conversely, the fact that industrial use cases often involve working with a very low volume of available data usually prohibits the training of custom deep learning models for depth estimation. As a solution, we demonstrate the viability of utilizing large, pre-trained foundational models for monocular depth estimation (such as Depth Anything variants) in industrial production. More specifically, we showcase successful applications of these models in the domain of 5-axis machining setup inspection. The proposed methodology enables adjusting the outputs of foundational depth models in a use case-specific setting, based on known values of 5-axis machine coordinates. Importantly, such adjustments are performed in a way that avoids fine-tuning of neural network parameters, and therefore our methodology can be applied even under extreme data scarcity conditions, using less than 50 images. Moreover, we showcase two downstream applications of the proposed monocular depth estimation approach - depth-based semantic segmentation of objects of interest, as well as depth-based 6D pose estimation. It is shown that the absolute relative error (AbsRel) of our depth estimations can be as low as 2.54% on average, which in turn leads to very precise semantic segmentation (89.4% IoU) and object pose estimations (6.39 mm ADD for objects with dimensions 200 mm×200 mm×60 mm). Lastly, the proposed segmentation and pose estimation methods exhibit excellent generalization capabilities, maintaining strong accuracy even when tested on images of completely new setups.