Combined Infinity Laplacian and Non-Local Means Models Applied to Depth Map Restoration

Scene depth information is a key component of any robotic mobile application. Range sensors, such as LiDAR, sonar, or radar, capture depth data of a scene. However, the data captured by these sensors frequently presents missing regions or information with a low confidence level. These missing regions in the depth data could be large areas without information, making it difficult to make decisions, for instance, for an autonomous vehicle. Recovering depth data has become a primary activity for computer vision applications. This work proposes and evaluates an interpolation model to infer dense depth maps from a Lab color space reference picture and an incomplete-depth image embedded in a completion pipeline. The complete proposal pipeline comprises convolutional layers and a convex combination of the infinity Laplacian and non-local means model. The proposed model infers dense depth maps by considering depth data and utilizing clues from a color picture of the scene, along with a metric for computing differences between two pixels. The work contributes (i) the convex combination of the two models to interpolate the data, and (ii) the proposal of a class of function suitable for balancing between different models. The obtained results show that the model outperforms similar models in the KITTI dataset and outperforms our previous implementation in the NYU_v2 dataset, dropping the MSE by 34.86%, 3.35%, and 34.42% for 4×, 8×, 16× upsampling tasks, respectively.