Detecting reconstruction-unfriendly surgical scenes using a few supporting frames to improve reconstruction-friendly frame selection for 3D reconstruction

High-quality 3D reconstruction of surgical scenes is essential for intra-operative navigation and clinical decision support. A key yet underexplored challenge lies in the automatic selection of frames that are truly suitable for reconstruction. Existing approaches to keyframe selection, which typically detect frames based on visual changes, are not tailored to endoscopic videos and often misclassify out-of-body or visually degraded frames as valid. In this work, we introduce a novel framework for automatic, reconstruction-oriented frame selection, and we define these frames as reconstruction-friendly frames. Specifically, we establish the first dataset of surgical videos annotated with reconstruction-oriented and reconstruction-unfriendly scenes, and revise a few-shot learning method augmented by a color histogram–based representation refinement strategy. Our approach enables the system to automatically identify and select those frames that contribute to precise 3D reconstruction, while effectively filtering out unsuitable ones. Experimental evaluations demonstrate that the proposed method achieves superior detection of reconstruction-unfriendly frames, even with limited annotated data.