MISNet: Multi-task interaction Siamese network for 3D point cloud semantic change detection

• Two publicly available realistic 3D semantic change detection datasets. • A novel multi-task learning architecture for large-scale 3D point cloud semantic change detection. • A multi-dimensional change encoding module that refines the cross-temporal neighborhood variations. • A change-guided semantic refinement module that enhances the representation of semantic features. • A semantic-awareness change interaction module that complements the characteristic of change. 3D semantic change detection enables the detection and identification of both changes in urban objects and their semantic categories, providing fine-grained change information for downstream applications. Existing methods rely on single-branch architectures with predefined output labels, which is simple but suffers from complex output definitions and ineffective multi-task coupling, compounded by scarce annotated 3D realistic data. To overcome these challenges, firstly, two 3D realistic semantic change detection datasets are constructed and published, named HKSCD and UtrechtCD , which utilize oblique photogrammetry point clouds and LiDAR point clouds to describe 9 semantic categories and 2 change types in Hong Kong, China, and Utrecht, Netherlands, covering 15 square kilometers with 370 million points. Secondly, a Multi-task Interaction Siamese Network (MISNet) for 3D point cloud semantic change detection is proposed. It deeply couples semantic segmentation and change detection, enabling the simultaneous prediction of both tasks within a unified architecture. The proposed multi-dimensional change encoding module computes cross-temporal neighborhood relationships from multiple dimensions to extract accurate point cloud change features. Additionally, the change-guided semantic refinement module and the semantic-awareness change interaction module leverage change information to support semantic consistency and utilize semantic information to assist inter-class change detection to promote cross-task consistent modeling underpinned by the cross-learning strategy. Extensive experiments demonstrate that MISNet achieves mIoU of 84.15% (HKSCD), 85.15% (UtrechtCD), and 89.58% (Urb3DCD-V2), outperforming existing methods by + 2.21%, +1.43%, and + 1.46%, respectively. The code and dataset are available at https://github.com/zhanwenxiao/UrbanSCD and https://github.com/zhanwenxiao/MISNet .