BANet: Boundary-Assistant Encoder-Decoder Network for Semantic Segmentation

Recently, boundary information has gained great attraction for semantic segmentation. This paper presents a novel encoder-decoder network, called BANet, for accurate semantic segmentation, where boundary information is employed as an additional assistance for producing more consistent segmentation outputs. BANet is composed of three components: the pre-trained backbone using dilated-ResNet101, semantic flow branch (SFB) and boundary flow branch (BFB) for semantic segmentation and boundary detection, respectively. More specifically, to delineate more accurate object shapes and boundaries, a global attention block (GAB) is designed in SFB as global guidance for high-level feature. On the other hand, BFB directly extracts features on boundaries, avoiding the unexpected interference from the non-boundary parts. Finally, we adopt a joint loss function to further optimize the segmentation results and boundary outputs synchronously. Moreover, compared with previous state-of-the-art methods, e.g., non-local block and ASPP module, our BFB leverages detection accuracy and computational efficiency in a lightweight fashion. To evaluate BANet, we have conducted extensive experiments on several semantic segmentation datasets: Cityscapes, PASCAL Context, and ADE20K. The experimental results show that, with the aid of boundary information, BANet is able to produce more consistent segmentation predictions with accurately delineated object shapes and boundaries, leading to the state-of-the-art performance on Cityscapes, and competitive results on PASCAL Context and ADE20K with respect to recent semantic segmentation networks.