IGCV3: Interleaved Low-Rank Group Convolutions for Efficient Deep Neural\n Networks

In this paper, we are interested in building lightweight and efficient\nconvolutional neural networks. Inspired by the success of two design patterns,\ncomposition of structured sparse kernels, e.g., interleaved group convolutions\n(IGC), and composition of low-rank kernels, e.g., bottle-neck modules, we study\nthe combination of such two design patterns, using the composition of\nstructured sparse low-rank kernels, to form a convolutional kernel. Rather than\nintroducing a complementary condition over channels, we introduce a loose\ncomplementary condition, which is formulated by imposing the complementary\ncondition over super-channels, to guide the design for generating a dense\nconvolutional kernel. The resulting network is called IGCV3. We empirically\ndemonstrate that the combination of low-rank and sparse kernels boosts the\nperformance and the superiority of our proposed approach to the\nstate-of-the-arts, IGCV2 and MobileNetV2 over image classification on CIFAR and\nImageNet and object detection on COCO.\n