Parallel Multi-Dimensional LSTM, With Application to Fast Biomedical\n Volumetric Image Segmentation

Convolutional Neural Networks (CNNs) can be shifted across 2D images or 3D\nvideos to segment them. They have a fixed input size and typically perceive\nonly small local contexts of the pixels to be classified as foreground or\nbackground. In contrast, Multi-Dimensional Recurrent NNs (MD-RNNs) can perceive\nthe entire spatio-temporal context of each pixel in a few sweeps through all\npixels, especially when the RNN is a Long Short-Term Memory (LSTM). Despite\nthese theoretical advantages, however, unlike CNNs, previous MD-LSTM variants\nwere hard to parallelize on GPUs. Here we re-arrange the traditional cuboid\norder of computations in MD-LSTM in pyramidal fashion. The resulting\nPyraMiD-LSTM is easy to parallelize, especially for 3D data such as stacks of\nbrain slice images. PyraMiD-LSTM achieved best known pixel-wise brain image\nsegmentation results on MRBrainS13 (and competitive results on EM-ISBI12).\n