Applying deep learning to imaging sonar for the automated detection, classification and counting of untagged fish in fish passages

The Ocean Aware project, led by Innovasea and funded through Canada’s Ocean Supercluster, is developing the next generation of underwater observation systems to transform fishing, aquaculture, and marine energy. One facet of this project is developing innovative methods for real-time tracking of untagged fish and species at risk around man-made infrastructures, such as hydropower dams, that present barriers to fish passage. The system is based on applying modern Deep Learning techniques to automatically detect and classify fish and their species from high resolution imaging sonar. In this paper, we present our results from applying adaptations of the widely used YOLO machine learning model to detect and classify multiple species of fish from a public dataset containing eight distinct species of fish from the Ocqueoc River captured by a high resolution DIDSON imaging sonar. Our results demonstrate that deep learning models can indeed be used to detect, classify species, and track fish using high resolution imaging sonar. Although there has been extensive research in the literature identifying particular fish, such as eel versus non-eel and seal versus fish, to our knowledge this is the first successful application of deep learning for classifying multiple fish species with high resolution imaging sonar.