Deep learning object detection for tracing the plasma portion of whole blood from images of medical sample containers

In this work, we explore the use of a deep learning neural network architecture for doing object detection of medical fluid samples, from color images. The ability of reliable identification and localization of liquid phases inside sample containers can help in automating significant parts of the work carried out in medical laboratories and hospitals. Our focus is on tracing the liquid levels of the plasma portion of whole blood, which may be further used in aliquoting, platelet rich plasma treatments automation, complete blood count analysis and others. The majority of sample containers are transparent and may contain or may not, one or more barcodes attached on the external surface, in addition to the manufacturer’s labels. Consequently, the available clearance window to view the liquid contents is of variable size and shape, and may be sufficiently large only from few view angles of the sample container. To address the clearance window challenge, we introduce a new input representation technique, named vertical image stitching, that takes multiple images of a rotating sample container, extracts a region of interest from each of them and then stitches them together vertically in a single output image, called vertically stitched image. Based on the vertical stitched image, we introduce a new data augmentation technique named vertical stitch permutation, which enables obtaining 10 times more sample variations in the dataset. Finally, we convert the 2D bounding box localization computation into 1D points localization by removing from the loss function the bounding box components corresponding to the box height. We performed experiments on a dataset of 976 unique samples, containing 17 sample containers from 11 manufacturers. We trained YoloV4- tiny with the above mentioned techniques and achieved a mean average precision of 98.09% (at intersection over union threshold of 0.90) with an average intersection over union of 95.18%.