Developing a Modular Food Label Reader with Application to Vegan/Vegetarian Products

Text analysis in images is an important subfield of computer vision and extends objects’ recognition to language purposes. In the consumer world, food ingredients can be complex and hard to analyze if keeping to a specific dietary restriction. Herein, a modular approach to food label reading is developed using text analytics to solve the problem. Key to text analysis is detecting, segmenting, recognizing, and comparing text to keywords of interest. Herein a modular process is developed using these steps to create a label reader. As a demonstration, this process is used to find non-vegetarian and non-vegan ingredients on a given food ingredient list and return an answer if a product is vegetarian, vegan, or none. Testing is demonstrated on live food labels, which have text and background of various colors as well as different shading/lighting conditions, and provides a comparison of four optical character recognition (OCR) methods. Finally, this paper describes further modularization into an Android App.