Kinect cane: An assistive system for the visually impaired based on three-dimensional object recognition

This paper proposes a novel assistive system for the visually impaired. The system is composed of a Microsoft Kinect sensor, keypad-type controller, tactile device, laptop computer and so on. The system can recognize three-dimensional objects from depth data generated by the Kinect sensor, and inform visually impaired users not only about the existence of objects, but also about their classes such as chairs and upward stairs. Ordinarily, the system works as a conventional white cane. When a user instructs the system to find the object of a particular class, the system executes the recognition scheme that is designed to find the instructed object. If the object is found in the field of view of the Kinect sensor, the tactile device provides vibration feedback. The recognition schemes are applied to actual scenes. The experimental results indicate that the system is promising as means of helping the visually impaired find the desired objects.