Toward Assured Autonomy Using Neurosymbolic Components and Systems

Neurosymbolic techniques are expected to deliver more functionalities and better performance in autonomous systems, but their assurance remains a challenge. There are various roles such components can play in an autonomous vehicle, for instance, world model maintenance, planning, and trajectory control. The world model is an internal representation of the external environment of the vehicle that is continuously updated based on new sensory inputs; the planning component generates waypoints for the vehicle to reach, while the trajectory controller produces the fine-grain control commands for the vehicle's path. This chapter presents how these three functions can be implemented using Neurosymbolic techniques, and presents results and the lessons learned. The components were developed in the context of a UAV executing a specific mission: localization of a specific object in an urban area, and evaluated in a virtual environment. An assurance technique based on model checking is presented that can be applied to a class of Neurosymbolic components that include finite-state control with neural components.