Interacting with embodied agents that can see

In this paper, we describe user experiences with a system equipped with cognitive vision that interacts with the user in the context of personal assistance in the office. A cognitive vision computer can see the user and user responses and react to situations that happen in the environment, crossing the boundary between the virtual and the physical world. How should such a seeing computer interact with its users? Three different interface styles -- a traditional GUI, a cartoon-like embodied agent and a realistic embodied agent -- are tested in two tasks where users are actively observed by a (simulated) cognitive vision system. The system assists them in problem solving. Both the non-embodied and the embodied interaction styles offer the user certain advantages and the pros and cons based on the experiment results are discussed in terms of performance, intelligence, trust, comfort, and social presence.