Two-handed gesture in multi-modal natural dialog

Tracking both hands in free-space with accompanying speech input can augment the user's ability to communicate with computers. This paper discusses the kinds of situations which call for two-handed input and not just the single hand, and reports a prototype in which two-handed gestures serve to input concepts, both static and dynamic, manipulate displayed items, and specify actions to be taken. Future directions include enlargement of the vocabulary of two-handed “coverbal” gestures and the modulation by gaze of gestural intent.