User reactions as access mechanism: An exploration based on captions for images

Words are problematic for describing images, but they are a convenient and traditional way of describing requests. Users can given voice to their reactions to images—how well they suit needs. User-generated reactions might provide word-based descriptors helpful to subsequent users and requiring minimal system resources to produce. Shifting focus from description of documents to description of reactions is accomplished by gathering verbal captions and responses to images. User generation of captions and verbal responses within a collection of 300 diverse images is demonstrated and analyzed. Functional adjectival descriptors appear in 20% of the responses and functional narrative (conversational) descriptors appear in 80% of the responses. Issues of larger scale analysis, implementation, and possible shifts in understanding of representation for retrieval are discussed.