Evaluating a Visual Information Retrieval Interface: AspInquery at TREC-6

We built two Information Retrieval systems that were targeted for the TREC-6 "aspect oriented" retrieval track. The systems were built to test the usefulness of different visualizations in an interactive IR setting---in particular, an "aspect window" for the chosen task, and a 3-D visualization of document inter-relationships. We studied 24 users of the system and report the usage of different user interface elements, whether experienced users outperformed novices, and whether spatial reasoning ability was a good predictor of effective use of 3-D. INTRODUCTION Much if not most usage of Information Retrieval (IR) is in an interactive setting, where an individual must use an IR system to obtain information that they need. But historically, most IR evaluations are based on comparing batch mode runs of IR systems on fixed corpora and measuring precision (the percentage of documents that are retrieved that are relevant) and recall (the percentage of relevant documents that are retrieved)....