Interfaces for comparative evaluation of interactive video search with expert & non-expert users.
As part of DCU's TRECVid Interactive Search activities, I developed a interactive search system which integrated with the KSpace video search engine to enable a user to query and
The traditional approach to presenting video search results is to maximise recall by offering a user as many potentially relevant shots as possible within a limited amount of time. 'Context'-oriented systems opt to allocate a portion of theresults presentation space to providing additional contextual cues about the returned results. In video retrieval these cues often include temporal information such as a shot's location within the overall video broadcast and/or its neighbouring shots.
We developed two interfaces with identical retrieval functionality in order to measure the effects of such context on user performance. The first system had a 'recall-oriented' interface, where results from a query were presented as a ranked list of shots. The second was 'context=oriented', with results presented as a ranked list of broadcasts.
In the 2007 TRECVid evaluation, 10 users participated in the experiments, of which 8 were novices and 2 experts. Participants completed a number of retrieval topics using both the recall-oriented and context-oriented systems. In 2008, we completed a multi-site, multi-interface experiment. Three institutes participated involving 36 users, 12 each from Dublin City University (DCU, Ireland), University of Glasgow (GU, Scotland) and Centrum Wiskunde & Informatica (CWI, the Netherlands). The interface was additionally deployed during the VideOlympics real-time evaluation at CIVR in 2007.