Query Generation for Multimodal Documents

Abstract

This paper studies the problem of generatinglikely queries for multimodal documents withimages. Our application scenario is enablingefficient ``first-stage retrieval'' of relevant doc-uments, by attaching generated queries to doc-uments before indexing. We can then indexthis expanded text to efficiently narrow downto candidate matches using inverted index, sothat expensive reranking can follow. Our eval-uation results show that our proposed multi-modal representation meaningfully improvesrelevance ranking.More importantly, ourframework can achieve the state of the art inthe first stage retrieval scenarios

Tasks

Reproductions