Traditional IR rivals neural models on the MS MARCO Document Ranking Leaderboard

Code

• github.com/oaqa/FlexNeuART
  OfficialIn paperpytorch★ 223
• github.com/oaqa/knn4qa
  pytorch★ 223

Abstract

This short document describes a traditional IR system that achieved MRR@100 equal to 0.298 on the MS MARCO Document Ranking leaderboard (on 2020-12-06). Although inferior to most BERT-based models, it outperformed several neural runs (as well as all non-neural ones), including two submissions that used a large pretrained Transformer model for re-ranking. We provide software and data to reproduce our results.

Tasks

Reproductions