Massively Multilingual Sentence Embeddings for Zero-Shot Cross-Lingual Transfer and Beyond
Mikel Artetxe, Holger Schwenk
Code Available — Be the first to reproduce this paper.
ReproduceCode
- github.com/facebookresearch/LASEROfficialIn paperpytorch★ 3,661
- github.com/Unbabel/COMETpytorch★ 729
- github.com/jeongukjae/smaller-labsetf★ 19
- github.com/jiamingkong/infoxlm_paddlepaddle★ 4
- github.com/Tony4469/laser-agirpytorch★ 0
- github.com/facebookresearch/vizseqnone★ 0
- github.com/raymondhs/fairseq-laserpytorch★ 0
- github.com/imamathcat/LASER_Dependenciespytorch★ 0
- github.com/kmkwon94/ainize-laserpytorch★ 0
- github.com/prabhakar267/LASER-improvedpytorch★ 0
Abstract
We introduce an architecture to learn joint multilingual sentence representations for 93 languages, belonging to more than 30 different families and written in 28 different scripts. Our system uses a single BiLSTM encoder with a shared BPE vocabulary for all languages, which is coupled with an auxiliary decoder and trained on publicly available parallel corpora. This enables us to learn a classifier on top of the resulting embeddings using English annotated data only, and transfer it to any of the 93 languages without any modification. Our experiments in cross-lingual natural language inference (XNLI dataset), cross-lingual document classification (MLDoc dataset) and parallel corpus mining (BUCC dataset) show the effectiveness of our approach. We also introduce a new test set of aligned sentences in 112 languages, and show that our sentence embeddings obtain strong results in multilingual similarity search even for low-resource languages. Our implementation, the pre-trained encoder and the multilingual test set are available at https://github.com/facebookresearch/LASER
Tasks
Benchmark Results
| Dataset | Model | Metric | Claimed | Verified | Status |
|---|---|---|---|---|---|
| BUCC Chinese-to-English | Massively Multilingual Sentence Embeddings | F1 score | 92.27 | — | Unverified |
| BUCC French-to-English | Massively Multilingual Sentence Embeddings | F1 score | 93.91 | — | Unverified |
| BUCC German-to-English | Massively Multilingual Sentence Embeddings | F1 score | 96.19 | — | Unverified |
| BUCC Russian-to-English | Massively Multilingual Sentence Embeddings | F1 score | 93.3 | — | Unverified |