Large-Scale Contextualised Language Modelling for Norwegian

Code

• github.com/ltgoslo/NorBERT
  OfficialIn papertf★ 44
• github.com/ltgoslo/norec_sentence
  OfficialIn papernone★ 4

Abstract

We present the ongoing NorLM initiative to support the creation and use of very large contextualised language models for Norwegian (and in principle other Nordic languages), including a ready-to-use software environment, as well as an experience report for data preparation and training. This paper introduces the first large-scale monolingual language models for Norwegian, based on both the ELMo and BERT frameworks. In addition to detailing the training process, we present contrastive benchmark results on a suite of NLP tasks for Norwegian. For additional background and access to the data, models, and software, please see http://norlm.nlpl.eu

Tasks

Reproductions