BERT has a Mouth, and It Must Speak: BERT as a Markov Random Field Language Model

Code

• github.com/nyu-dl/bert-gen
  OfficialIn paperpytorch★ 0
• github.com/kyunghyuncho/bert-gen
  OfficialIn paperpytorch★ 0
• github.com/MS-P3/code6
  mindspore★ 0
• github.com/gandharvsuri/Conditional-Generative-model-on-BERT
  none★ 0
• github.com/FrogTravel/PMLDL
  pytorch★ 0
• github.com/swastishreya/BERT_as_a_conditional_generative_model
  pytorch★ 0
• github.com/2023-MindSpore-1/ms-code-12/tree/main/Bert
  mindspore★ 0
• github.com/JuanJoseMV/neuraltextgen
  pytorch★ 0
• github.com/woailaosang/repo_treasure
  none★ 0
• github.com/MS-P3/code7/tree/main/visual_bert
  mindspore★ 0

Abstract

We show that BERT (Devlin et al., 2018) is a Markov random field language model. This formulation gives way to a natural procedure to sample sentences from BERT. We generate from BERT and find that it can produce high-quality, fluent generations. Compared to the generations of a traditional left-to-right language model, BERT generates sentences that are more diverse but of slightly worse quality.

Tasks

Reproductions