SOTAVerified

Language Modelling

A language model is a model of natural language. Language models are useful for a variety of tasks, including speech recognition, machine translation, natural language generation (generating more human-like text), optical character recognition, route optimization, handwriting recognition, grammar induction, and information retrieval.

Large language models (LLMs), currently their most advanced form, are predominantly based on transformers trained on larger datasets (frequently using words scraped from the public internet). They have superseded recurrent neural network-based models, which had previously superseded the purely statistical models, such as word n-gram language model.

Source: Wikipedia

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 13401–13450 of 17610 papers

TitleStatusHype
BPM_MT: Enhanced Backchannel Prediction Model using Multi-Task Learning—0
Effective Fine-Tuning Methods for Cross-lingual Adaptation—0
How Length Prediction Influence the Performance of Non-Autoregressive Translation?—0
APGN: Adversarial and Parameter Generation Networks for Multi-Source Cross-Domain Dependency Parsing—0
A Language Model-based Generative Classifier for Sentence-level Discourse Parsing—0
Distilling Relation Embeddings from Pretrained Language Models—0
Can Character-based Language Models Improve Downstream Task Performances In Low-Resource And Noisy Language Scenarios?—0
BART for Post-Correction of OCR Newspaper Text—0
Corporate Bankruptcy Prediction with BERT Model—0
“It doesn’t look good for a date”: Transforming Critiques into Preferences for Conversational Recommendation SystemsCode0
Exploring Multitask Learning for Low-Resource Abstractive SummarizationCode0
Improving Synonym Recommendation Using Sentence Context—0
Domain-adaptation of spherical embeddings—0
A practical perspective on connective generation—0
Chinese WPLC: A Chinese Dataset for Evaluating Pretrained Language Models on Word Prediction Given Long-Range Context—0
APIRecX: Cross-Library API Recommendation via Pre-Trained Language Model—0
Intrinsic evaluation of language models for code-switchingCode0
Casting the Same Sentiment Classification ProblemCode0
Cryptocurrency Day Trading and Framing Prediction in Microblog Discourse—0
IGA: An Intent-Guided Authoring Assistant—0
Distilling Word Meaning in Context from Pre-trained Language ModelsCode0
Hyperparameter Power Impact in Transformer Language Model TrainingCode0
Zero-shot cross-lingual Meaning Representation Transfer: Annotation of Hungarian using the Prague Functional Generative Description—0
What Can a Generative Language Model Answer About a Passage?—0
What BERT Based Language Model Learns in Spoken Transcripts: An Empirical Study—0
Unsupervised Multi-View Post-OCR Error Correction With Language Models—0
What’s in Your Head? Emergent Behaviour in Multi-Task Transformer Models—0
Unsupervised Discovery of Unaccusative and Unergative Verbs—0
Unsupervised Adverbial Identification in Modern Chinese Literature—0
Who’s on First?: Probing the Learning and Representation Capabilities of Language Models on Deterministic Closed DomainsCode0
Language Clustering for Multilingual Named Entity Recognition—0
Transfer Learning with Shallow Decoders: BSC at WMT2021’s Multilingual Low-Resource Translation for Indo-European Languages Shared TaskCode0
UnClE: Explicitly Leveraging Semantic Similarity to Reduce the Parameters of Word Embeddings—0
Multilingual Sequence Labeling Approach to solve Lexical Normalization—0
Multi-task Learning in Argument Mining for Persuasive Online Discussions—0
Stacked AMR Parsing with Silver DataCode0
OpenFraming: Open-sourced Tool for Computational Framing Analysis of Multilingual DataCode0
KLMo: Knowledge Graph Enhanced Pretrained Language Model with Fine-Grained RelationshipsCode0
Learning Cross-lingual Representations for Event Coreference Resolution with Multi-view Alignment and Optimal Transport—0
The Acceptability Delta Criterion: Testing Knowledge of Language using the Gradience of Sentence Acceptability—0
The World of an Octopus: How Reporting Bias Influences a Language Model’s Perception of ColorCode0
Paraphrasing Compound Nominalizations—0
Language Model Pretraining and Transfer Learning for Very Low Resource Languages—0
ODIST: Open World Classification via Distributionally Shifted Instances—0
On the Role of Corpus Ordering in Language Modeling—0
Scaffolded input promotes atomic organization in the recurrent neural network language modelCode0
Label-Enhanced Hierarchical Contextualized Representation for Sequential Metaphor Identification—0
Named Entity Recognition in Historic Legal Text: A Transformer and State Machine Ensemble Method—0
ProSPer: Probing Human and Neural Network Language Model Understanding of Spatial PerspectiveCode0
SpellBERT: A Lightweight Pretrained Model for Chinese Spelling Check—0
Show:102550
← PrevPage 269 of 353Next →
All datasetsWikiText-103Penn Treebank (Word Level)enwik8The PileWikiText-2LAMBADAOne Billion WordText8Penn Treebank (Character Level)Hutter PrizeOpenWebTextSALMon

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1Decay RNNValidation perplexity76.67—Unverified
2GRUValidation perplexity53.78—Unverified
3LSTMValidation perplexity52.73—Unverified
4LSTMTest perplexity48.7—Unverified
5Temporal CNNTest perplexity45.2—Unverified
6TCNTest perplexity45.19—Unverified
7GCNN-8Test perplexity44.9—Unverified
8Neural cache model (size = 100)Test perplexity44.8—Unverified
9Neural cache model (size = 2,000)Test perplexity40.8—Unverified
10GPT-2 SmallTest perplexity37.5—Unverified
#ModelMetricClaimedVerifiedStatus
1TCNTest perplexity108.47—Unverified
2Seq-U-NetTest perplexity107.95—Unverified
3GRU (Bai et al., 2018)Test perplexity92.48—Unverified
4R-TransformerTest perplexity84.38—Unverified
5Zaremba et al. (2014) - LSTM (medium)Test perplexity82.7—Unverified
6Gal & Ghahramani (2016) - Variational LSTM (medium)Test perplexity79.7—Unverified
7LSTM (Bai et al., 2018)Test perplexity78.93—Unverified
8Zaremba et al. (2014) - LSTM (large)Test perplexity78.4—Unverified
9Gal & Ghahramani (2016) - Variational LSTM (large)Test perplexity75.2—Unverified
10Inan et al. (2016) - Variational RHNTest perplexity66—Unverified
#ModelMetricClaimedVerifiedStatus
1LSTM (7 layers)Bit per Character (BPC)1.67—Unverified
2HypernetworksBit per Character (BPC)1.34—Unverified
3SHA-LSTM (4 layers, h=1024, no attention head)Bit per Character (BPC)1.33—Unverified
4LN HM-LSTMBit per Character (BPC)1.32—Unverified
5ByteNetBit per Character (BPC)1.31—Unverified
6Recurrent Highway NetworksBit per Character (BPC)1.27—Unverified
7Large FS-LSTM-4Bit per Character (BPC)1.25—Unverified
8Large mLSTMBit per Character (BPC)1.24—Unverified
9AWD-LSTM (3 layers)Bit per Character (BPC)1.23—Unverified
10Cluster-Former (#C=512)Bit per Character (BPC)1.22—Unverified
#ModelMetricClaimedVerifiedStatus
1Smaller Transformer 126M (pre-trained)Test perplexity33—Unverified
2OPT 125MTest perplexity32.26—Unverified
3Larger Transformer 771M (pre-trained)Test perplexity28.1—Unverified
4OPT 1.3BTest perplexity19.55—Unverified
5GPT-Neo 125MTest perplexity17.83—Unverified
6OPT 2.7BTest perplexity17.81—Unverified
7Smaller Transformer 126M (fine-tuned)Test perplexity12—Unverified
8GPT-Neo 1.3BTest perplexity11.46—Unverified
9Transformer 125MTest perplexity10.7—Unverified
10GPT-Neo 2.7BTest perplexity10.44—Unverified