SOTAVerified

Language Modelling

A language model is a model of natural language. Language models are useful for a variety of tasks, including speech recognition, machine translation, natural language generation (generating more human-like text), optical character recognition, route optimization, handwriting recognition, grammar induction, and information retrieval.

Large language models (LLMs), currently their most advanced form, are predominantly based on transformers trained on larger datasets (frequently using words scraped from the public internet). They have superseded recurrent neural network-based models, which had previously superseded the purely statistical models, such as word n-gram language model.

Source: Wikipedia

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 16851–16900 of 17610 papers

TitleStatusHype
Unsupervised Discovery of Linguistic Structure Including Two-level Acoustic Patterns Using Three Cascaded Stages of Iterative Optimization—0
Statistical Machine Translation Improvement based on Phrase Selection—0
Feature Extraction for Native Language Identification Using Language Modeling—0
Evaluating the Impact of Using a Domain-specific Bilingual Lexicon on the Performance of a Hybrid Machine Translation Approach—0
An LDA-based Topic Selection Approach to Language Model Adaptation for Handwritten Text Recognition—0
Enriching Word Sense Embeddings with Translational Context—0
Data Selection With Fewer Words—0
Extended Translation Models in Phrase-based Decoding—0
Edinburgh's Syntax-Based Systems at WMT 2015—0
Input Seed Features for Guiding the Generation Process: A Statistical Approach for Spanish—0
Exact Decoding with Multi Bottom-Up Tree Transducers—0
VERTa: a Linguistically-motivated Metric at the WMT15 Metrics Task—0
Statistical Machine Translation with Automatic Identification of Translationese—0
The RWTH Aachen German-English Machine Translation System for WMT 2015—0
The University of Illinois submission to the WMT 2015 Shared Translation Task—0
The Edinburgh/JHU Phrase-based Machine Translation Systems for WMT 2015—0
The AFRL-MITLL WMT15 System: There's More than One Way to Decode It!—0
Predicting Machine Translation Adequacy with Document Embeddings—0
Results of the WMT15 Tuning Shared Task—0
Sentiment Analysis on Monolingual, Multilingual and Code-Switching Twitter Corpora—0
ParFDA for Fast Deployment of Accurate Statistical Machine Translation Systems, Benchmarks, and Statistics—0
Qualitative investigation of the display of speech recognition results for communication with deaf people—0
Morphological Segmentation and OPUS for Finnish-English Machine Translation—0
SHEF-NN: Translation Quality Estimation with Neural Networks—0
Natural Language Generation from Pictographs—0
Investigations on Phrase-based Decoding with Recurrent Neural Network Language and Translation Models—0
The Karlsruhe Institute of Technology Translation Systems for the WMT 2015—0
Predicting Pronouns across Languages with Continuous Word Spaces—0
Baseline Models for Pronoun Prediction and Pronoun-Aware Translation—0
Automatic Post-Editing for the DiscoMT Pronoun Translation Task—0
Hierarchical Recurrent Neural Network for Document Modeling—0
A Joint Dependency Model of Morphological and Syntactic Structure for Statistical Machine Translation—0
Improved Arabic Dialect Classification with Social Media Data—0
Improving Statistical Machine Translation with a Multilingual Paraphrase Database—0
GhostWriter: Using an LSTM for Automatic Rap Lyric Generation—0
Arabic Diacritization with Recurrent Neural Networks—0
Can Symbol Grounding Improve Low-Level NLP? Word Segmentation as a Case Study—0
A Coarse-Grained Model for Optimal Coupling of ASR and SMT Systems for Speech Translation—0
An Empirical Comparison Between N-gram and Syntactic Language Models for Word Ordering—0
Compact, Efficient and Unlimited Capacity: Language Modeling with Compressed Suffix Trees—0
Bilingual Structured Language Models for Statistical Machine Translation—0
A Discriminative Training Procedure for Continuous Translation Models—0
Hierarchical Latent Words Language Models for Robust Modeling to Out-Of Domain Tasks—0
A Dynamic Programming Algorithm for Computing N-gram Posteriors from Lattices—0
Graph-Based Collective Lexical Selection for Statistical Machine Translation—0
A Comparison between Count and Neural Network Models Based on Joint Translation and Reordering Sequences—0
A Binarized Neural Network Joint Model for Machine Translation—0
Distributed Representations for Unsupervised Semantic Role Labeling—0
How to Avoid Unwanted Pregnancies: Domain Adaptation using Neural Network Models—0
Reinforcing the Topic of Embeddings with Theta Pure Dependence for Text Classification—0
Show:102550
← PrevPage 338 of 353Next →
All datasetsWikiText-103Penn Treebank (Word Level)enwik8The PileWikiText-2LAMBADAOne Billion WordText8Penn Treebank (Character Level)Hutter PrizeOpenWebTextSALMon

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1Decay RNNValidation perplexity76.67—Unverified
2GRUValidation perplexity53.78—Unverified
3LSTMValidation perplexity52.73—Unverified
4LSTMTest perplexity48.7—Unverified
5Temporal CNNTest perplexity45.2—Unverified
6TCNTest perplexity45.19—Unverified
7GCNN-8Test perplexity44.9—Unverified
8Neural cache model (size = 100)Test perplexity44.8—Unverified
9Neural cache model (size = 2,000)Test perplexity40.8—Unverified
10GPT-2 SmallTest perplexity37.5—Unverified
#ModelMetricClaimedVerifiedStatus
1TCNTest perplexity108.47—Unverified
2Seq-U-NetTest perplexity107.95—Unverified
3GRU (Bai et al., 2018)Test perplexity92.48—Unverified
4R-TransformerTest perplexity84.38—Unverified
5Zaremba et al. (2014) - LSTM (medium)Test perplexity82.7—Unverified
6Gal & Ghahramani (2016) - Variational LSTM (medium)Test perplexity79.7—Unverified
7LSTM (Bai et al., 2018)Test perplexity78.93—Unverified
8Zaremba et al. (2014) - LSTM (large)Test perplexity78.4—Unverified
9Gal & Ghahramani (2016) - Variational LSTM (large)Test perplexity75.2—Unverified
10Inan et al. (2016) - Variational RHNTest perplexity66—Unverified
#ModelMetricClaimedVerifiedStatus
1LSTM (7 layers)Bit per Character (BPC)1.67—Unverified
2HypernetworksBit per Character (BPC)1.34—Unverified
3SHA-LSTM (4 layers, h=1024, no attention head)Bit per Character (BPC)1.33—Unverified
4LN HM-LSTMBit per Character (BPC)1.32—Unverified
5ByteNetBit per Character (BPC)1.31—Unverified
6Recurrent Highway NetworksBit per Character (BPC)1.27—Unverified
7Large FS-LSTM-4Bit per Character (BPC)1.25—Unverified
8Large mLSTMBit per Character (BPC)1.24—Unverified
9AWD-LSTM (3 layers)Bit per Character (BPC)1.23—Unverified
10Cluster-Former (#C=512)Bit per Character (BPC)1.22—Unverified
#ModelMetricClaimedVerifiedStatus
1Smaller Transformer 126M (pre-trained)Test perplexity33—Unverified
2OPT 125MTest perplexity32.26—Unverified
3Larger Transformer 771M (pre-trained)Test perplexity28.1—Unverified
4OPT 1.3BTest perplexity19.55—Unverified
5GPT-Neo 125MTest perplexity17.83—Unverified
6OPT 2.7BTest perplexity17.81—Unverified
7Smaller Transformer 126M (fine-tuned)Test perplexity12—Unverified
8GPT-Neo 1.3BTest perplexity11.46—Unverified
9Transformer 125MTest perplexity10.7—Unverified
10GPT-Neo 2.7BTest perplexity10.44—Unverified