SOTAVerified

Language Modelling

A language model is a model of natural language. Language models are useful for a variety of tasks, including speech recognition, machine translation, natural language generation (generating more human-like text), optical character recognition, route optimization, handwriting recognition, grammar induction, and information retrieval.

Large language models (LLMs), currently their most advanced form, are predominantly based on transformers trained on larger datasets (frequently using words scraped from the public internet). They have superseded recurrent neural network-based models, which had previously superseded the purely statistical models, such as word n-gram language model.

Source: Wikipedia

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 8801–8850 of 17610 papers

TitleStatusHype
Language Model Augmented Relevance Score—0
Language Model Based Chinese Handwriting Address Recognition—0
Language Model Based Grammatical Error Correction without Annotated Training Data—0
Language Model Bootstrapping Using Neural Machine Translation For Conversational Speech Recognition—0
Language Model Can Do Knowledge Tracing: Simple but Effective Method to Integrate Language Model and Knowledge Tracing Task—0
Language Model Cascades: Token-level uncertainty and beyond—0
Language model compression with weighted low-rank factorization—0
Language Model Decoding as Direct Metrics Optimization—0
Language Model Detoxification in Dialogue with Contextualized Stance Control—0
Language model developers should report train-test overlap—0
Language model driven: a PROTAC generation pipeline with dual constraints of structure and property—0
Language Model-Driven Data Pruning Enables Efficient Active Learning—0
LEMON: LanguagE ModeL for Negative Sampling of Knowledge Graph Embeddings—0
Language Model-Driven Unsupervised Neural Machine Translation—0
Language Model-Enhanced Message Passing for Heterophilic Graph Learning—0
Language Model Evaluation Beyond Perplexity—0
Language Model Evaluation in Open-ended Text Generation—0
Language Model Evolutionary Algorithms for Recommender Systems: Benchmarks and Algorithm Comparisons—0
Language model fusion for streaming end to end speech recognition—0
Language Model-Guided Knowledge Subgraphs for Question Answering—0
Language Modeling at Scale—0
Language Modeling by Clustering with Word Embeddings for Text Readability Assessment—0
Language Modeling for Code-Mixing: The Role of Linguistic Theory based Synthetic Data—0
Language Modeling for Code-Switched Data: Challenges and Approaches—0
Mathematical Reasoning via Self-supervised Skip-tree Training—0
Language Modeling for Morphologically Rich Languages: Character-Aware Modeling for Word-Level Prediction—0
Language Modeling for Spoken Dialogue System based on Filtering using Predicate-Argument Structures—0
Language Modeling for the Future of Finance: A Quantitative Survey into Metrics, Tasks, and Data Opportunities—0
Language Modeling, Lexical Translation, Reordering: The Training Process of NMT through the Lens of Classical SMT—0
Language Modeling on a SpiNNaker 2 Neuromorphic Chip—0
Language Modeling Teaches You More Syntax than Translation Does: Lessons Learned Through Auxiliary Task Analysis—0
Language Modeling Teaches You More than Translation Does: Lessons Learned Through Auxiliary Syntactic Task Analysis—0
Language Modeling Teaches You More than Translation Does: Lessons Learned Through Auxiliary Task Analysis—0
Language Modeling through Long Term Memory Network—0
Language Modeling using LMUs: 10x Better Data Efficiency or Improved Scaling Compared to Transformers—0
Language Modeling with a General Second-Order RNN—0
Language Modeling with Deep Transformers—0
Language Modeling with Functional Head Constraint for Code Switching Speech Recognition—0
Language Modeling with Generative AdversarialNetworks—0
Language Modeling with Graph Temporal Convolutional Networks—0
Language Modeling with Highway LSTM—0
Language Modeling with Latent Situations—0
Language modeling with Neural trans-dimensional random fields—0
Language Modeling with Power Low Rank Ensembles—0
Language Modeling with Recurrent Highway Hypernetworks—0
Language Modeling with Reduced Densities—0
Language Modeling with Shared Grammar—0
Language model integration based on memory control for sequence to sequence speech recognition—0
Language Model-In-The-Loop: Data Optimal Approach to Learn-To-Recommend Actions in Text Games—0
Language Model is All You Need: Natural Language Understanding as Question Answering—0
Show:102550
← PrevPage 177 of 353Next →
All datasetsWikiText-103Penn Treebank (Word Level)enwik8The PileWikiText-2LAMBADAOne Billion WordText8Penn Treebank (Character Level)Hutter PrizeOpenWebTextSALMon

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1Decay RNNValidation perplexity76.67—Unverified
2GRUValidation perplexity53.78—Unverified
3LSTMValidation perplexity52.73—Unverified
4LSTMTest perplexity48.7—Unverified
5Temporal CNNTest perplexity45.2—Unverified
6TCNTest perplexity45.19—Unverified
7GCNN-8Test perplexity44.9—Unverified
8Neural cache model (size = 100)Test perplexity44.8—Unverified
9Neural cache model (size = 2,000)Test perplexity40.8—Unverified
10GPT-2 SmallTest perplexity37.5—Unverified
#ModelMetricClaimedVerifiedStatus
1TCNTest perplexity108.47—Unverified
2Seq-U-NetTest perplexity107.95—Unverified
3GRU (Bai et al., 2018)Test perplexity92.48—Unverified
4R-TransformerTest perplexity84.38—Unverified
5Zaremba et al. (2014) - LSTM (medium)Test perplexity82.7—Unverified
6Gal & Ghahramani (2016) - Variational LSTM (medium)Test perplexity79.7—Unverified
7LSTM (Bai et al., 2018)Test perplexity78.93—Unverified
8Zaremba et al. (2014) - LSTM (large)Test perplexity78.4—Unverified
9Gal & Ghahramani (2016) - Variational LSTM (large)Test perplexity75.2—Unverified
10Inan et al. (2016) - Variational RHNTest perplexity66—Unverified
#ModelMetricClaimedVerifiedStatus
1LSTM (7 layers)Bit per Character (BPC)1.67—Unverified
2HypernetworksBit per Character (BPC)1.34—Unverified
3SHA-LSTM (4 layers, h=1024, no attention head)Bit per Character (BPC)1.33—Unverified
4LN HM-LSTMBit per Character (BPC)1.32—Unverified
5ByteNetBit per Character (BPC)1.31—Unverified
6Recurrent Highway NetworksBit per Character (BPC)1.27—Unverified
7Large FS-LSTM-4Bit per Character (BPC)1.25—Unverified
8Large mLSTMBit per Character (BPC)1.24—Unverified
9AWD-LSTM (3 layers)Bit per Character (BPC)1.23—Unverified
10Cluster-Former (#C=512)Bit per Character (BPC)1.22—Unverified
#ModelMetricClaimedVerifiedStatus
1Smaller Transformer 126M (pre-trained)Test perplexity33—Unverified
2OPT 125MTest perplexity32.26—Unverified
3Larger Transformer 771M (pre-trained)Test perplexity28.1—Unverified
4OPT 1.3BTest perplexity19.55—Unverified
5GPT-Neo 125MTest perplexity17.83—Unverified
6OPT 2.7BTest perplexity17.81—Unverified
7Smaller Transformer 126M (fine-tuned)Test perplexity12—Unverified
8GPT-Neo 1.3BTest perplexity11.46—Unverified
9Transformer 125MTest perplexity10.7—Unverified
10GPT-Neo 2.7BTest perplexity10.44—Unverified