SOTAVerified

Language Modelling

A language model is a model of natural language. Language models are useful for a variety of tasks, including speech recognition, machine translation, natural language generation (generating more human-like text), optical character recognition, route optimization, handwriting recognition, grammar induction, and information retrieval.

Large language models (LLMs), currently their most advanced form, are predominantly based on transformers trained on larger datasets (frequently using words scraped from the public internet). They have superseded recurrent neural network-based models, which had previously superseded the purely statistical models, such as word n-gram language model.

Source: Wikipedia

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 10501–10550 of 17610 papers

TitleStatusHype
GLADIS: A General and Large Acronym Disambiguation BenchmarkCode1
Learning a Fourier Transform for Linear Relative Positional Encodings in Transformers—0
TransFool: An Adversarial Attack against Neural Machine Translation ModelsCode0
Creating a Large Language Model of a Philosopher—0
Accelerating Large Language Model Decoding with Speculative SamplingCode2
Improving Rare Words Recognition through Homophone Extension and Unified Writing for Low-resource Cantonese Speech Recognition—0
Multimodal Chain-of-Thought Reasoning in Language ModelsCode4
Semantic Coherence Markers for the Early Diagnosis of the Alzheimer DiseaseCode0
Stochastic Contextual Bandits with Long Horizon Rewards—0
Co-Writing with Opinionated Language Models Affects Users' Views—0
Collaborating with language models for embodied reasoning—0
Weight Prediction Boosts the Convergence of AdamW—0
Large Language Models Can Be Easily Distracted by Irrelevant ContextCode1
The Power of External Memory in Increasing Predictive Model Capacity—0
In-Context Retrieval-Augmented Language ModelsCode2
Grounding Language Models to Images for Multimodal Inputs and OutputsCode2
FLAME: A small language model for spreadsheet formulas—0
BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language ModelsCode4
Red teaming ChatGPT via Jailbreaking: Bias, Robustness, Reliability and Toxicity—0
UzbekTagger: The rule-based POS tagger for Uzbek language—0
Tagging before Alignment: Integrating Multi-Modal Tags for Video-Text Retrieval—0
REPLUG: Retrieval-Augmented Black-Box Language ModelsCode3
Learning to Speak from Text: Zero-Shot Multilingual Text-to-Speech with Unsupervised Text PretrainingCode1
Multi-modal Large Language Model Enhanced Pseudo 3D Perception Framework for Visual Commonsense Reasoning—0
On Robustness of Prompt-based Semantic Parsing with Large Pre-trained Language Model: An Empirical Study on Codex—0
Knowledge Transfer from Pre-trained Language Models to Cif-based Speech Recognizers via Hierarchical DistillationCode1
Crawling the Internal Knowledge-Base of Language Models—0
ESC: Exploration with Soft Commonsense Constraints for Zero-shot Object Navigation—0
Vicarious Offense and Noise Audit of Offensive Speech Classifiers: Unifying Human and Machine Disagreement on What is OffensiveCode0
Unifying Molecular and Textual Representations via Multi-task Language ModellingCode1
On Pre-trained Language Models for AntibodyCode1
Towards Equitable Representation in Text-to-Image Synthesis Models with the Cross-Cultural Understanding Benchmark (CCUB) DatasetCode0
Truth Machines: Synthesizing Veracity in AI Language Models—0
Context-Aware Differential Privacy for Language Modeling—0
Large Language Models Are Latent Variable Models: Explaining and Finding Good Demonstrations for In-Context LearningCode1
Prompt-Based Editing for Text Style TransferCode1
Theme-driven Keyphrase Extraction to Analyze Social Media Discourse—0
SWARM Parallelism: Training Large Models Can Be Surprisingly Communication-EfficientCode1
ThoughtSource: A central hub for large language model reasoning dataCode3
Semi-Parametric Video-Grounded Text Generation—0
Probing Out-of-Distribution Robustness of Language Models with Parameter-Efficient Transfer Learning—0
Context Matters: A Strategy to Pre-train Language Model for Science Education—0
Call for Papers -- The BabyLM Challenge: Sample-efficient pretraining on a developmentally plausible corpusCode1
Talk the Walk: Synthetic Data Generation for Conversational Music Recommendation—0
Byte Pair Encoding for Symbolic MusicCode1
Case-Based Reasoning with Language Models for Classification of Logical FallaciesCode0
GPU-based Private Information Retrieval for On-Device Machine Learning InferenceCode1
Task formulation for Extracting Social Determinants of Health from Clinical Narratives—0
Parameter-Efficient Low-Resource Dialogue State Tracking by Prompt Tuning—0
Domain-Agnostic Molecular Generation with Chemical FeedbackCode1
Show:102550
← PrevPage 211 of 353Next →
All datasetsWikiText-103Penn Treebank (Word Level)enwik8The PileWikiText-2LAMBADAOne Billion WordText8Penn Treebank (Character Level)Hutter PrizeOpenWebTextSALMon

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1Decay RNNValidation perplexity76.67—Unverified
2GRUValidation perplexity53.78—Unverified
3LSTMValidation perplexity52.73—Unverified
4LSTMTest perplexity48.7—Unverified
5Temporal CNNTest perplexity45.2—Unverified
6TCNTest perplexity45.19—Unverified
7GCNN-8Test perplexity44.9—Unverified
8Neural cache model (size = 100)Test perplexity44.8—Unverified
9Neural cache model (size = 2,000)Test perplexity40.8—Unverified
10GPT-2 SmallTest perplexity37.5—Unverified
#ModelMetricClaimedVerifiedStatus
1TCNTest perplexity108.47—Unverified
2Seq-U-NetTest perplexity107.95—Unverified
3GRU (Bai et al., 2018)Test perplexity92.48—Unverified
4R-TransformerTest perplexity84.38—Unverified
5Zaremba et al. (2014) - LSTM (medium)Test perplexity82.7—Unverified
6Gal & Ghahramani (2016) - Variational LSTM (medium)Test perplexity79.7—Unverified
7LSTM (Bai et al., 2018)Test perplexity78.93—Unverified
8Zaremba et al. (2014) - LSTM (large)Test perplexity78.4—Unverified
9Gal & Ghahramani (2016) - Variational LSTM (large)Test perplexity75.2—Unverified
10Inan et al. (2016) - Variational RHNTest perplexity66—Unverified
#ModelMetricClaimedVerifiedStatus
1LSTM (7 layers)Bit per Character (BPC)1.67—Unverified
2HypernetworksBit per Character (BPC)1.34—Unverified
3SHA-LSTM (4 layers, h=1024, no attention head)Bit per Character (BPC)1.33—Unverified
4LN HM-LSTMBit per Character (BPC)1.32—Unverified
5ByteNetBit per Character (BPC)1.31—Unverified
6Recurrent Highway NetworksBit per Character (BPC)1.27—Unverified
7Large FS-LSTM-4Bit per Character (BPC)1.25—Unverified
8Large mLSTMBit per Character (BPC)1.24—Unverified
9AWD-LSTM (3 layers)Bit per Character (BPC)1.23—Unverified
10Cluster-Former (#C=512)Bit per Character (BPC)1.22—Unverified
#ModelMetricClaimedVerifiedStatus
1Smaller Transformer 126M (pre-trained)Test perplexity33—Unverified
2OPT 125MTest perplexity32.26—Unverified
3Larger Transformer 771M (pre-trained)Test perplexity28.1—Unverified
4OPT 1.3BTest perplexity19.55—Unverified
5GPT-Neo 125MTest perplexity17.83—Unverified
6OPT 2.7BTest perplexity17.81—Unverified
7Smaller Transformer 126M (fine-tuned)Test perplexity12—Unverified
8GPT-Neo 1.3BTest perplexity11.46—Unverified
9Transformer 125MTest perplexity10.7—Unverified
10GPT-Neo 2.7BTest perplexity10.44—Unverified