SOTAVerified

Language Modelling

A language model is a model of natural language. Language models are useful for a variety of tasks, including speech recognition, machine translation, natural language generation (generating more human-like text), optical character recognition, route optimization, handwriting recognition, grammar induction, and information retrieval.

Large language models (LLMs), currently their most advanced form, are predominantly based on transformers trained on larger datasets (frequently using words scraped from the public internet). They have superseded recurrent neural network-based models, which had previously superseded the purely statistical models, such as word n-gram language model.

Source: Wikipedia

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 5801–5850 of 17610 papers

TitleStatusHype
Align and Aggregate: Compositional Reasoning with Video Alignment and Answer Aggregation for Video Question-Answering—0
AlignCap: Aligning Speech Emotion Captioning to Human Preferences—0
AlignDistil: Token-Level Language Model Alignment as Adaptive Policy Distillation—0
Aligned Weight Regularizers for Pruning Pretrained Neural Networks—0
Aligned Weight Regularizers for Pruning Pretrained Neural Networks—0
AlignGPT: Multi-modal Large Language Models with Adaptive Alignment Capability—0
Aligning context-based statistical models of language with brain activity during reading—0
Aligning Crowd-sourced Human Feedback for Reinforcement Learning on Code Generation by Large Language Models—0
Aligning Dialogue Agents with Global Feedback via Large Language Model Reward Decomposition—0
Aligning Human Knowledge with Visual Concepts Towards Explainable Medical Image Classification—0
Aligning Large Multimodal Models with Factually Augmented RLHF—0
Aligning MAGMA by Few-Shot Learning and Finetuning—0
Aligning Model Evaluations with Human Preferences: Mitigating Token Count Bias in Language Model Assessments—0
Aligning Neural Machine Translation Models: Human Feedback in Training and Inference—0
Aligning Speech to Languages to Enhance Code-switching Speech Recognition—0
Unsupervised Large Language Model Alignment for Information Retrieval via Contrastive Feedback—0
Aligning the Pretraining and Finetuning Objectives of Language Models—0
Aligning Visual Prototypes with BERT Embeddings for Few-Shot Learning—0
Alignment at Work: Using Language to Distinguish the Internalization and Self-Regulation Components of Cultural Fit in Organizations—0
A Linear Baseline Classifier for Cross-Lingual Pronoun Prediction—0
A Linear Dynamical System Model for Text—0
ALISA: Accelerating Large Language Model Inference via Sparsity-Aware KV Caching—0
ALISE: Accelerating Large Language Model Serving with Speculative Scheduling—0
A Literature Review of Literature Reviews in Pattern Analysis and Machine Intelligence—0
A Little Help Goes a Long Way: Efficient LLM Training by Leveraging Small LMs—0
ALLaM: Large Language Models for Arabic and English—0
A LLM Assisted Exploitation of AI-Guardian—0
All or None: Identifiable Linear Properties of Next-token Predictors in Language Modeling—0
ALMA: Alignment with Minimal Annotation—0
Almanac: Retrieval-Augmented Language Models for Clinical Medicine—0
Almost Unsupervised Text to Speech and Automatic Speech Recognition—0
A Log-Linear Model for Unsupervised Text Normalization—0
AlphaTuning: Quantization-Aware Parameter-Efficient Adaptation of Large-Scale Pre-Trained Language Models—0
AL-QASIDA: Analyzing LLM Quality and Accuracy Systematically in Dialectal Arabic—0
Alternate Endings: Improving Prosody for Incremental Neural TTS with Predicted Future Text Input—0
Alternating Recurrent Dialog Model with Large-scale Pre-trained Language Models—0
Alzheimer's Dementia Detection Using Perplexity from Paired Large Language Models—0
A Machine Learning Method to Distinguish Machine Translation from Human Translation—0
A machine translation system combining rule-based machine translation and statistical post-editing—0
A Masked language model for multi-source EHR trajectories contextual representation learning—0
A Masked Segmental Language Model for Unsupervised Natural Language Segmentation—0
A Mathematical Exploration of Why Language Models Help Solve Downstream Tasks—0
AMBERT: A Pre-trained Language Model with Multi-Grained Tokenization—0
AMD:Anatomical Motion Diffusion with Interpretable Motion Decomposition and Fusion—0
A Measurement-Based Quantum-Like Language Model for Text Matching—0
A Measure-Theoretic Characterization of Tight Language Models—0
A Mechanistic Explanatory Strategy for XAI—0
A Medical Multimodal Large Language Model for Pediatric Pneumonia—0
A Memory-Augmented LLM-Driven Method for Autonomous Merging of 3D Printing Work Orders—0
American Stories: A Large-Scale Structured Text Dataset of Historical U.S. Newspapers—0
Show:102550
← PrevPage 117 of 353Next →
All datasetsWikiText-103Penn Treebank (Word Level)enwik8The PileWikiText-2LAMBADAOne Billion WordText8Penn Treebank (Character Level)Hutter PrizeOpenWebTextSALMon

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1Decay RNNValidation perplexity76.67—Unverified
2GRUValidation perplexity53.78—Unverified
3LSTMValidation perplexity52.73—Unverified
4LSTMTest perplexity48.7—Unverified
5Temporal CNNTest perplexity45.2—Unverified
6TCNTest perplexity45.19—Unverified
7GCNN-8Test perplexity44.9—Unverified
8Neural cache model (size = 100)Test perplexity44.8—Unverified
9Neural cache model (size = 2,000)Test perplexity40.8—Unverified
10GPT-2 SmallTest perplexity37.5—Unverified
#ModelMetricClaimedVerifiedStatus
1TCNTest perplexity108.47—Unverified
2Seq-U-NetTest perplexity107.95—Unverified
3GRU (Bai et al., 2018)Test perplexity92.48—Unverified
4R-TransformerTest perplexity84.38—Unverified
5Zaremba et al. (2014) - LSTM (medium)Test perplexity82.7—Unverified
6Gal & Ghahramani (2016) - Variational LSTM (medium)Test perplexity79.7—Unverified
7LSTM (Bai et al., 2018)Test perplexity78.93—Unverified
8Zaremba et al. (2014) - LSTM (large)Test perplexity78.4—Unverified
9Gal & Ghahramani (2016) - Variational LSTM (large)Test perplexity75.2—Unverified
10Inan et al. (2016) - Variational RHNTest perplexity66—Unverified
#ModelMetricClaimedVerifiedStatus
1LSTM (7 layers)Bit per Character (BPC)1.67—Unverified
2HypernetworksBit per Character (BPC)1.34—Unverified
3SHA-LSTM (4 layers, h=1024, no attention head)Bit per Character (BPC)1.33—Unverified
4LN HM-LSTMBit per Character (BPC)1.32—Unverified
5ByteNetBit per Character (BPC)1.31—Unverified
6Recurrent Highway NetworksBit per Character (BPC)1.27—Unverified
7Large FS-LSTM-4Bit per Character (BPC)1.25—Unverified
8Large mLSTMBit per Character (BPC)1.24—Unverified
9AWD-LSTM (3 layers)Bit per Character (BPC)1.23—Unverified
10Cluster-Former (#C=512)Bit per Character (BPC)1.22—Unverified
#ModelMetricClaimedVerifiedStatus
1Smaller Transformer 126M (pre-trained)Test perplexity33—Unverified
2OPT 125MTest perplexity32.26—Unverified
3Larger Transformer 771M (pre-trained)Test perplexity28.1—Unverified
4OPT 1.3BTest perplexity19.55—Unverified
5GPT-Neo 125MTest perplexity17.83—Unverified
6OPT 2.7BTest perplexity17.81—Unverified
7Smaller Transformer 126M (fine-tuned)Test perplexity12—Unverified
8GPT-Neo 1.3BTest perplexity11.46—Unverified
9Transformer 125MTest perplexity10.7—Unverified
10GPT-Neo 2.7BTest perplexity10.44—Unverified