SOTAVerified

Language Modelling

A language model is a model of natural language. Language models are useful for a variety of tasks, including speech recognition, machine translation, natural language generation (generating more human-like text), optical character recognition, route optimization, handwriting recognition, grammar induction, and information retrieval.

Large language models (LLMs), currently their most advanced form, are predominantly based on transformers trained on larger datasets (frequently using words scraped from the public internet). They have superseded recurrent neural network-based models, which had previously superseded the purely statistical models, such as word n-gram language model.

Source: Wikipedia

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 6901–6950 of 17610 papers

TitleStatusHype
ScaffoldGPT: A Scaffold-based GPT Model for Drug Optimization—0
Scalable and Accurate Self-supervised Multimodal Representation Learning without Aligned Video and Text Data—0
Scalable and Transferable Black-Box Jailbreaks for Language Models via Persona Modulation—0
Scalable Bayesian Learning of Recurrent Neural Networks for Language Modeling—0
Scalable Ensembling For Mitigating Reward Overoptimisation—0
Scalable language model adaptation for spoken dialogue systems—0
Scalable Language Models with Posterior Inference of Latent Thought Vectors—0
LPNL: Scalable Link Prediction with Large Language Models—0
Scalable LLM Math Reasoning Acceleration with Low-rank Distillation—0
Scalable Modified Kneser-Ney Language Model Estimation—0
Scalable Neural Learning for Verifiable Consistency with Temporal Specifications—0
Scalable Syntax-Aware Language Models Using Knowledge Distillation—0
Scalable Vision Language Model Training via High Quality Data Curation—0
Scaling Context, Not Parameters: Training a Compact 7B Language Model for Efficient Long-Context Processing—0
Scaling Embedding Layers in Language Models—0
Scaling Language Model Size in Cross-Device Federated Learning—0
Scaling Large Language Model Training on Frontier with Low-Bandwidth Partitioning—0
Scaling Laws for Adversarial Attacks on Language Model Activations—0
Scaling Laws for Deep Learning—0
Scaling Laws for Differentially Private Language Models—0
Scaling Laws for Discriminative Classification in Large Language Models—0
Scaling Laws for Economic Productivity: Experimental Evidence in LLM-Assisted Translation—0
Scaling Laws for Pre-training Agents and World Models—0
Scaling Law with Learning Rate Annealing—0
Scaling Memory-Augmented Neural Networks with Sparse Reads and Writes—0
Scaling Parameter-Constrained Language Models with Quality Data—0
Scaling Recurrent Neural Network Language Models—0
Scaling Studies for Efficient Parameter Search and Parallelism for Large Language Model Pre-training—0
Scaling Technology Acceptance Analysis with Large Language Model (LLM) Annotation Systems—0
Scaling Up Summarization: Leveraging Large Language Models for Long Text Extractive Summarization—0
SCCA: Shifted Cross Chunk Attention for long contextual semantic expansion—0
Scenario-based Multi-product Advertising Copywriting Generation for E-Commerce—0
SceneCraft: An LLM Agent for Synthesizing 3D Scene as Blender Code—0
SceneGPT: A Language Model for 3D Scene Understanding—0
Scene-LLM: Extending Language Model for 3D Visual Understanding and Reasoning—0
SceneScript: Reconstructing Scenes With An Autoregressive Structured Language Model—0
Scene Text Recognition with Image-Text Matching-guided Dictionary—0
Scene Transformer: A unified architecture for predicting future trajectories of multiple agents—0
SceneX: Procedural Controllable Large-scale Scene Generation—0
SCE: Scalable Consistency Ensembles Make Blackbox Large Language Model Generation More Reliable—0
Schema Augmentation for Zero-Shot Domain Adaptation in Dialogue State Tracking—0
Schema-Free Dependency Parsing via Sequence Generation—0
Schema-Free Dependency Parsing via Sequence Generation—0
Schema Graph-Guided Prompt for Multi-Domain Dialogue State Tracking—0
Schemato -- An LLM for Netlist-to-Schematic Conversion—0
The Diminishing Returns of Masked Language Models to Science—0
SciDFM: A Large Language Model with Mixture-of-Experts for Science—0
From Complexity to Clarity: How AI Enhances Perceptions of Scientists and the Public's Understanding of Science—0
Scientific Hypothesis Generation by a Large Language Model: Laboratory Validation in Breast Cancer Treatment—0
scInterpreter: Training Large Language Models to Interpret scRNA-seq Data for Cell Type Annotation—0
Show:102550
← PrevPage 139 of 353Next →
All datasetsWikiText-103Penn Treebank (Word Level)enwik8The PileWikiText-2LAMBADAOne Billion WordText8Penn Treebank (Character Level)Hutter PrizeOpenWebTextSALMon

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1Decay RNNValidation perplexity76.67—Unverified
2GRUValidation perplexity53.78—Unverified
3LSTMValidation perplexity52.73—Unverified
4LSTMTest perplexity48.7—Unverified
5Temporal CNNTest perplexity45.2—Unverified
6TCNTest perplexity45.19—Unverified
7GCNN-8Test perplexity44.9—Unverified
8Neural cache model (size = 100)Test perplexity44.8—Unverified
9Neural cache model (size = 2,000)Test perplexity40.8—Unverified
10GPT-2 SmallTest perplexity37.5—Unverified
#ModelMetricClaimedVerifiedStatus
1TCNTest perplexity108.47—Unverified
2Seq-U-NetTest perplexity107.95—Unverified
3GRU (Bai et al., 2018)Test perplexity92.48—Unverified
4R-TransformerTest perplexity84.38—Unverified
5Zaremba et al. (2014) - LSTM (medium)Test perplexity82.7—Unverified
6Gal & Ghahramani (2016) - Variational LSTM (medium)Test perplexity79.7—Unverified
7LSTM (Bai et al., 2018)Test perplexity78.93—Unverified
8Zaremba et al. (2014) - LSTM (large)Test perplexity78.4—Unverified
9Gal & Ghahramani (2016) - Variational LSTM (large)Test perplexity75.2—Unverified
10Inan et al. (2016) - Variational RHNTest perplexity66—Unverified
#ModelMetricClaimedVerifiedStatus
1LSTM (7 layers)Bit per Character (BPC)1.67—Unverified
2HypernetworksBit per Character (BPC)1.34—Unverified
3SHA-LSTM (4 layers, h=1024, no attention head)Bit per Character (BPC)1.33—Unverified
4LN HM-LSTMBit per Character (BPC)1.32—Unverified
5ByteNetBit per Character (BPC)1.31—Unverified
6Recurrent Highway NetworksBit per Character (BPC)1.27—Unverified
7Large FS-LSTM-4Bit per Character (BPC)1.25—Unverified
8Large mLSTMBit per Character (BPC)1.24—Unverified
9AWD-LSTM (3 layers)Bit per Character (BPC)1.23—Unverified
10Cluster-Former (#C=512)Bit per Character (BPC)1.22—Unverified
#ModelMetricClaimedVerifiedStatus
1Smaller Transformer 126M (pre-trained)Test perplexity33—Unverified
2OPT 125MTest perplexity32.26—Unverified
3Larger Transformer 771M (pre-trained)Test perplexity28.1—Unverified
4OPT 1.3BTest perplexity19.55—Unverified
5GPT-Neo 125MTest perplexity17.83—Unverified
6OPT 2.7BTest perplexity17.81—Unverified
7Smaller Transformer 126M (fine-tuned)Test perplexity12—Unverified
8GPT-Neo 1.3BTest perplexity11.46—Unverified
9Transformer 125MTest perplexity10.7—Unverified
10GPT-Neo 2.7BTest perplexity10.44—Unverified