SOTAVerified

Language Modelling

A language model is a model of natural language. Language models are useful for a variety of tasks, including speech recognition, machine translation, natural language generation (generating more human-like text), optical character recognition, route optimization, handwriting recognition, grammar induction, and information retrieval.

Large language models (LLMs), currently their most advanced form, are predominantly based on transformers trained on larger datasets (frequently using words scraped from the public internet). They have superseded recurrent neural network-based models, which had previously superseded the purely statistical models, such as word n-gram language model.

Source: Wikipedia

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 3251–3300 of 17610 papers

TitleStatusHype
PLaMo-100B: A Ground-Up Language Model Designed for Japanese Proficiency—0
TurboRAG: Accelerating Retrieval-Augmented Generation with Precomputed KV Caches for Chunked TextCode2
OneNet: A Fine-Tuning Free Framework for Few-Shot Entity Linking via Large Language Model PromptingCode1
Efficiently Learning at Test-Time: Active Fine-Tuning of LLMsCode2
Sample then Identify: A General Framework for Risk Control and Assessment in Multimodal Large Language Models—0
Recent advancements in LLM Red-Teaming: Techniques, Defenses, and Ethical Considerations—0
Generating long-horizon stock "buy" signals with a neural language model—0
QuAILoRA: Quantization-Aware Initialization for LoRA—0
Exploring Prompt Engineering: A Systematic Review with SWOT Analysis—0
TinyClick: Single-Turn Agent for Empowering GUI Automation—0
AuditWen:An Open-Source Large Language Model for AuditCode1
Enhancing Vision-Language Model Pre-training with Image-text Pair Pruning Based on Word FrequencyCode0
Multi-Task Program Error Repair and Explanatory Diagnosis—0
Exploring Efficient Foundational Multi-modal Models for Video Summarization—0
Towards Universality: Studying Mechanistic Similarity Across Language Model Architectures—0
Let's Ask GNN: Empowering Large Language Model for Graph In-Context Learning—0
Sylber: Syllabic Embedding Representation of Speech from Raw AudioCode2
Towards Interpreting Visual Information Processing in Vision-Language ModelsCode2
Large Language Model Compression with Neural Architecture Search—0
Pixtral 12BCode11
Boosting Few-Shot Detection with Large Language Models and Layout-to-Image Synthesis—0
TinyEmo: Scaling down Emotional Reasoning via Metric ProjectionCode0
β-calibration of Language Model Confidence Scores for Generative QA—0
Compositional Entailment Learning for Hyperbolic Vision-Language ModelsCode2
Simplicity Prevails: Rethinking Negative Preference Optimization for LLM UnlearningCode1
Reproducing and Extending Experiments in Behavioral Strategy with Large Language Models—0
FltLM: An Intergrated Long-Context Large Language Model for Effective Context Filtering and Understanding—0
Joint Fine-tuning and Conversion of Pretrained Speech and Language Models towards Linear ComplexityCode0
Personal Intelligence System UniLM: Hybrid On-Device Small Language Model and Server-Based Large Language Model for Malay Nusantara—0
Uncovering Factor Level Preferences to Improve Human-Model Alignment—0
Stuffed Mamba: State Collapse and State Capacity of RNN-Based Long-Context Modeling—0
Applying Refusal-Vector Ablation to Llama 3.1 70B Agents—0
Application of NotebookLM, a Large Language Model with Retrieval-Augmented Generation, for Lung Cancer Staging—0
Enhancing SPARQL Generation by Triplet-order-sensitive Pre-trainingCode0
BUMBLE: Unifying Reasoning and Acting with Vision-Language Models for Building-wide Mobile ManipulationCode2
Retrieving, Rethinking and Revising: The Chain-of-Verification Can Improve Retrieval Augmented Generation—0
DecorateLM: Data Engineering through Corpus Rating, Tagging, and Editing with Language Models—0
Vector-ICL: In-context Learning with Continuous Vector RepresentationsCode1
Accelerated Preference Optimization for Large Language Model Alignment—0
ParallelSpec: Parallel Drafter for Efficient Speculative Decoding—0
FG-PRM: Fine-grained Hallucination Detection and Mitigation in Language Model Mathematical Reasoning—0
Think While You Generate: Discrete Diffusion with Planned DenoisingCode2
Jet Expansions of Residual Computation—0
ClaimBrush: A Novel Framework for Automated Patent Claim Refinement Based on Large Language Models—0
Training-free Diffusion Model Alignment with Sampling DemonsCode1
RL, but don't do anything I wouldn't doCode0
A second-order-like optimizer with adaptive gradient scaling for deep learningCode0
TeaserGen: Generating Teasers for Long Documentaries—0
TapType: Ten-finger text entry on everyday surfaces via Bayesian inference—0
Multi-Session Client-Centered Treatment Outcome Evaluation in Psychotherapy—0
Show:102550
← PrevPage 66 of 353Next →
All datasetsWikiText-103Penn Treebank (Word Level)enwik8The PileWikiText-2LAMBADAOne Billion WordText8Penn Treebank (Character Level)Hutter PrizeOpenWebTextSALMon

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1Decay RNNValidation perplexity76.67—Unverified
2GRUValidation perplexity53.78—Unverified
3LSTMValidation perplexity52.73—Unverified
4LSTMTest perplexity48.7—Unverified
5Temporal CNNTest perplexity45.2—Unverified
6TCNTest perplexity45.19—Unverified
7GCNN-8Test perplexity44.9—Unverified
8Neural cache model (size = 100)Test perplexity44.8—Unverified
9Neural cache model (size = 2,000)Test perplexity40.8—Unverified
10GPT-2 SmallTest perplexity37.5—Unverified
#ModelMetricClaimedVerifiedStatus
1TCNTest perplexity108.47—Unverified
2Seq-U-NetTest perplexity107.95—Unverified
3GRU (Bai et al., 2018)Test perplexity92.48—Unverified
4R-TransformerTest perplexity84.38—Unverified
5Zaremba et al. (2014) - LSTM (medium)Test perplexity82.7—Unverified
6Gal & Ghahramani (2016) - Variational LSTM (medium)Test perplexity79.7—Unverified
7LSTM (Bai et al., 2018)Test perplexity78.93—Unverified
8Zaremba et al. (2014) - LSTM (large)Test perplexity78.4—Unverified
9Gal & Ghahramani (2016) - Variational LSTM (large)Test perplexity75.2—Unverified
10Inan et al. (2016) - Variational RHNTest perplexity66—Unverified
#ModelMetricClaimedVerifiedStatus
1LSTM (7 layers)Bit per Character (BPC)1.67—Unverified
2HypernetworksBit per Character (BPC)1.34—Unverified
3SHA-LSTM (4 layers, h=1024, no attention head)Bit per Character (BPC)1.33—Unverified
4LN HM-LSTMBit per Character (BPC)1.32—Unverified
5ByteNetBit per Character (BPC)1.31—Unverified
6Recurrent Highway NetworksBit per Character (BPC)1.27—Unverified
7Large FS-LSTM-4Bit per Character (BPC)1.25—Unverified
8Large mLSTMBit per Character (BPC)1.24—Unverified
9AWD-LSTM (3 layers)Bit per Character (BPC)1.23—Unverified
10Cluster-Former (#C=512)Bit per Character (BPC)1.22—Unverified
#ModelMetricClaimedVerifiedStatus
1Smaller Transformer 126M (pre-trained)Test perplexity33—Unverified
2OPT 125MTest perplexity32.26—Unverified
3Larger Transformer 771M (pre-trained)Test perplexity28.1—Unverified
4OPT 1.3BTest perplexity19.55—Unverified
5GPT-Neo 125MTest perplexity17.83—Unverified
6OPT 2.7BTest perplexity17.81—Unverified
7Smaller Transformer 126M (fine-tuned)Test perplexity12—Unverified
8GPT-Neo 1.3BTest perplexity11.46—Unverified
9Transformer 125MTest perplexity10.7—Unverified
10GPT-Neo 2.7BTest perplexity10.44—Unverified