SOTAVerified

Language Modelling

A language model is a model of natural language. Language models are useful for a variety of tasks, including speech recognition, machine translation, natural language generation (generating more human-like text), optical character recognition, route optimization, handwriting recognition, grammar induction, and information retrieval.

Large language models (LLMs), currently their most advanced form, are predominantly based on transformers trained on larger datasets (frequently using words scraped from the public internet). They have superseded recurrent neural network-based models, which had previously superseded the purely statistical models, such as word n-gram language model.

Source: Wikipedia

Papers

Showing 801850 of 17610 papers

TitleStatusHype
LLM-A*: Large Language Model Enhanced Incremental Heuristic Search on Path PlanningCode2
Encoder vs Decoder: Comparative Analysis of Encoder and Decoder Language Models on Multilingual NLU TasksCode2
Breaking the Ceiling of the LLM Community by Treating Token Generation as a Classification for EnsemblingCode2
Holmes-VAD: Towards Unbiased and Explainable Video Anomaly Detection via Multi-modal LLMCode2
AgentReview: Exploring Peer Review Dynamics with LLM AgentsCode2
Talk With Human-like Agents: Empathetic Dialogue Through Perceptible Acoustic Reception and ReactionCode2
Watch Every Step! LLM Agent Learning via Iterative Step-Level Process RefinementCode2
ISR-DPO: Aligning Large Multimodal Models for Videos by Iterative Self-Retrospective DPOCode2
Large Scale Transfer Learning for Tabular Data via Language ModelingCode2
mDPO: Conditional Preference Optimization for Multimodal Large Language ModelsCode2
GAMA: A Large Audio-Language Model with Advanced Audio Understanding and Complex Reasoning AbilitiesCode2
BEACON: Benchmark for Comprehensive RNA Tasks and Language ModelsCode2
On Softmax Direct Preference Optimization for RecommendationCode2
StreamBench: Towards Benchmarking Continuous Improvement of Language AgentsCode2
Enhancing Diagnostic Accuracy in Rare and Common Fundus Diseases with a Knowledge-Rich Vision-Language ModelCode2
Explore the Limits of Omni-modal Pretraining at ScaleCode2
Discovering Preference Optimization Algorithms with and for Large Language ModelsCode2
RS-Agent: Automating Remote Sensing Tasks through Intelligent AgentCode2
MATES: Model-Aware Data Selection for Efficient Pretraining with Data Influence ModelsCode2
LocLLM: Exploiting Generalizable Human Keypoint Localization via Large Language ModelCode2
Simplified and Generalized Masked Diffusion for Discrete DataCode2
BLSP-Emo: Towards Empathetic Large Speech-Language ModelsCode2
Tool-Planner: Task Planning with Clusters across Multiple ToolsCode2
Jailbreak Vision Language Models via Bi-Modal Adversarial PromptCode2
Your Absorbing Discrete Diffusion Secretly Models the Conditional Distributions of Clean DataCode2
Small-E: Small Language Model with Linear Attention for Efficient Speech SynthesisCode2
PosterLLaVa: Constructing a Unified Multi-modal Layout Generator with LLMCode2
Pruner-Zero: Evolving Symbolic Pruning Metric from scratch for Large Language ModelsCode2
Visual-Text Cross Alignment: Refining the Similarity Score in Vision-Language ModelsCode2
DriVLMe: Enhancing LLM-based Autonomous Driving Agents with Embodied and Social ExperiencesCode2
Block Transformer: Global-to-Local Language Modeling for Fast InferenceCode2
From Redundancy to Relevance: Information Flow in LVLMs Across Reasoning TasksCode2
Generative Pre-trained Speech Language Model with Efficient Hierarchical TransformerCode2
TabPedia: Towards Comprehensive Visual Table Understanding with Concept SynergyCode2
The Geometry of Categorical and Hierarchical Concepts in Large Language ModelsCode2
SUBLLM: A Novel Efficient Architecture with Token Sequence Subsampling for LLMCode2
GeoReasoner: Geo-localization with Reasoning in Street Views using a Large Vision-Language ModelCode2
Dragonfly: Multi-Resolution Zoom-In Encoding Enhances Vision-Language ModelsCode2
Helix: Serving Large Language Models over Heterogeneous GPUs and Network via Max-FlowCode2
Aligning Language Models with Demonstrated FeedbackCode2
Query2CAD: Generating CAD models using natural language queriesCode2
ABodyBuilder3: Improved and scalable antibody structure predictionsCode2
LLaMEA: A Large Language Model Evolutionary Algorithm for Automatically Generating MetaheuristicsCode2
Weak-to-Strong Search: Align Large Language Models via Searching over Small Language ModelsCode2
Matryoshka Query Transformer for Large Vision-Language ModelsCode2
Benchmarking and Improving Detail Image CaptionCode2
Aligning to Thousands of Preferences via System Message GeneralizationCode2
Knowledge Circuits in Pretrained TransformersCode2
Reason3D: Searching and Reasoning 3D Segmentation via Large Language ModelCode2
Motion-Agent: A Conversational Framework for Human Motion Generation with LLMsCode2
Show:102550
← PrevPage 17 of 353Next →

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1Decay RNNValidation perplexity76.67Unverified
2GRUValidation perplexity53.78Unverified
3LSTMValidation perplexity52.73Unverified
4LSTMTest perplexity48.7Unverified
5Temporal CNNTest perplexity45.2Unverified
6TCNTest perplexity45.19Unverified
7GCNN-8Test perplexity44.9Unverified
8Neural cache model (size = 100)Test perplexity44.8Unverified
9Neural cache model (size = 2,000)Test perplexity40.8Unverified
10GPT-2 SmallTest perplexity37.5Unverified
#ModelMetricClaimedVerifiedStatus
1TCNTest perplexity108.47Unverified
2Seq-U-NetTest perplexity107.95Unverified
3GRU (Bai et al., 2018)Test perplexity92.48Unverified
4R-TransformerTest perplexity84.38Unverified
5Zaremba et al. (2014) - LSTM (medium)Test perplexity82.7Unverified
6Gal & Ghahramani (2016) - Variational LSTM (medium)Test perplexity79.7Unverified
7LSTM (Bai et al., 2018)Test perplexity78.93Unverified
8Zaremba et al. (2014) - LSTM (large)Test perplexity78.4Unverified
9Gal & Ghahramani (2016) - Variational LSTM (large)Test perplexity75.2Unverified
10Inan et al. (2016) - Variational RHNTest perplexity66Unverified
#ModelMetricClaimedVerifiedStatus
1LSTM (7 layers)Bit per Character (BPC)1.67Unverified
2HypernetworksBit per Character (BPC)1.34Unverified
3SHA-LSTM (4 layers, h=1024, no attention head)Bit per Character (BPC)1.33Unverified
4LN HM-LSTMBit per Character (BPC)1.32Unverified
5ByteNetBit per Character (BPC)1.31Unverified
6Recurrent Highway NetworksBit per Character (BPC)1.27Unverified
7Large FS-LSTM-4Bit per Character (BPC)1.25Unverified
8Large mLSTMBit per Character (BPC)1.24Unverified
9AWD-LSTM (3 layers)Bit per Character (BPC)1.23Unverified
10Cluster-Former (#C=512)Bit per Character (BPC)1.22Unverified
#ModelMetricClaimedVerifiedStatus
1Smaller Transformer 126M (pre-trained)Test perplexity33Unverified
2OPT 125MTest perplexity32.26Unverified
3Larger Transformer 771M (pre-trained)Test perplexity28.1Unverified
4OPT 1.3BTest perplexity19.55Unverified
5GPT-Neo 125MTest perplexity17.83Unverified
6OPT 2.7BTest perplexity17.81Unverified
7Smaller Transformer 126M (fine-tuned)Test perplexity12Unverified
8GPT-Neo 1.3BTest perplexity11.46Unverified
9Transformer 125MTest perplexity10.7Unverified
10GPT-Neo 2.7BTest perplexity10.44Unverified