SOTAVerified

Language Modelling

A language model is a model of natural language. Language models are useful for a variety of tasks, including speech recognition, machine translation, natural language generation (generating more human-like text), optical character recognition, route optimization, handwriting recognition, grammar induction, and information retrieval.

Large language models (LLMs), currently their most advanced form, are predominantly based on transformers trained on larger datasets (frequently using words scraped from the public internet). They have superseded recurrent neural network-based models, which had previously superseded the purely statistical models, such as word n-gram language model.

Source: Wikipedia

Papers

Showing 901950 of 17610 papers

TitleStatusHype
LLM-Seg: Bridging Image Segmentation and Large Language Model ReasoningCode2
Behavior Trees Enable Structured Programming of Language Model AgentsCode2
HGRN2: Gated Linear RNNs with State ExpansionCode2
LaVy: Vietnamese Multimodal Large Language ModelCode2
From Words to Numbers: Your Large Language Model Is Secretly A Capable Regressor When Given In-Context ExamplesCode2
UMBRAE: Unified Multimodal Brain DecodingCode2
Optimization Methods for Personalizing Large Language Models through Retrieval AugmentationCode2
Test-Time Zero-Shot Temporal Action LocalizationCode2
MotionChain: Conversational Motion Controllers via Multimodal PromptsCode2
Stream of Search (SoS): Learning to Search in LanguageCode2
Direct Preference Optimization of Video Large Multimodal Models from Language Model RewardCode2
ARAGOG: Advanced RAG Output GradingCode2
Draw-and-Understand: Leveraging Visual Prompts to Enable MLLMs to Comprehend What You WantCode2
VHM: Versatile and Honest Vision Language Model for Remote Sensing Image AnalysisCode2
Multi-Frame, Lightweight & Efficient Vision-Language Models for Question Answering in Autonomous DrivingCode2
Change-Agent: Towards Interactive Comprehensive Remote Sensing Change Interpretation and AnalysisCode2
Can Language Beat Numerical Regression? Language-Based Multimodal Trajectory PredictionCode2
An Image Grid Can Be Worth a Video: Zero-shot Video Question Answering Using a VLMCode2
Have Faith in Faithfulness: Going Beyond Circuit Overlap When Finding Model MechanismsCode2
MIND Your Language: A Multilingual Dataset for Cross-lingual News RecommendationCode2
Data Mixing Laws: Optimizing Data Mixtures by Predicting Language Modeling PerformanceCode2
DreamLIP: Language-Image Pre-training with Long CaptionsCode2
RepairAgent: An Autonomous, LLM-Based Agent for Program RepairCode2
Understanding Long Videos with Multimodal Language ModelsCode2
LLaVA-PruMerge: Adaptive Token Reduction for Efficient Large Multimodal ModelsCode2
LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language ModelsCode2
Cross-Domain Pre-training with Language Models for Transferable Time Series RepresentationsCode2
Advancing Time Series Classification with Multimodal Language ModelingCode2
LLM3:Large Language Model-based Task and Motion Planning with Motion Failure ReasoningCode2
SelfIE: Self-Interpretation of Large Language Model EmbeddingsCode2
Generative Region-Language Pretraining for Open-Ended Object DetectionCode2
VideoAgent: Long-form Video Understanding with Large Language Model as AgentCode2
What Was Your Prompt? A Remote Keylogging Attack on AI AssistantsCode2
LLM-Assisted Light: Leveraging Large Language Model Capabilities for Human-Mimetic Traffic Signal Control in Complex Urban EnvironmentsCode2
Generative Pretrained Structured Transformers: Unsupervised Syntactic Language Models at ScaleCode2
SOTOPIA-π: Interactive Learning of Socially Intelligent Language AgentsCode2
Language models scale reliably with over-training and on downstream tasksCode2
CoIN: A Benchmark of Continual Instruction tuNing for Multimodel Large Language ModelCode2
Characterization of Large Language Model Development in the DatacenterCode2
Decomposing Disease Descriptions for Enhanced Pathology Detection: A Multi-Aspect Vision-Language Pre-training FrameworkCode2
VLKEB: A Large Vision-Language Model Knowledge Editing BenchmarkCode2
KnowCoder: Coding Structured Knowledge into LLMs for Universal Information ExtractionCode2
Beyond Text: Frozen Large Language Models in Visual Signal ComprehensionCode2
Monitoring AI-Modified Content at Scale: A Case Study on the Impact of ChatGPT on AI Conference Peer ReviewsCode2
Smart-Infinity: Fast Large Language Model Training using Near-Storage Processing on a Real SystemCode2
CAT: Enhancing Multimodal Large Language Model to Answer Questions in Dynamic Audio-Visual ScenariosCode2
Online Adaptation of Language Models with a Memory of Amortized ContextsCode2
Backtracing: Retrieving the Cause of the QueryCode2
MeaCap: Memory-Augmented Zero-shot Image CaptioningCode2
ESM All-Atom: Multi-scale Protein Language Model for Unified Molecular ModelingCode2
Show:102550
← PrevPage 19 of 353Next →

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1Decay RNNValidation perplexity76.67Unverified
2GRUValidation perplexity53.78Unverified
3LSTMValidation perplexity52.73Unverified
4LSTMTest perplexity48.7Unverified
5Temporal CNNTest perplexity45.2Unverified
6TCNTest perplexity45.19Unverified
7GCNN-8Test perplexity44.9Unverified
8Neural cache model (size = 100)Test perplexity44.8Unverified
9Neural cache model (size = 2,000)Test perplexity40.8Unverified
10GPT-2 SmallTest perplexity37.5Unverified
#ModelMetricClaimedVerifiedStatus
1TCNTest perplexity108.47Unverified
2Seq-U-NetTest perplexity107.95Unverified
3GRU (Bai et al., 2018)Test perplexity92.48Unverified
4R-TransformerTest perplexity84.38Unverified
5Zaremba et al. (2014) - LSTM (medium)Test perplexity82.7Unverified
6Gal & Ghahramani (2016) - Variational LSTM (medium)Test perplexity79.7Unverified
7LSTM (Bai et al., 2018)Test perplexity78.93Unverified
8Zaremba et al. (2014) - LSTM (large)Test perplexity78.4Unverified
9Gal & Ghahramani (2016) - Variational LSTM (large)Test perplexity75.2Unverified
10Inan et al. (2016) - Variational RHNTest perplexity66Unverified
#ModelMetricClaimedVerifiedStatus
1LSTM (7 layers)Bit per Character (BPC)1.67Unverified
2HypernetworksBit per Character (BPC)1.34Unverified
3SHA-LSTM (4 layers, h=1024, no attention head)Bit per Character (BPC)1.33Unverified
4LN HM-LSTMBit per Character (BPC)1.32Unverified
5ByteNetBit per Character (BPC)1.31Unverified
6Recurrent Highway NetworksBit per Character (BPC)1.27Unverified
7Large FS-LSTM-4Bit per Character (BPC)1.25Unverified
8Large mLSTMBit per Character (BPC)1.24Unverified
9AWD-LSTM (3 layers)Bit per Character (BPC)1.23Unverified
10Cluster-Former (#C=512)Bit per Character (BPC)1.22Unverified
#ModelMetricClaimedVerifiedStatus
1Smaller Transformer 126M (pre-trained)Test perplexity33Unverified
2OPT 125MTest perplexity32.26Unverified
3Larger Transformer 771M (pre-trained)Test perplexity28.1Unverified
4OPT 1.3BTest perplexity19.55Unverified
5GPT-Neo 125MTest perplexity17.83Unverified
6OPT 2.7BTest perplexity17.81Unverified
7Smaller Transformer 126M (fine-tuned)Test perplexity12Unverified
8GPT-Neo 1.3BTest perplexity11.46Unverified
9Transformer 125MTest perplexity10.7Unverified
10GPT-Neo 2.7BTest perplexity10.44Unverified