SOTAVerified

Language Modelling

A language model is a model of natural language. Language models are useful for a variety of tasks, including speech recognition, machine translation, natural language generation (generating more human-like text), optical character recognition, route optimization, handwriting recognition, grammar induction, and information retrieval.

Large language models (LLMs), currently their most advanced form, are predominantly based on transformers trained on larger datasets (frequently using words scraped from the public internet). They have superseded recurrent neural network-based models, which had previously superseded the purely statistical models, such as word n-gram language model.

Source: Wikipedia

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 5251–5300 of 17610 papers

TitleStatusHype
TestNUC: Enhancing Test-Time Computing Approaches through Neighboring Unlabeled Data ConsistencyCode0
Nexus: An Omni-Perceptive And -Interactive Model for Language, Audio, And Vision—0
On the Importance of Text Preprocessing for Multimodal Representation Learning and Pathology Report Generation—0
Kanana: Compute-efficient Bilingual Language Models—0
Pathology Report Generation and Multimodal Representation Learning for Cutaneous Melanocytic Lesions—0
MindMem: Multimodal for Predicting Advertisement Memorability Using LLMs and Deep Learning—0
TextGames: Learning to Self-Play Text-Based Puzzle Games via Language Model ReasoningCode0
Large Language Model Driven Agents for Simulating Echo Chamber Formation—0
PyEvalAI: AI-assisted evaluation of Jupyter Notebooks for immediate personalized feedback—0
LDGen: Enhancing Text-to-Image Synthesis via Large Language Model-Driven Language Representation—0
A Combinatorial Identities Benchmark for Theorem Proving via Automated Theorem Generation—0
Iterative Counterfactual Data AugmentationCode0
AfroXLMR-Comet: Multilingual Knowledge Distillation with Attention Matching for Low-Resource languages—0
Beyond In-Distribution Success: Scaling Curves of CoT Granularity for Language Model GeneralizationCode0
Faster, Cheaper, Better: Multi-Objective Hyperparameter Optimization for LLM and RAG Systems—0
from Benign import Toxic: Jailbreaking the Language Model via Adversarial Metaphors—0
Broadening Discovery through Structural Models: Multimodal Combination of Local and Structural Properties for Predicting Chemical Features—0
Enhancing DNA Foundation Models to Address Masking Inefficiencies—0
Independent Mobility GPT (IDM-GPT): A Self-Supervised Multi-Agent Large Language Model Framework for Customized Traffic Mobility Analysis Using Machine Learning Models—0
AMPO: Active Multi-Preference Optimization—0
Can LLMs Explain Themselves Counterfactually?—0
Your Language Model May Think Too Rigidly: Achieving Reasoning Consistency with Symmetry-Enhanced Training—0
VALUE: Value-Aware Large Language Model for Query Rewriting via Weighted Trie in Sponsored Search—0
Zero-shot Load Forecasting for Integrated Energy Systems: A Large Language Model-based Framework with Multi-task Learning—0
Language Model Re-rankers are Steered by Lexical Similarities—0
Time series forecasting based on optimized LLM for fault prediction in distribution power grid insulators—0
Reasoning with Latent Thoughts: On the Power of Looped Transformers—0
Unbiased and Sign Compression in Distributed Learning: Comparing Noise Resilience via SDEs—0
Thus Spake Long-Context Large Language Model—0
Predicting Liquidity-Aware Bond Yields using Causal GANs and Deep Reinforcement Learning with LLM Evaluation—0
Real-time Monitoring of Economic Shocks using Company Websites—0
SPARC: Score Prompting and Adaptive Fusion for Zero-Shot Multi-Label Recognition in Vision-Language Models—0
Knowledge Distillation with Training Wheels—0
Understanding the Uncertainty of LLM Explanations: A Perspective Based on Reasoning Topology—0
Sarang at DEFACTIFY 4.0: Detecting AI-Generated Text Using Noised Data and an Ensemble of DeBERTa Models—0
DIS-CO: Discovering Copyrighted Content in VLMs Training DataCode0
IGDA: Interactive Graph Discovery through Large Language Model Agents—0
Intention Recognition in Real-Time Interactive Navigation MapsCode0
Forecasting Rare Language Model Behaviors—0
Improving Interactive Diagnostic Ability of a Large Language Model Agent Through Clinical Experience Learning—0
How Do Large Language Monkeys Get Their Power (Laws)?—0
Balancing Speech Understanding and Generation Using Continual Pre-training for Codec-based Speech LLM—0
From Perceptions to Decisions: Wildfire Evacuation Decision Prediction with Behavioral Theory-informed LLMsCode0
An Enhanced Large Language Model For Cross Modal Query Understanding System Using DL-KeyBERT Based CAZSSCL-MPGPT—0
Retrieval-Augmented Visual Question Answering via Built-in Autoregressive Search Engines—0
Sequence-level Large Language Model Training with Contrastive Preference Optimization—0
MV-CLAM: Multi-View Molecular Interpretation with Cross-Modal Projection via Language ModelCode0
LitLinker: Supporting the Ideation of Interdisciplinary Contexts with Large Language Models for Teaching Literature in Elementary SchoolsCode0
Large Language Model for Lossless Image Compression with Visual Prompts—0
Recurrent Knowledge Identification and Fusion for Language Model Continual Learning—0
Show:102550
← PrevPage 106 of 353Next →
All datasetsWikiText-103Penn Treebank (Word Level)enwik8The PileWikiText-2LAMBADAOne Billion WordText8Penn Treebank (Character Level)Hutter PrizeOpenWebTextSALMon

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1Decay RNNValidation perplexity76.67—Unverified
2GRUValidation perplexity53.78—Unverified
3LSTMValidation perplexity52.73—Unverified
4LSTMTest perplexity48.7—Unverified
5Temporal CNNTest perplexity45.2—Unverified
6TCNTest perplexity45.19—Unverified
7GCNN-8Test perplexity44.9—Unverified
8Neural cache model (size = 100)Test perplexity44.8—Unverified
9Neural cache model (size = 2,000)Test perplexity40.8—Unverified
10GPT-2 SmallTest perplexity37.5—Unverified
#ModelMetricClaimedVerifiedStatus
1TCNTest perplexity108.47—Unverified
2Seq-U-NetTest perplexity107.95—Unverified
3GRU (Bai et al., 2018)Test perplexity92.48—Unverified
4R-TransformerTest perplexity84.38—Unverified
5Zaremba et al. (2014) - LSTM (medium)Test perplexity82.7—Unverified
6Gal & Ghahramani (2016) - Variational LSTM (medium)Test perplexity79.7—Unverified
7LSTM (Bai et al., 2018)Test perplexity78.93—Unverified
8Zaremba et al. (2014) - LSTM (large)Test perplexity78.4—Unverified
9Gal & Ghahramani (2016) - Variational LSTM (large)Test perplexity75.2—Unverified
10Inan et al. (2016) - Variational RHNTest perplexity66—Unverified
#ModelMetricClaimedVerifiedStatus
1LSTM (7 layers)Bit per Character (BPC)1.67—Unverified
2HypernetworksBit per Character (BPC)1.34—Unverified
3SHA-LSTM (4 layers, h=1024, no attention head)Bit per Character (BPC)1.33—Unverified
4LN HM-LSTMBit per Character (BPC)1.32—Unverified
5ByteNetBit per Character (BPC)1.31—Unverified
6Recurrent Highway NetworksBit per Character (BPC)1.27—Unverified
7Large FS-LSTM-4Bit per Character (BPC)1.25—Unverified
8Large mLSTMBit per Character (BPC)1.24—Unverified
9AWD-LSTM (3 layers)Bit per Character (BPC)1.23—Unverified
10Cluster-Former (#C=512)Bit per Character (BPC)1.22—Unverified
#ModelMetricClaimedVerifiedStatus
1Smaller Transformer 126M (pre-trained)Test perplexity33—Unverified
2OPT 125MTest perplexity32.26—Unverified
3Larger Transformer 771M (pre-trained)Test perplexity28.1—Unverified
4OPT 1.3BTest perplexity19.55—Unverified
5GPT-Neo 125MTest perplexity17.83—Unverified
6OPT 2.7BTest perplexity17.81—Unverified
7Smaller Transformer 126M (fine-tuned)Test perplexity12—Unverified
8GPT-Neo 1.3BTest perplexity11.46—Unverified
9Transformer 125MTest perplexity10.7—Unverified
10GPT-Neo 2.7BTest perplexity10.44—Unverified