SOTAVerified

Language Modelling

A language model is a model of natural language. Language models are useful for a variety of tasks, including speech recognition, machine translation, natural language generation (generating more human-like text), optical character recognition, route optimization, handwriting recognition, grammar induction, and information retrieval.

Large language models (LLMs), currently their most advanced form, are predominantly based on transformers trained on larger datasets (frequently using words scraped from the public internet). They have superseded recurrent neural network-based models, which had previously superseded the purely statistical models, such as word n-gram language model.

Source: Wikipedia

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 5601–5650 of 17610 papers

TitleStatusHype
Who Wrote This? The Key to Zero-Shot LLM-Generated Text Detection Is GECScore—0
DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language ModelCode9
KV Cache is 1 Bit Per Channel: Efficient Large Language Model Inference with Coupled Quantization—0
SWE-agent: Agent-Computer Interfaces Enable Automated Software EngineeringCode11
QuakeBERT: Accurate Classification of Social Media Texts for Rapid Earthquake Impact Assessment—0
TED: Accelerate Model Training by Internal Generalization—0
AntiFold: Improved antibody structure-based design using inverse foldingCode2
VSA4VQA: Scaling a Vector Symbolic Architecture to Visual Question Answering on Natural Images—0
ID-centric Pre-training for Recommendation—0
Lory: Fully Differentiable Mixture-of-Experts for Autoregressive Language Model Pre-training—0
Adapting Dual-encoder Vision-language Models for Paraphrased Retrieval—0
Liberating Seen Classes: Boosting Few-Shot and Zero-Shot Text Classification via Anchor Generation and Classification Reframing—0
Federated Reinforcement Learning with Constraint Heterogeneity—0
Enhancing Q-Learning with Large Language Model Heuristics—0
Oracle-Checker Scheme for Evaluating a Generative Large Language Model—0
Snake Learning: A Communication- and Computation-Efficient Distributed Learning Framework for 6G—0
Explainable Fake News Detection With Large Language Model via Defense Among Competing WisdomCode2
Visual grounding for desktop graphical user interfaces—0
Leveraging Lecture Content for Improved Feedback: Explorations with GPT-4 and Retrieval Augmented Generation—0
ClothPPO: A Proximal Policy Optimization Enhancing Framework for Robotic Cloth Manipulation with Observation-Aligned Action Spaces—0
High Order Reasoning for Time Critical Recommendation in Evidence-based Medicine—0
Get more for less: Principled Data Selection for Warming Up Fine-Tuning in LLMs—0
Exploring prompts to elicit memorization in masked language model-based named entity recognition—0
A self-supervised text-vision framework for automated brain abnormality detection—0
Labeling supervised fine-tuning data with the scaling lawCode7
Mozart's Touch: A Lightweight Multi-modal Music Generation Framework Based on Pre-Trained Large ModelsCode1
Source-Free Domain Adaptation Guided by Vision and Vision-Language Pre-TrainingCode0
IQLS: Framework for leveraging Metadata to enable Large Language Model based queries to complex, versatile Data—0
EDA Corpus: A Large Language Model Dataset for Enhanced Interaction with OpenROADCode1
PhilHumans: Benchmarking Machine Learning for Personal Health—0
Recall Them All: Retrieval-Augmented Language Models for Long Object List Extraction from Long Documents—0
Exploring Extreme Quantization in Spiking Language Models—0
Relations Prediction for Knowledge Graph Completion using Large Language Models—0
Sub-goal Distillation: A Method to Improve Small Language AgentsCode0
Identifying Narrative Patterns and Outliers in Holocaust Testimonies Using Topic Modeling—0
CRCL at SemEval-2024 Task 2: Simple prompt optimizationsCode0
A Normative Framework for Benchmarking Consumer Fairness in Large Language Model Recommender System—0
A Survey of Time Series Foundation Models: Generalizing Time Series Representation with Large Language ModelCode2
LLM as Dataset Analyst: Subpopulation Structure Discovery with Large Language ModelCode0
CodeGRAG: Bridging the Gap between Natural Language and Programming Language via Graphical Retrieval Augmented Generation—0
CALRec: Contrastive Alignment of Generative LLMs for Sequential Recommendation—0
Protein binding affinity prediction under multiple substitutions applying eGNNs on Residue and Atomic graphs combined with Language model information: eGRAL—0
DALLMi: Domain Adaption for LLM-based Multi-label ClassifierCode0
Exploiting ChatGPT for Diagnosing Autism-Associated Language Disorders and Identifying Distinct FeaturesCode0
Efficient Heterogeneous Large Language Model Decoding with Model-Attention Disaggregation—0
Automating the Analysis of Public Saliency and Attitudes towards Biodiversity from Digital Media—0
Generative Relevance Feedback and Convergence of Adaptive Re-Ranking: University of Glasgow Terrier Team at TREC DL 2023Code0
CoS: Enhancing Personalization and Mitigating Bias with Context SteeringCode1
Prometheus 2: An Open Source Language Model Specialized in Evaluating Other Language ModelsCode5
Context-Aware Clustering using Large Language Models—0
Show:102550
← PrevPage 113 of 353Next →
All datasetsWikiText-103Penn Treebank (Word Level)enwik8The PileWikiText-2LAMBADAOne Billion WordText8Penn Treebank (Character Level)Hutter PrizeOpenWebTextSALMon

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1Decay RNNValidation perplexity76.67—Unverified
2GRUValidation perplexity53.78—Unverified
3LSTMValidation perplexity52.73—Unverified
4LSTMTest perplexity48.7—Unverified
5Temporal CNNTest perplexity45.2—Unverified
6TCNTest perplexity45.19—Unverified
7GCNN-8Test perplexity44.9—Unverified
8Neural cache model (size = 100)Test perplexity44.8—Unverified
9Neural cache model (size = 2,000)Test perplexity40.8—Unverified
10GPT-2 SmallTest perplexity37.5—Unverified
#ModelMetricClaimedVerifiedStatus
1TCNTest perplexity108.47—Unverified
2Seq-U-NetTest perplexity107.95—Unverified
3GRU (Bai et al., 2018)Test perplexity92.48—Unverified
4R-TransformerTest perplexity84.38—Unverified
5Zaremba et al. (2014) - LSTM (medium)Test perplexity82.7—Unverified
6Gal & Ghahramani (2016) - Variational LSTM (medium)Test perplexity79.7—Unverified
7LSTM (Bai et al., 2018)Test perplexity78.93—Unverified
8Zaremba et al. (2014) - LSTM (large)Test perplexity78.4—Unverified
9Gal & Ghahramani (2016) - Variational LSTM (large)Test perplexity75.2—Unverified
10Inan et al. (2016) - Variational RHNTest perplexity66—Unverified
#ModelMetricClaimedVerifiedStatus
1LSTM (7 layers)Bit per Character (BPC)1.67—Unverified
2HypernetworksBit per Character (BPC)1.34—Unverified
3SHA-LSTM (4 layers, h=1024, no attention head)Bit per Character (BPC)1.33—Unverified
4LN HM-LSTMBit per Character (BPC)1.32—Unverified
5ByteNetBit per Character (BPC)1.31—Unverified
6Recurrent Highway NetworksBit per Character (BPC)1.27—Unverified
7Large FS-LSTM-4Bit per Character (BPC)1.25—Unverified
8Large mLSTMBit per Character (BPC)1.24—Unverified
9AWD-LSTM (3 layers)Bit per Character (BPC)1.23—Unverified
10Cluster-Former (#C=512)Bit per Character (BPC)1.22—Unverified
#ModelMetricClaimedVerifiedStatus
1Smaller Transformer 126M (pre-trained)Test perplexity33—Unverified
2OPT 125MTest perplexity32.26—Unverified
3Larger Transformer 771M (pre-trained)Test perplexity28.1—Unverified
4OPT 1.3BTest perplexity19.55—Unverified
5GPT-Neo 125MTest perplexity17.83—Unverified
6OPT 2.7BTest perplexity17.81—Unverified
7Smaller Transformer 126M (fine-tuned)Test perplexity12—Unverified
8GPT-Neo 1.3BTest perplexity11.46—Unverified
9Transformer 125MTest perplexity10.7—Unverified
10GPT-Neo 2.7BTest perplexity10.44—Unverified