SOTAVerified

Language Modelling

A language model is a model of natural language. Language models are useful for a variety of tasks, including speech recognition, machine translation, natural language generation (generating more human-like text), optical character recognition, route optimization, handwriting recognition, grammar induction, and information retrieval.

Large language models (LLMs), currently their most advanced form, are predominantly based on transformers trained on larger datasets (frequently using words scraped from the public internet). They have superseded recurrent neural network-based models, which had previously superseded the purely statistical models, such as word n-gram language model.

Source: Wikipedia

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 9501–9550 of 17610 papers

TitleStatusHype
LLMs learn governing principles of dynamical systems, revealing an in-context neural scaling lawCode0
Human-mediated Large Language Models for Robotic Intervention in Children with Autism Spectrum Disorders—0
CroissantLLM: A Truly Bilingual French-English Language ModelCode0
Exploring Spatial Schema Intuitions in Large Language and Vision Models—0
From PARIS to LE-PARIS: Toward Patent Response Automation with Recommender Systems and Collaborative Large Language Models—0
Generation, Distillation and Evaluation of Motivational Interviewing-Style Reflections with a Foundational Language Model—0
Health-LLM: Personalized Retrieval-Augmented Disease Prediction System—0
HiQA: A Hierarchical Contextual Augmentation RAG for Multi-Documents QA—0
Institutional Platform for Secure Self-Service Large Language Model Exploration—0
Computational Experiments Meet Large Language Model Based Agents: A Survey and Perspective—0
Assertion Detection Large Language Model In-context Learning LoRA Fine-tuningCode0
Comparing Template-based and Template-free Language Model ProbingCode0
How Useful is Continued Pre-Training for Generative Unsupervised Domain Adaptation?—0
Enhancing Large Language Model with Decomposed Reasoning for Emotion Cause Pair Extraction—0
Employing Label Models on ChatGPT Answers Improves Legal Text Entailment Performance—0
SPECTRUM: Speaker-Enhanced Pre-Training for Long Dialogue Summarization—0
SpeechComposer: Unifying Multiple Speech Tasks with Prompt Composition—0
LoRec: Large Language Model for Robust Sequential Recommendation against Poisoning AttacksCode0
[Lions: 1] and [Tigers: 2] and [Bears: 3], Oh My! Literary Coreference Annotation with LLMs—0
Towards Visual Syntactical Understanding—0
Two Heads Are Better Than One: Integrating Knowledge from Knowledge Graphs and Large Language Models for Entity Alignment—0
Customizing Language Model Responses with Contrastive In-Context Learning—0
Arrows of Time for Large Language ModelsCode0
Improving Reinforcement Learning from Human Feedback with Efficient Reward Model Ensemble—0
Adapting Amidst Degradation: Cross Domain Li-ion Battery Health Estimation via Physics-Guided Test-Time Training—0
H2O-Danube-1.8B Technical Report—0
Gradient-Based Language Model Red Teaming—0
EvoMerge: Neuroevolution for Large Language Models—0
Engineering A Large Language Model From Scratch—0
FakeClaim: A Multiple Platform-driven Dataset for Identification of Fake News on 2023 Israel-Hamas WarCode0
C4Q: A Chatbot for Quantum—0
Development and Testing of a Novel Large Language Model-Based Clinical Decision Support Systems for Medication Safety in 12 Clinical Specialties—0
Assistive Large Language Model Agents for Socially-Aware Negotiation Dialogues—0
Credit Risk Meets Large Language Models: Building a Risk Indicator from Loan Descriptions in P2P Lending—0
InternLM-XComposer2: Mastering Free-form Text-Image Composition and Comprehension in Vision-Language Large Model—0
DressCode: Autoregressively Sewing and Generating Garments from Text Guidance—0
A Linguistic Comparison between Human and ChatGPT-Generated ConversationsCode0
Diverse, but Divisive: LLMs Can Exaggerate Gender Differences in Opinion Related to Harms of Misinformation—0
BPDec: Unveiling the Potential of Masked Language Modeling Decoder in BERT pretraining—0
"You tell me": A Dataset of GPT-4-Based Behaviour Change Support Conversations—0
X-PEFT: eXtremely Parameter-Efficient Fine-Tuning for Extreme Multi-Profile Scenarios—0
LLaMandement: Large Language Models for Summarization of French Legislative Proposals—0
Rephrasing the Web: A Recipe for Compute and Data-Efficient Language Modeling—0
Vocabulary-Defined Semantics: Latent Space Clustering for Improving In-Context Learning—0
LCV2: An Efficient Pretraining-Free Framework for Grounded Visual Question Answering—0
KAUCUS: Knowledge Augmented User Simulators for Training Language Model Assistants—0
Tradeoffs Between Alignment and Helpfulness in Language Models with Representation EngineeringCode0
LLaVA-MoLE: Sparse Mixture of LoRA Experts for Mitigating Data Conflicts in Instruction Finetuning MLLMs—0
Routers in Vision Mixture of Experts: An Empirical Study—0
Textual Entailment for Effective Triple Validation in Object PredictionCode0
Show:102550
← PrevPage 191 of 353Next →
All datasetsWikiText-103Penn Treebank (Word Level)enwik8The PileWikiText-2LAMBADAOne Billion WordText8Penn Treebank (Character Level)Hutter PrizeOpenWebTextSALMon

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1Decay RNNValidation perplexity76.67—Unverified
2GRUValidation perplexity53.78—Unverified
3LSTMValidation perplexity52.73—Unverified
4LSTMTest perplexity48.7—Unverified
5Temporal CNNTest perplexity45.2—Unverified
6TCNTest perplexity45.19—Unverified
7GCNN-8Test perplexity44.9—Unverified
8Neural cache model (size = 100)Test perplexity44.8—Unverified
9Neural cache model (size = 2,000)Test perplexity40.8—Unverified
10GPT-2 SmallTest perplexity37.5—Unverified
#ModelMetricClaimedVerifiedStatus
1TCNTest perplexity108.47—Unverified
2Seq-U-NetTest perplexity107.95—Unverified
3GRU (Bai et al., 2018)Test perplexity92.48—Unverified
4R-TransformerTest perplexity84.38—Unverified
5Zaremba et al. (2014) - LSTM (medium)Test perplexity82.7—Unverified
6Gal & Ghahramani (2016) - Variational LSTM (medium)Test perplexity79.7—Unverified
7LSTM (Bai et al., 2018)Test perplexity78.93—Unverified
8Zaremba et al. (2014) - LSTM (large)Test perplexity78.4—Unverified
9Gal & Ghahramani (2016) - Variational LSTM (large)Test perplexity75.2—Unverified
10Inan et al. (2016) - Variational RHNTest perplexity66—Unverified
#ModelMetricClaimedVerifiedStatus
1LSTM (7 layers)Bit per Character (BPC)1.67—Unverified
2HypernetworksBit per Character (BPC)1.34—Unverified
3SHA-LSTM (4 layers, h=1024, no attention head)Bit per Character (BPC)1.33—Unverified
4LN HM-LSTMBit per Character (BPC)1.32—Unverified
5ByteNetBit per Character (BPC)1.31—Unverified
6Recurrent Highway NetworksBit per Character (BPC)1.27—Unverified
7Large FS-LSTM-4Bit per Character (BPC)1.25—Unverified
8Large mLSTMBit per Character (BPC)1.24—Unverified
9AWD-LSTM (3 layers)Bit per Character (BPC)1.23—Unverified
10Cluster-Former (#C=512)Bit per Character (BPC)1.22—Unverified
#ModelMetricClaimedVerifiedStatus
1Smaller Transformer 126M (pre-trained)Test perplexity33—Unverified
2OPT 125MTest perplexity32.26—Unverified
3Larger Transformer 771M (pre-trained)Test perplexity28.1—Unverified
4OPT 1.3BTest perplexity19.55—Unverified
5GPT-Neo 125MTest perplexity17.83—Unverified
6OPT 2.7BTest perplexity17.81—Unverified
7Smaller Transformer 126M (fine-tuned)Test perplexity12—Unverified
8GPT-Neo 1.3BTest perplexity11.46—Unverified
9Transformer 125MTest perplexity10.7—Unverified
10GPT-Neo 2.7BTest perplexity10.44—Unverified