SOTAVerified

Language Modelling

A language model is a model of natural language. Language models are useful for a variety of tasks, including speech recognition, machine translation, natural language generation (generating more human-like text), optical character recognition, route optimization, handwriting recognition, grammar induction, and information retrieval.

Large language models (LLMs), currently their most advanced form, are predominantly based on transformers trained on larger datasets (frequently using words scraped from the public internet). They have superseded recurrent neural network-based models, which had previously superseded the purely statistical models, such as word n-gram language model.

Source: Wikipedia

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 5751–5800 of 17610 papers

TitleStatusHype
Attacks on Third-Party APIs of Large Language ModelsCode0
Hi-Gen: Generative Retrieval For Large-Scale Personalized E-commerce Search—0
Knowledge Graph Completion using Structural and Textual EmbeddingsCode0
Generalization Measures for Zero-Shot Cross-Lingual Transfer—0
CORM: Cache Optimization with Recent Message for Large Language Model Inference—0
Studying Large Language Model Behaviors Under Context-Memory Conflicts With Real DocumentsCode1
Nyonic Technical ReportCode1
Can Foundational Large Language Models Assist with Conducting Pharmaceuticals Manufacturing Investigations?—0
A Comprehensive Survey on Evaluating Large Language Model Applications in the Medical Industry—0
Fusion of Domain-Adapted Vision and Language Models for Medical Visual Question Answering—0
Towards Efficient Patient Recruitment for Clinical Trials: Application of a Prompt-Based Learning Model—0
Beyond ESM2: Graph-Enhanced Protein Sequence Modeling with Efficient Clustering—0
Detecting Conceptual Abstraction in LLMs—0
Breaking Walls: Pioneering Automatic Speech Recognition for Central Kurdish: End-to-End Transformer Paradigm—0
From Complexity to Clarity: How AI Enhances Perceptions of Scientists and the Public's Understanding of Science—0
Detection of circular permutations by Protein Language ModelsCode0
BattleAgent: Multi-modal Dynamic Emulation on Historical Battles to Complement Historical AnalysisCode1
CoST: Contrastive Quantization based Semantic Tokenization for Generative Recommendation—0
Setting up the Data Printer with Improved English to Ukrainian Machine TranslationCode1
Student Data Paradox and Curious Case of Single Student-Tutor Model: Regressive Side Effects of Training LLMs for Personalized Learning—0
Visual Delta Generator with Large Multi-modal Models for Semi-supervised Composed Image Retrieval—0
XC-Cache: Cross-Attending to Cached Context for Efficient LLM Inference—0
CultureBank: An Online Community-Driven Knowledge Base Towards Culturally Aware Language TechnologiesCode1
ClinicalAgent: Clinical Trial Multi-Agent System with Large Language Model-based Reasoning—0
Retrieval Augmented Generation for Domain-specific Question Answering—0
RealTCD: Temporal Causal Discovery from Interventional Data with Large Language Model—0
Aligning LLM Agents by Learning Latent Preference from User EditsCode1
Multi-Head Mixture-of-ExpertsCode1
Multimodal Large Language Model is a Human-Aligned Annotator for Text-to-Image Generation—0
Pegasus-v1 Technical Report—0
Re-Thinking Inverse Graphics With Large Language Models—0
Pushing the Boundaries of Text to Motion with Arbitrary Text: A New Task—0
Q-Tuning: Queue-based Prompt Tuning for Lifelong Few-shot Language Learning—0
Mélange: Cost Efficient Large Language Model Serving by Exploiting GPU HeterogeneityCode1
WangLab at MEDIQA-CORR 2024: Optimized LLM-based Programs for Medical Error Detection and Correction—0
Understanding the role of FFNs in driving multilingual behaviour in LLMs—0
OpenELM: An Efficient Language Model Family with Open Training and Inference FrameworkCode9
Performance Characterization of Expert Router for Scalable LLM Inference—0
Self-Supervised Alignment with Mutual Information: Learning to Follow Principles without Preference LabelsCode1
Pixels and Predictions: Potential of GPT-4V in Meteorological Imagery Analysis and Forecast Communication—0
An empirical study of LLaMA3 quantization: from LLMs to MLLMsCode2
SpaceByte: Towards Deleting Tokenization from Large Language ModelingCode2
A Multimodal Automated Interpretability Agent—0
From LLM to NMT: Advancing Low-Resource Machine Translation with Claude—0
CoFInAl: Enhancing Action Quality Assessment with Coarse-to-Fine Instruction AlignmentCode1
VALOR-EVAL: Holistic Coverage and Faithfulness Evaluation of Large Vision-Language ModelsCode1
Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone—0
Self-Bootstrapped Visual-Language Model for Knowledge Selection and Question AnsweringCode0
PARAMANU-GANITA: Language Model with Mathematical Capabilities—0
Socratic Planner: Self-QA-Based Zero-Shot Planning for Embodied Instruction Following—0
Show:102550
← PrevPage 116 of 353Next →
All datasetsWikiText-103Penn Treebank (Word Level)enwik8The PileWikiText-2LAMBADAOne Billion WordText8Penn Treebank (Character Level)Hutter PrizeOpenWebTextSALMon

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1Decay RNNValidation perplexity76.67—Unverified
2GRUValidation perplexity53.78—Unverified
3LSTMValidation perplexity52.73—Unverified
4LSTMTest perplexity48.7—Unverified
5Temporal CNNTest perplexity45.2—Unverified
6TCNTest perplexity45.19—Unverified
7GCNN-8Test perplexity44.9—Unverified
8Neural cache model (size = 100)Test perplexity44.8—Unverified
9Neural cache model (size = 2,000)Test perplexity40.8—Unverified
10GPT-2 SmallTest perplexity37.5—Unverified
#ModelMetricClaimedVerifiedStatus
1TCNTest perplexity108.47—Unverified
2Seq-U-NetTest perplexity107.95—Unverified
3GRU (Bai et al., 2018)Test perplexity92.48—Unverified
4R-TransformerTest perplexity84.38—Unverified
5Zaremba et al. (2014) - LSTM (medium)Test perplexity82.7—Unverified
6Gal & Ghahramani (2016) - Variational LSTM (medium)Test perplexity79.7—Unverified
7LSTM (Bai et al., 2018)Test perplexity78.93—Unverified
8Zaremba et al. (2014) - LSTM (large)Test perplexity78.4—Unverified
9Gal & Ghahramani (2016) - Variational LSTM (large)Test perplexity75.2—Unverified
10Inan et al. (2016) - Variational RHNTest perplexity66—Unverified
#ModelMetricClaimedVerifiedStatus
1LSTM (7 layers)Bit per Character (BPC)1.67—Unverified
2HypernetworksBit per Character (BPC)1.34—Unverified
3SHA-LSTM (4 layers, h=1024, no attention head)Bit per Character (BPC)1.33—Unverified
4LN HM-LSTMBit per Character (BPC)1.32—Unverified
5ByteNetBit per Character (BPC)1.31—Unverified
6Recurrent Highway NetworksBit per Character (BPC)1.27—Unverified
7Large FS-LSTM-4Bit per Character (BPC)1.25—Unverified
8Large mLSTMBit per Character (BPC)1.24—Unverified
9AWD-LSTM (3 layers)Bit per Character (BPC)1.23—Unverified
10Cluster-Former (#C=512)Bit per Character (BPC)1.22—Unverified
#ModelMetricClaimedVerifiedStatus
1Smaller Transformer 126M (pre-trained)Test perplexity33—Unverified
2OPT 125MTest perplexity32.26—Unverified
3Larger Transformer 771M (pre-trained)Test perplexity28.1—Unverified
4OPT 1.3BTest perplexity19.55—Unverified
5GPT-Neo 125MTest perplexity17.83—Unverified
6OPT 2.7BTest perplexity17.81—Unverified
7Smaller Transformer 126M (fine-tuned)Test perplexity12—Unverified
8GPT-Neo 1.3BTest perplexity11.46—Unverified
9Transformer 125MTest perplexity10.7—Unverified
10GPT-Neo 2.7BTest perplexity10.44—Unverified