SOTAVerified

Language Modelling

A language model is a model of natural language. Language models are useful for a variety of tasks, including speech recognition, machine translation, natural language generation (generating more human-like text), optical character recognition, route optimization, handwriting recognition, grammar induction, and information retrieval.

Large language models (LLMs), currently their most advanced form, are predominantly based on transformers trained on larger datasets (frequently using words scraped from the public internet). They have superseded recurrent neural network-based models, which had previously superseded the purely statistical models, such as word n-gram language model.

Source: Wikipedia

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 13301–13350 of 17610 papers

TitleStatusHype
Frequency-aware SGD for Efficient Embedding Learning with Provable Benefits—0
From a Lossless (~1.5:1) Compression Algorithm for Llama2 7B Weights to Variable Precision, Variable Range, Compressed Numeric Data Types for CNNs and LLMs—0
From Attention to Atoms: Spectral Dictionary Learning for Fast, Interpretable Language Models—0
From Beginner to Expert: Modeling Medical Knowledge into General LLMs—0
From Caesar Cipher to Unsupervised Learning: A New Method for Classifier Parameter Estimation—0
From Captions to Rewards (CAREVL): Leveraging Large Language Model Experts for Enhanced Reward Modeling in Large Vision-Language Models—0
From Characters to Words: Hierarchical Pre-trained Language Model for Open-vocabulary Language Understanding—0
From Characters to Words to in Between: Do We Capture Morphology?—0
From Complex to Simple: Unraveling the Cognitive Tree for Reasoning with Small Language Models—0
From Concept to Manufacturing: Evaluating Vision-Language Models for Engineering Design—0
From Context to Action: Analysis of the Impact of State Representation and Context on the Generalization of Multi-Turn Web Navigation Agents—0
From Cooking Recipes to Robot Task Trees -- Improving Planning Correctness and Task Efficiency by Leveraging LLMs with a Knowledge Network—0
From Dogwhistles to Bullhorns: Unveiling Coded Rhetoric with Language Models—0
From FreEM to D'AlemBERT: a Large Corpus and a Language Model for Early Modern French—0
From FreEM to D’AlemBERT: a Large Corpus and a Language Model for Early Modern French—0
From General to Specific: Tailoring Large Language Models for Personalized Healthcare—0
From Hallucinations to Facts: Enhancing Language Models with Curated Knowledge Graphs—0
From Human to Machine Psychology: A Conceptual Framework for Understanding Well-Being in Large Language Model—0
From Idea to CAD: A Language Model-Driven Multi-Agent System for Collaborative Design—0
From Information Bottleneck To Activation Norm Penalty—0
From Instructions to Constraints: Language Model Alignment with Automatic Constraint Verification—0
From Keywords to Structured Summaries: Streamlining Scholarly Information Access—0
From Language Models over Tokens to Language Models over Characters—0
From Language to Language-ish: How Brain-Like is an LSTM's Representation of Nonsensical Language Stimuli?—0
From LLM to NMT: Advancing Low-Resource Machine Translation with Claude—0
From Machine Learning to Machine Reasoning—0
From Macro to Micro: Probing Dataset Diversity in Language Model Fine-Tuning—0
From melodic note sequences to pitches using word2vec—0
From Narratives to Numbers: Valid Inference Using Language Model Predictions from Verbal Autopsy Narratives—0
From n-gram to Attention: How Model Architectures Learn and Propagate Bias in Language Modeling—0
From Nodes to Networks: Evolving Recurrent Neural Networks—0
ASETF: A Novel Method for Jailbreak Attack on LLMs through Translate Suffix Embeddings—0
From PARIS to LE-PARIS: Toward Patent Response Automation with Recommender Systems and Collaborative Large Language Models—0
From Pixels to Graphs: using Scene and Knowledge Graphs for HD-EPIC VQA Challenge—0
From Prediction to Application: Language Model-based Code Knowledge Tracing with Domain Adaptive Pre-Training and Automatic Feedback System with Pedagogical Prompting for Comprehensive Programming Education—0
From Prejudice to Parity: A New Approach to Debiasing Large Language Model Word Embeddings—0
From Questions to Insightful Answers: Building an Informed Chatbot for University Resources—0
From RAG to Agentic: Validating Islamic-Medicine Responses with LLM Agents—0
From Retrieval to Generation: Comparing Different Approaches—0
From Retrieval to Generation: Efficient and Effective Entity Set Expansion—0
From Reward Shaping to Q-Shaping: Achieving Unbiased Learning with LLM-Guided Knowledge—0
From r to Q^*: Your Language Model is Secretly a Q-Function—0
From Scattered Sources to Comprehensive Technology Landscape: A Recommendation-based Retrieval Approach—0
From Show to Tell: A Survey on Deep Learning-based Image Captioning—0
From Single Agent to Multi-Agent: Improving Traffic Signal Control—0
From Superficial Patterns to Semantic Understanding: Fine-Tuning Language Models on Contrast Sets—0
From Tarzan to Tolkien: Controlling the Language Proficiency Level of LLMs for Content Generation—0
From task structures to world models: What do LLMs know?—0
From Text to Source: Results in Detecting Large Language Model-Generated Content—0
From tools to thieves: Measuring and understanding public perceptions of AI through crowdsourced metaphors—0
Show:102550
← PrevPage 267 of 353Next →
All datasetsWikiText-103Penn Treebank (Word Level)enwik8The PileWikiText-2LAMBADAOne Billion WordText8Penn Treebank (Character Level)Hutter PrizeOpenWebTextSALMon

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1Decay RNNValidation perplexity76.67—Unverified
2GRUValidation perplexity53.78—Unverified
3LSTMValidation perplexity52.73—Unverified
4LSTMTest perplexity48.7—Unverified
5Temporal CNNTest perplexity45.2—Unverified
6TCNTest perplexity45.19—Unverified
7GCNN-8Test perplexity44.9—Unverified
8Neural cache model (size = 100)Test perplexity44.8—Unverified
9Neural cache model (size = 2,000)Test perplexity40.8—Unverified
10GPT-2 SmallTest perplexity37.5—Unverified
#ModelMetricClaimedVerifiedStatus
1TCNTest perplexity108.47—Unverified
2Seq-U-NetTest perplexity107.95—Unverified
3GRU (Bai et al., 2018)Test perplexity92.48—Unverified
4R-TransformerTest perplexity84.38—Unverified
5Zaremba et al. (2014) - LSTM (medium)Test perplexity82.7—Unverified
6Gal & Ghahramani (2016) - Variational LSTM (medium)Test perplexity79.7—Unverified
7LSTM (Bai et al., 2018)Test perplexity78.93—Unverified
8Zaremba et al. (2014) - LSTM (large)Test perplexity78.4—Unverified
9Gal & Ghahramani (2016) - Variational LSTM (large)Test perplexity75.2—Unverified
10Inan et al. (2016) - Variational RHNTest perplexity66—Unverified
#ModelMetricClaimedVerifiedStatus
1LSTM (7 layers)Bit per Character (BPC)1.67—Unverified
2HypernetworksBit per Character (BPC)1.34—Unverified
3SHA-LSTM (4 layers, h=1024, no attention head)Bit per Character (BPC)1.33—Unverified
4LN HM-LSTMBit per Character (BPC)1.32—Unverified
5ByteNetBit per Character (BPC)1.31—Unverified
6Recurrent Highway NetworksBit per Character (BPC)1.27—Unverified
7Large FS-LSTM-4Bit per Character (BPC)1.25—Unverified
8Large mLSTMBit per Character (BPC)1.24—Unverified
9AWD-LSTM (3 layers)Bit per Character (BPC)1.23—Unverified
10Cluster-Former (#C=512)Bit per Character (BPC)1.22—Unverified
#ModelMetricClaimedVerifiedStatus
1Smaller Transformer 126M (pre-trained)Test perplexity33—Unverified
2OPT 125MTest perplexity32.26—Unverified
3Larger Transformer 771M (pre-trained)Test perplexity28.1—Unverified
4OPT 1.3BTest perplexity19.55—Unverified
5GPT-Neo 125MTest perplexity17.83—Unverified
6OPT 2.7BTest perplexity17.81—Unverified
7Smaller Transformer 126M (fine-tuned)Test perplexity12—Unverified
8GPT-Neo 1.3BTest perplexity11.46—Unverified
9Transformer 125MTest perplexity10.7—Unverified
10GPT-Neo 2.7BTest perplexity10.44—Unverified