SOTAVerified

Language Modelling

A language model is a model of natural language. Language models are useful for a variety of tasks, including speech recognition, machine translation, natural language generation (generating more human-like text), optical character recognition, route optimization, handwriting recognition, grammar induction, and information retrieval.

Large language models (LLMs), currently their most advanced form, are predominantly based on transformers trained on larger datasets (frequently using words scraped from the public internet). They have superseded recurrent neural network-based models, which had previously superseded the purely statistical models, such as word n-gram language model.

Source: Wikipedia

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 10251–10300 of 17610 papers

TitleStatusHype
PhyloTransformer: A Discriminative Model for Mutation Prediction Based on a Multi-head Self-attention Mechanism—0
Physically Grounded Vision-Language Models for Robotic Manipulation—0
Physics of Language Models: Part 3.2, Knowledge Manipulation—0
Piano Transcription by Hierarchical Language Modeling with Pretrained Roll-based Encoders—0
PIC a Different Word: A Simple Model for Lexical Substitution in Context—0
Picking Pearl From Seabed: Extracting Artefacts from Noisy Issue Triaging Collaborative Conversations for Hybrid Cloud Services—0
PickLLM: Context-Aware RL-Assisted Large Language Model Routing—0
PIDformer: Transformer Meets Control Theory—0
PIE-QG: Paraphrased Information Extraction for Unsupervised Question Generation from Small Corpora—0
PIGLeT: Language Grounding Through Neuro-Symbolic Interaction in a 3D World—0
Piloting Copilot, Codex, and StarCoder2: Hot Temperature, Cold Prompts, or Black Magic?—0
PIN: A Novel Parallel Interactive Network for Spoken Language Understanding—0
PINGAN Omini-Sinitic at SemEval-2021 Task 4:Reading Comprehension of Abstract Meaning—0
PingAnTech at SMM4H task1: Multiple pre-trained model approaches for Adverse Drug Reactions—0
Pinyin-bert: A new solution to Chinese pinyin to character conversion task—0
PipeSpec: Breaking Stage Dependencies in Hierarchical LLM Decoding—0
Mind the Gap: Assessing Temporal Generalization in Neural Language Models—0
Pivotal Role of Language Modeling in Recommender Systems: Enriching Task-specific and Task-agnostic Representation Learning—0
Pivot Based Language Modeling for Improved Neural Domain Adaptation—0
PIXAR: Auto-Regressive Language Modeling in Pixel Space—0
Pixel-Aligned Language Model—0
Pixel Aligned Language Models—0
Pixels and Predictions: Potential of GPT-4V in Meteorological Imagery Analysis and Forecast Communication—0
PJAIT Systems for the IWSLT 2015 Evaluation Campaign Enhanced by Comparable Corpora—0
PJAIT Systems for the WMT 2016—0
PJIIT's systems for WMT 2017 Conference—0
PLaD: Preference-based Large Language Model Distillation with Pseudo-Preference Pairs—0
PLaMo-100B: A Ground-Up Language Model Designed for Japanese Proficiency—0
Plan ahead: Self-Supervised Text Planning for Paragraph Completion Task—0
Plan and Budget: Effective and Efficient Test-Time Scaling on Large Language Model Reasoning—0
PlanFitting: Personalized Exercise Planning with Large Language Model-driven Conversational Agent—0
PlanGPT: Enhancing Urban Planning with Tailored Language Model and Efficient Retrieval—0
Planning with Diffusion Models for Target-Oriented Dialogue Systems—0
Planning with Large Language Models for Code Generation—0
CAPE: Corrective Actions from Precondition Errors using Large Language Models—0
Planning with Logical Graph-based Language Model for Instruction Generation—0
Planning with Sequence Models through Iterative Energy Minimization—0
Planning with Vision-Language Models and a Use Case in Robot-Assisted Teaching—0
Plan of Thoughts: Heuristic-Guided Problem Solving with Large Language Models—0
Plan-R1: Safe and Feasible Trajectory Planning as Language Modeling—0
Plan-Seq-Learn: Language Model Guided RL for Solving Long Horizon Robotics Tasks—0
PlantBert: An Open Source Language Model for Plant Science—0
Plant in Cupboard, Orange on Rably, Inat Aphone. Benchmarking Incremental Learning of Situation and Language Model using a Text-Simulated Situated Environment—0
Platform-Independent and Curriculum-Oriented Intelligent Assistant for Higher Education—0
player2vec: A Language Modeling Approach to Understand Player Behavior in Games—0
Player-Driven Emergence in LLM-Driven Game Narrative—0
Playing Text-Based Games with Common Sense—0
Playing the Werewolf game with artificial intelligence for language understanding—0
Play the Shannon Game With Language Models: A Human-Free Approach to Summary Evaluation—0
Play to Your Strengths: Collaborative Intelligence of Conventional Recommender Models and Large Language Models—0
Show:102550
← PrevPage 206 of 353Next →
All datasetsWikiText-103Penn Treebank (Word Level)enwik8The PileWikiText-2LAMBADAOne Billion WordText8Penn Treebank (Character Level)Hutter PrizeOpenWebTextSALMon

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1Decay RNNValidation perplexity76.67—Unverified
2GRUValidation perplexity53.78—Unverified
3LSTMValidation perplexity52.73—Unverified
4LSTMTest perplexity48.7—Unverified
5Temporal CNNTest perplexity45.2—Unverified
6TCNTest perplexity45.19—Unverified
7GCNN-8Test perplexity44.9—Unverified
8Neural cache model (size = 100)Test perplexity44.8—Unverified
9Neural cache model (size = 2,000)Test perplexity40.8—Unverified
10GPT-2 SmallTest perplexity37.5—Unverified
#ModelMetricClaimedVerifiedStatus
1TCNTest perplexity108.47—Unverified
2Seq-U-NetTest perplexity107.95—Unverified
3GRU (Bai et al., 2018)Test perplexity92.48—Unverified
4R-TransformerTest perplexity84.38—Unverified
5Zaremba et al. (2014) - LSTM (medium)Test perplexity82.7—Unverified
6Gal & Ghahramani (2016) - Variational LSTM (medium)Test perplexity79.7—Unverified
7LSTM (Bai et al., 2018)Test perplexity78.93—Unverified
8Zaremba et al. (2014) - LSTM (large)Test perplexity78.4—Unverified
9Gal & Ghahramani (2016) - Variational LSTM (large)Test perplexity75.2—Unverified
10Inan et al. (2016) - Variational RHNTest perplexity66—Unverified
#ModelMetricClaimedVerifiedStatus
1LSTM (7 layers)Bit per Character (BPC)1.67—Unverified
2HypernetworksBit per Character (BPC)1.34—Unverified
3SHA-LSTM (4 layers, h=1024, no attention head)Bit per Character (BPC)1.33—Unverified
4LN HM-LSTMBit per Character (BPC)1.32—Unverified
5ByteNetBit per Character (BPC)1.31—Unverified
6Recurrent Highway NetworksBit per Character (BPC)1.27—Unverified
7Large FS-LSTM-4Bit per Character (BPC)1.25—Unverified
8Large mLSTMBit per Character (BPC)1.24—Unverified
9AWD-LSTM (3 layers)Bit per Character (BPC)1.23—Unverified
10Cluster-Former (#C=512)Bit per Character (BPC)1.22—Unverified
#ModelMetricClaimedVerifiedStatus
1Smaller Transformer 126M (pre-trained)Test perplexity33—Unverified
2OPT 125MTest perplexity32.26—Unverified
3Larger Transformer 771M (pre-trained)Test perplexity28.1—Unverified
4OPT 1.3BTest perplexity19.55—Unverified
5GPT-Neo 125MTest perplexity17.83—Unverified
6OPT 2.7BTest perplexity17.81—Unverified
7Smaller Transformer 126M (fine-tuned)Test perplexity12—Unverified
8GPT-Neo 1.3BTest perplexity11.46—Unverified
9Transformer 125MTest perplexity10.7—Unverified
10GPT-Neo 2.7BTest perplexity10.44—Unverified