SOTAVerified

Language Modelling

A language model is a model of natural language. Language models are useful for a variety of tasks, including speech recognition, machine translation, natural language generation (generating more human-like text), optical character recognition, route optimization, handwriting recognition, grammar induction, and information retrieval.

Large language models (LLMs), currently their most advanced form, are predominantly based on transformers trained on larger datasets (frequently using words scraped from the public internet). They have superseded recurrent neural network-based models, which had previously superseded the purely statistical models, such as word n-gram language model.

Source: Wikipedia

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 8601–8650 of 17610 papers

TitleStatusHype
KAER: A Knowledge Augmented Pre-Trained Language Model for Entity Resolution—0
KOALA: A Kalman Optimization Algorithm with Loss Adaptivity—0
KALA: Knowledge-Augmented Language Model Adaptation—0
Kaleido Diffusion: Improving Conditional Diffusion Models with Autoregressive Latent Modeling—0
KaLM: Knowledge-aligned Autoregressive Language Modeling via Dual-view Knowledge Graph Contrastive Learning—0
Kanana: Compute-efficient Bilingual Language Models—0
Kangaroo: A Powerful Video-Language Model Supporting Long-context Video Input—0
KaraSinger: Score-Free Singing Voice Synthesis with VQ-VAE using Mel-spectrograms—0
Balanced End-to-End Monolingual pre-training for Low-Resourced Indic Languages Code-Switching Speech Recognition—0
KARL: Knowledge-Aware Reasoning Memory Modeling with Reinforcement Learning of Vector Space—0
KAUCUS: Knowledge Augmented User Simulators for Training Language Model Assistants—0
Kawenn\'on:nis: the Wordmaker for Kanyen'k\'eha—0
KDLSQ-BERT: A Quantized Bert Combining Knowledge Distillation with Learned Step Size Quantization—0
KECP: Knowledge Enhanced Contrastive Prompting for Few-shot Extractive Question Answering—0
K-Edit: Language Model Editing with Contextual Knowledge Awareness—0
KEDRec-LM: A Knowledge-distilled Explainable Drug Recommendation Large Language Model—0
Keep Decoding Parallel with Effective Knowledge Distillation from Language Models to End-to-end Speech Recognisers—0
Keeping in Time: Adding Temporal Context to Sentiment Analysis Models—0
Keep It or Not: Word Level Quality Estimation for Post-Editing—0
Keep Learning: Self-supervised Meta-learning for Learning from Inference—0
KemenkeuGPT: Leveraging a Large Language Model on Indonesia's Government Financial Data and Regulations to Enhance Decision Making—0
KEPLET: Knowledge-Enhanced Pretrained Language Model with Topic Entity Awareness—0
Key Concept Identification for Medical Information Retrieval—0
Key ingredients for effective zero-shot cross-lingual knowledge transfer in generative tasks—0
Key Phrase Classification in Complex Assignments—0
Keyphrase Prediction With Pre-trained Language Model—0
Key-Point-Driven Mathematical Reasoning Distillation of Large Language Model—0
Keyword Augmented Retrieval: Novel framework for Information Retrieval integrated with speech interface—0
KEYword based Sampling (KEYS) for Large Language Models—0
Keyword Extraction from Short Texts with a Text-To-Text Transfer Transformer—0
KgPLM: Knowledge-guided Language Model Pre-training via Generative and Discriminative Learning—0
KG-RAG: Bridging the Gap Between Knowledge and Creativity—0
Khattat: Enhancing Readability and Concept Representation of Semantic Typography—0
KINLP at SemEval-2023 Task 12: Kinyarwanda Tweet Sentiment Analysis—0
KinyaBERT: a Morphology-aware Kinyarwanda Language Model—0
KIQA: Knowledge-Infused Question Answering Model for Financial Table-Text Data—0
KisMATH: Do LLMs Have Knowledge of Implicit Structures in Mathematical Reasoning?—0
Kite: Automatic speech recognition for unmanned aerial vehicles—0
Kitten: a tool for normalizing HTML and extracting its textual content—0
kk2018 at SemEval-2020 Task 9: Adversarial Training for Code-Mixing Sentiment Classification—0
K-Level Reasoning: Establishing Higher Order Beliefs in Large Language Models for Strategic Reasoning—0
KMMLU: Measuring Massive Multitask Language Understanding in Korean—0
k-Neighbor Based Curriculum Sampling for Sequence Prediction—0
Knesset-DictaBERT: A Hebrew Language Model for Parliamentary Proceedings—0
KNIFE: Distilling Reasoning Knowledge From Free-Text Rationales—0
kNN-Adapter: Efficient Domain Adaptation for Black-Box Language Models—0
KnowDA: All-in-One Knowledge Mixture Model for Data Augmentation in Low-Resource NLP—0
KnowFormer: Revisiting Transformers for Knowledge Graph Reasoning—0
Knowing Where to Focus: Attention-Guided Alignment for Text-based Person Search—0
Knowledgeable Agents by Offline Reinforcement Learning from Large Language Model Rollouts—0
Show:102550
← PrevPage 173 of 353Next →
All datasetsWikiText-103Penn Treebank (Word Level)enwik8The PileWikiText-2LAMBADAOne Billion WordText8Penn Treebank (Character Level)Hutter PrizeOpenWebTextSALMon

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1Decay RNNValidation perplexity76.67—Unverified
2GRUValidation perplexity53.78—Unverified
3LSTMValidation perplexity52.73—Unverified
4LSTMTest perplexity48.7—Unverified
5Temporal CNNTest perplexity45.2—Unverified
6TCNTest perplexity45.19—Unverified
7GCNN-8Test perplexity44.9—Unverified
8Neural cache model (size = 100)Test perplexity44.8—Unverified
9Neural cache model (size = 2,000)Test perplexity40.8—Unverified
10GPT-2 SmallTest perplexity37.5—Unverified
#ModelMetricClaimedVerifiedStatus
1TCNTest perplexity108.47—Unverified
2Seq-U-NetTest perplexity107.95—Unverified
3GRU (Bai et al., 2018)Test perplexity92.48—Unverified
4R-TransformerTest perplexity84.38—Unverified
5Zaremba et al. (2014) - LSTM (medium)Test perplexity82.7—Unverified
6Gal & Ghahramani (2016) - Variational LSTM (medium)Test perplexity79.7—Unverified
7LSTM (Bai et al., 2018)Test perplexity78.93—Unverified
8Zaremba et al. (2014) - LSTM (large)Test perplexity78.4—Unverified
9Gal & Ghahramani (2016) - Variational LSTM (large)Test perplexity75.2—Unverified
10Inan et al. (2016) - Variational RHNTest perplexity66—Unverified
#ModelMetricClaimedVerifiedStatus
1LSTM (7 layers)Bit per Character (BPC)1.67—Unverified
2HypernetworksBit per Character (BPC)1.34—Unverified
3SHA-LSTM (4 layers, h=1024, no attention head)Bit per Character (BPC)1.33—Unverified
4LN HM-LSTMBit per Character (BPC)1.32—Unverified
5ByteNetBit per Character (BPC)1.31—Unverified
6Recurrent Highway NetworksBit per Character (BPC)1.27—Unverified
7Large FS-LSTM-4Bit per Character (BPC)1.25—Unverified
8Large mLSTMBit per Character (BPC)1.24—Unverified
9AWD-LSTM (3 layers)Bit per Character (BPC)1.23—Unverified
10Cluster-Former (#C=512)Bit per Character (BPC)1.22—Unverified
#ModelMetricClaimedVerifiedStatus
1Smaller Transformer 126M (pre-trained)Test perplexity33—Unverified
2OPT 125MTest perplexity32.26—Unverified
3Larger Transformer 771M (pre-trained)Test perplexity28.1—Unverified
4OPT 1.3BTest perplexity19.55—Unverified
5GPT-Neo 125MTest perplexity17.83—Unverified
6OPT 2.7BTest perplexity17.81—Unverified
7Smaller Transformer 126M (fine-tuned)Test perplexity12—Unverified
8GPT-Neo 1.3BTest perplexity11.46—Unverified
9Transformer 125MTest perplexity10.7—Unverified
10GPT-Neo 2.7BTest perplexity10.44—Unverified