SOTAVerified

Language Modelling

A language model is a model of natural language. Language models are useful for a variety of tasks, including speech recognition, machine translation, natural language generation (generating more human-like text), optical character recognition, route optimization, handwriting recognition, grammar induction, and information retrieval.

Large language models (LLMs), currently their most advanced form, are predominantly based on transformers trained on larger datasets (frequently using words scraped from the public internet). They have superseded recurrent neural network-based models, which had previously superseded the purely statistical models, such as word n-gram language model.

Source: Wikipedia

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 8701–8750 of 17610 papers

TitleStatusHype
KoACD: The First Korean Adolescent Dataset for Cognitive Distortion Analysis—0
Kongzi: A Historical Large Language Model with Fact Enhancement—0
K-ON: Stacking Knowledge On the Head Layer of Large Language Model—0
Korean Language Modeling via Syntactic Guide—0
Korean Tokenization for Beam Search Rescoring in Speech Recognition—0
Korektor -- A System for Contextual Spell-Checking and Diacritics Completion—0
Kotlin ML Pack: Technical Report—0
KPT: Keyword-guided Pre-training for Grounded Dialog Generation—0
KptLLM++: Towards Generic Keypoint Comprehension with Large Language Model—0
KptLLM: Unveiling the Power of Large Language Model for Keypoint Comprehension—0
Kraken: Inherently Parallel Transformers For Efficient Multi-Device Inference—0
Krikri: Advancing Open Large Language Models for Greek—0
Kriya - The SFU System for Translation Task at WMT-12—0
KronA: Parameter Efficient Tuning with Kronecker Adapter—0
KroneckerBERT: Learning Kronecker Decomposition for Pre-trained Language Models via Knowledge Distillation—0
KroneckerBERT: Significant Compression of Pre-trained Language Models Through Kronecker Decomposition and Knowledge Distillation—0
Kronecker Decomposition for GPT Compression—0
Krony-PT: GPT2 compressed with Kronecker Products—0
KSAnswer: Question-answering System of Kangwon National University and Sogang University in the 2016 BioASQ Challenge—0
KU\_ai at MEDIQA 2019: Domain-specific Pre-training and Transfer Learning for Medical NLI—0
Kuaiji: the First Chinese Accounting Large Language Model—0
KU-DMIS at EHRSQL 2024:Generating SQL query via question templatization in EHR—0
KU-DMIS-MSRA at RadSum23: Pre-trained Vision-Language Model for Radiology Report Summarization—0
KUL: Data-driven Approach to Temporal Parsing of Newswire Articles—0
KU Leuven at HOO-2012: A Hybrid Approach to Detection and Correction of Determiner and Preposition Errors in Non-native English Text—0
KUL@SMM4H’22: Template Augmented Adaptive Pre-training for Tweet Classification—0
KULTURE Bench: A Benchmark for Assessing Language Model in Korean Cultural Context—0
KunServe: Efficient Parameter-centric Memory Management for LLM Serving—0
KurTail : Kurtosis-based LLM Quantization—0
Kuwain 1.5B: An Arabic SLM via Language Injection—0
KV Cache is 1 Bit Per Channel: Efficient Large Language Model Inference with Coupled Quantization—0
KV-Runahead: Scalable Causal LLM Inference by Parallel Key-Value Cache Generation—0
KVShare: An LLM Service System with Efficient and Effective Multi-Tenant KV Cache Reuse—0
K-XLNet: A General Method for Combining Explicit Knowledge with Language Model Pretraining—0
KySS 1.0: a Framework for Automatic Evaluation of Chinese Input Method Engines—0
L2RS: A Learning-to-Rescore Mechanism for Automatic Speech Recognition—0
L3Cube-MahaNLP: Marathi Natural Language Processing Datasets, Models, and Library—0
L3 Ensembles: Lifelong Learning Approach for Ensemble of Foundational Language Models—0
L3i at SemEval-2022 Task 11: Straightforward Additional Context for Multilingual Named Entity Recognition—0
Lab-AI: Using Retrieval Augmentation to Enhance Language Models for Personalized Lab Test Interpretation in Clinical Medicine—0
LAB-Bench: Measuring Capabilities of Language Models for Biology Research—0
Label Alignment and Reassignment with Generalist Large Language Model for Enhanced Cross-Domain Named Entity Recognition—0
Label-Aware Automatic Verbalizer for Few-Shot Text Classification—0
Label-Context-Dependent Internal Language Model Estimation for CTC—0
Label-Enhanced Hierarchical Contextualized Representation for Sequential Metaphor Identification—0
Label-Guided Prompt for Multi-label Few-shot Aspect Category Detection—0
Label prompt for multi-label text classification—0
Label Name is Mantra: Unifying Point Cloud Segmentation across Heterogeneous Datasets—0
LabelPrompt: Effective Prompt-based Learning for Relation Classification—0
Label-Synchronous Neural Transducer for Adaptable Online E2E Speech Recognition—0
Show:102550
← PrevPage 175 of 353Next →
All datasetsWikiText-103Penn Treebank (Word Level)enwik8The PileWikiText-2LAMBADAOne Billion WordText8Penn Treebank (Character Level)Hutter PrizeOpenWebTextSALMon

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1Decay RNNValidation perplexity76.67—Unverified
2GRUValidation perplexity53.78—Unverified
3LSTMValidation perplexity52.73—Unverified
4LSTMTest perplexity48.7—Unverified
5Temporal CNNTest perplexity45.2—Unverified
6TCNTest perplexity45.19—Unverified
7GCNN-8Test perplexity44.9—Unverified
8Neural cache model (size = 100)Test perplexity44.8—Unverified
9Neural cache model (size = 2,000)Test perplexity40.8—Unverified
10GPT-2 SmallTest perplexity37.5—Unverified
#ModelMetricClaimedVerifiedStatus
1TCNTest perplexity108.47—Unverified
2Seq-U-NetTest perplexity107.95—Unverified
3GRU (Bai et al., 2018)Test perplexity92.48—Unverified
4R-TransformerTest perplexity84.38—Unverified
5Zaremba et al. (2014) - LSTM (medium)Test perplexity82.7—Unverified
6Gal & Ghahramani (2016) - Variational LSTM (medium)Test perplexity79.7—Unverified
7LSTM (Bai et al., 2018)Test perplexity78.93—Unverified
8Zaremba et al. (2014) - LSTM (large)Test perplexity78.4—Unverified
9Gal & Ghahramani (2016) - Variational LSTM (large)Test perplexity75.2—Unverified
10Inan et al. (2016) - Variational RHNTest perplexity66—Unverified
#ModelMetricClaimedVerifiedStatus
1LSTM (7 layers)Bit per Character (BPC)1.67—Unverified
2HypernetworksBit per Character (BPC)1.34—Unverified
3SHA-LSTM (4 layers, h=1024, no attention head)Bit per Character (BPC)1.33—Unverified
4LN HM-LSTMBit per Character (BPC)1.32—Unverified
5ByteNetBit per Character (BPC)1.31—Unverified
6Recurrent Highway NetworksBit per Character (BPC)1.27—Unverified
7Large FS-LSTM-4Bit per Character (BPC)1.25—Unverified
8Large mLSTMBit per Character (BPC)1.24—Unverified
9AWD-LSTM (3 layers)Bit per Character (BPC)1.23—Unverified
10Cluster-Former (#C=512)Bit per Character (BPC)1.22—Unverified
#ModelMetricClaimedVerifiedStatus
1Smaller Transformer 126M (pre-trained)Test perplexity33—Unverified
2OPT 125MTest perplexity32.26—Unverified
3Larger Transformer 771M (pre-trained)Test perplexity28.1—Unverified
4OPT 1.3BTest perplexity19.55—Unverified
5GPT-Neo 125MTest perplexity17.83—Unverified
6OPT 2.7BTest perplexity17.81—Unverified
7Smaller Transformer 126M (fine-tuned)Test perplexity12—Unverified
8GPT-Neo 1.3BTest perplexity11.46—Unverified
9Transformer 125MTest perplexity10.7—Unverified
10GPT-Neo 2.7BTest perplexity10.44—Unverified