SOTAVerified

Language Modelling

A language model is a model of natural language. Language models are useful for a variety of tasks, including speech recognition, machine translation, natural language generation (generating more human-like text), optical character recognition, route optimization, handwriting recognition, grammar induction, and information retrieval.

Large language models (LLMs), currently their most advanced form, are predominantly based on transformers trained on larger datasets (frequently using words scraped from the public internet). They have superseded recurrent neural network-based models, which had previously superseded the purely statistical models, such as word n-gram language model.

Source: Wikipedia

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 12551–12600 of 17610 papers

TitleStatusHype
MarSan at SemEval-2022 Task 11: Multilingual complex named entity recognition using T5 and transformer encoderCode0
Mask and Regenerate: A Classifier-based Approach for Unpaired Sentiment Transformation of Reviews for Electronic Commerce Websites.—0
L3i at SemEval-2022 Task 11: Straightforward Additional Context for Multilingual Named Entity Recognition—0
Learning Natural Language Generation with Truncated Reinforcement LearningCode0
TUG-CIC at SemEval-2021 Task 6: Two-stage Fine-tuning for Intended Sarcasm Detection—0
SwahBERT: Language Model of Swahili—0
JBNU-CCLab at SemEval-2022 Task 7: DeBERTa for Identifying Plausible Clarifications in Instructional Texts—0
Language Model Augmented Monotonic Attention for Simultaneous Translation—0
Self-supervised Product Title Rewrite for Product Listing Ads—0
KroneckerBERT: Significant Compression of Pre-trained Language Models Through Kronecker Decomposition and Knowledge Distillation—0
Modal Dependency Parsing via Language Model PrimingCode0
MT-Speech at SemEval-2022 Task 10: Incorporating Data Augmentation and Auxiliary Task with Cross-Lingual Pretrained Language Model for Structured Sentiment Analysis—0
Minimally-Supervised Relation Induction from Pre-trained Language Model—0
Masking Morphosyntactic Categories to Evaluate Salience for Schizophrenia Diagnosis—0
SPDB Innovation Lab at SemEval-2022 Task 10: A Novel End-to-End Structured Sentiment Analysis Model based on the ERNIE-M—0
Uncertainty and Inclusivity in Gender Bias Annotation: An Annotation Taxonomy and Annotated Datasets of British English Text—0
HuaAMS at SemEval-2022 Task 8: Combining Translation and Domain Pre-training for Cross-lingual News Article Similarity—0
An Annotated Dataset and Automatic Approaches for Discourse Mode Identification in Low-resource Bengali Language—0
An Empirical Study on Pseudo-log-likelihood Bias Measures for Masked Language Models Using Paraphrased Sentences—0
Intent Discovery for Enterprise Virtual Assistants: Applications of Utterance Embedding and Clustering to Intent Mining—0
A Dog Is Passing Over The Jet? A Text-Generation Dataset for Korean Commonsense Reasoning and Evaluation—0
GPT-2-based Human-in-the-loop Theatre Play Script Generation—0
Clinical Flair: A Pre-Trained Language Model for Spanish Clinical Natural Language ProcessingCode0
Improving Conversational Recommendation Systems’ Quality with Context-Aware Item Meta-Information—0
Data Augmentation with Dual Training for Offensive Span Detection—0
Identifying Human Needs through Social Media: A study on Indian cities during COVID-19Code0
Don’t Forget About Pronouns: Removing Gender Bias in Language Models Without Losing Factual Gender Information—0
Empathetic Persuasion: Reinforcing Empathy and Persuasiveness in Dialogue Systems—0
DANGNT-SGU at SemEval-2022 Task 11: Using Pre-trained Language Model for Complex Named Entity Recognition—0
AnaLog: Testing Analytical and Deductive Logic Learnability in Language Models—0
Improving Classification of Infrequent Cognitive Distortions: Domain-Specific Model vs. Data Augmentation—0
Infrrd.ai at SemEval-2022 Task 11: A system for named entity recognition using data augmentation, transformer-based sequence labeling model, and EnsembleCRF—0
”Diversity and Uncertainty in Moderation” are the Key to Data Selection for Multilingual Few-shot Transfer—0
HATE-ITA: New Baselines for Hate Speech Detection in ItalianCode0
Exploring the Effect of Dialect Mismatched Language Models in Telugu Automatic Speech Recognition—0
A Self-supervised Joint Training Framework for Document Reranking—0
Attention Fusion: a light yet efficient late fusion mechanism for task adaptation in NLU—0
Beyond Characters: Subword-level Morpheme Segmentation—0
Exposing the Limits of Video-Text Models through Contrast SetsCode0
esCorpius: A Massive Spanish Crawling Corpus—0
"Diversity and Uncertainty in Moderation" are the Key to Data Selection for Multilingual Few-shot Transfer—0
GSCLIP : A Framework for Explaining Distribution Shifts in Natural Language—0
Language model compression with weighted low-rank factorization—0
Two-Stage Classifier for COVID-19 Misinformation Detection Using BERT: a Study on Indonesian TweetsCode0
Towards a Data-Driven Requirements Engineering Approach: Automatic Analysis of User ReviewsCode0
Simple and Effective Multi-sentence TTS with Expressive and Coherent Prosody—0
Knowledge Distillation of Transformer-based Language Models Revisited—0
Contextual Density Ratio for Language Model Biasing of Sequence to Sequence ASR Systems—0
GPTs at Factify 2022: Prompt Aided Fact-Verification—0
Improving Deliberation by Text-Only and Semi-Supervised Training—0
Show:102550
← PrevPage 252 of 353Next →
All datasetsWikiText-103Penn Treebank (Word Level)enwik8The PileWikiText-2LAMBADAOne Billion WordText8Penn Treebank (Character Level)Hutter PrizeOpenWebTextSALMon

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1Decay RNNValidation perplexity76.67—Unverified
2GRUValidation perplexity53.78—Unverified
3LSTMValidation perplexity52.73—Unverified
4LSTMTest perplexity48.7—Unverified
5Temporal CNNTest perplexity45.2—Unverified
6TCNTest perplexity45.19—Unverified
7GCNN-8Test perplexity44.9—Unverified
8Neural cache model (size = 100)Test perplexity44.8—Unverified
9Neural cache model (size = 2,000)Test perplexity40.8—Unverified
10GPT-2 SmallTest perplexity37.5—Unverified
#ModelMetricClaimedVerifiedStatus
1TCNTest perplexity108.47—Unverified
2Seq-U-NetTest perplexity107.95—Unverified
3GRU (Bai et al., 2018)Test perplexity92.48—Unverified
4R-TransformerTest perplexity84.38—Unverified
5Zaremba et al. (2014) - LSTM (medium)Test perplexity82.7—Unverified
6Gal & Ghahramani (2016) - Variational LSTM (medium)Test perplexity79.7—Unverified
7LSTM (Bai et al., 2018)Test perplexity78.93—Unverified
8Zaremba et al. (2014) - LSTM (large)Test perplexity78.4—Unverified
9Gal & Ghahramani (2016) - Variational LSTM (large)Test perplexity75.2—Unverified
10Inan et al. (2016) - Variational RHNTest perplexity66—Unverified
#ModelMetricClaimedVerifiedStatus
1LSTM (7 layers)Bit per Character (BPC)1.67—Unverified
2HypernetworksBit per Character (BPC)1.34—Unverified
3SHA-LSTM (4 layers, h=1024, no attention head)Bit per Character (BPC)1.33—Unverified
4LN HM-LSTMBit per Character (BPC)1.32—Unverified
5ByteNetBit per Character (BPC)1.31—Unverified
6Recurrent Highway NetworksBit per Character (BPC)1.27—Unverified
7Large FS-LSTM-4Bit per Character (BPC)1.25—Unverified
8Large mLSTMBit per Character (BPC)1.24—Unverified
9AWD-LSTM (3 layers)Bit per Character (BPC)1.23—Unverified
10Cluster-Former (#C=512)Bit per Character (BPC)1.22—Unverified
#ModelMetricClaimedVerifiedStatus
1Smaller Transformer 126M (pre-trained)Test perplexity33—Unverified
2OPT 125MTest perplexity32.26—Unverified
3Larger Transformer 771M (pre-trained)Test perplexity28.1—Unverified
4OPT 1.3BTest perplexity19.55—Unverified
5GPT-Neo 125MTest perplexity17.83—Unverified
6OPT 2.7BTest perplexity17.81—Unverified
7Smaller Transformer 126M (fine-tuned)Test perplexity12—Unverified
8GPT-Neo 1.3BTest perplexity11.46—Unverified
9Transformer 125MTest perplexity10.7—Unverified
10GPT-Neo 2.7BTest perplexity10.44—Unverified