SOTAVerified

Language Modelling

A language model is a model of natural language. Language models are useful for a variety of tasks, including speech recognition, machine translation, natural language generation (generating more human-like text), optical character recognition, route optimization, handwriting recognition, grammar induction, and information retrieval.

Large language models (LLMs), currently their most advanced form, are predominantly based on transformers trained on larger datasets (frequently using words scraped from the public internet). They have superseded recurrent neural network-based models, which had previously superseded the purely statistical models, such as word n-gram language model.

Source: Wikipedia

Papers

Showing 1230112350 of 17610 papers

TitleStatusHype
KALM: Knowledge-Aware Integration of Local, Document, and Global Contexts for Long Document UnderstandingCode0
AlphaTuning: Quantization-Aware Parameter-Efficient Adaptation of Large-Scale Pre-Trained Language Models0
Large Language Models can Implement Policy Iteration0
Few-Shot Anaphora Resolution in Scientific Protocols via Mixtures of In-Context ExpertsCode0
UU-Tax at SemEval-2022 Task 3: Improving the generalizability of language models for taxonomy classification through data augmentationCode0
Using Interventions to Improve Out-of-Distribution Generalization of Text-Matching Recommendation Systems0
Novice Type Error Diagnosis with Natural Language Models0
PQLM -- Multilingual Decentralized Portable Quantum Language Model for Privacy Protection0
Prompt Compression and Contrastive Conditioning for Controllability and Toxicity Reduction in Language Models0
Just ClozE! A Novel Framework for Evaluating the Factual Consistency Faster in Abstractive SummarizationCode0
Improving the Sample Efficiency of Prompt Tuning with Domain AdaptationCode0
Conversational Semantic Role Labeling with Predicate-Oriented Latent Graph0
Improving Large-scale Paraphrase Acquisition and Generation0
Vision Transformer Based Model for Describing a Set of Images as a Story0
Revisiting Syllables in Language Modelling and their Application on Low-Resource Machine Translation0
Antibody Representation Learning for Drug Discovery0
Honest Students from Untrusted Teachers: Learning an Interpretable Question-Answering Pipeline from a Pretrained Language Model0
A Non-monotonic Self-terminating Language ModelCode0
Enriching Vulnerability Reports Through Automated and Augmented Description Summarization0
The Effectiveness of Masked Language Modeling and Adapters for Factual Knowledge InjectionCode0
The boundaries of meaning: a case study in neural machine translation0
MALM: Mixing Augmented Language Modeling for Zero-Shot Machine Translation0
Neural-Guided Program Synthesis of Information Extraction Rules Using Self-Supervision0
NSP-BERT: A Prompt-based Few-Shot Learner through an Original Pre-training Task —— Next Sentence Prediction0
Knowledge Distillation with Reptile Meta-Learning for Pretrained Language Model CompressionCode0
PingAnTech at SMM4H task1: Multiple pre-trained model approaches for Adverse Drug Reactions0
The COVID That Wasn’t: Counterfactual Journalism Using GPT0
Making Parameter-efficient Tuning More Efficient: A Unified Framework for Classification TasksCode0
The Role of Context in Detecting the Target of Hate Speech0
Section-Aware Commonsense Knowledge-Grounded Dialogue Generation with Pre-trained Language ModelCode0
Learnable Dependency-based Double Graph Structure for Aspect-based Sentiment Analysis0
Transferring Knowledge from Structure-aware Self-attention Language Model to Sequence-to-Sequence Semantic Parsing0
Predictive Text for Agglutinative and Polysynthetic Languages0
Towards Making the Most of Pre-trained Translation Model for Quality Estimation0
PLN CMM at SocialDisNER: Improving Detection of Disease Mentions in Tweets by Using Document-Level Features0
Speaker Clustering in Textual Dialogue with Pairwise Utterance Relation and Cross-corpus Dialogue Act Supervision0
mattica@SMM4H’22: Leveraging sentiment for stance & premise joint learning0
KUL@SMM4H’22: Template Augmented Adaptive Pre-training for Tweet Classification0
Transfer Learning Improves French Cross-Domain Dialect Identification: NRC @ VarDial 20220
Team AINLPML @ MuP in SDP 2021: Scientific Document Summarization by End-to-End Extractive and Abstractive Approach0
The Only Chance to Understand: Machine Translation of the Severely Endangered Low-resource Languages of Eurasia0
Taking Actions Separately: A Bidirectionally-Adaptive Transfer Learning Method for Low-Resource Neural Machine Translation0
Using Language Models to Improve Rule-based Linguistic Annotation of Modern Historical Japanese CorporaCode0
Using Structured Content Plans for Fine-grained Syntactic Control in Pretrained Language Model Generation0
UPER: Boosting Multi-Document Summarization with an Unsupervised Prompt-based ExtractorCode0
Unsupervised Data Augmentation for Aspect Based Sentiment Analysis0
Can Data Diversity Enhance Learning Generalization?0
Deciphering and Characterizing Out-of-Vocabulary Words for Morphologically Rich Languages0
A Japanese Masked Language Model for Academic DomainCode0
How about Time? Probing a Multilingual Language Model for Temporal RelationsCode0
Show:102550
← PrevPage 247 of 353Next →

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1Decay RNNValidation perplexity76.67Unverified
2GRUValidation perplexity53.78Unverified
3LSTMValidation perplexity52.73Unverified
4LSTMTest perplexity48.7Unverified
5Temporal CNNTest perplexity45.2Unverified
6TCNTest perplexity45.19Unverified
7GCNN-8Test perplexity44.9Unverified
8Neural cache model (size = 100)Test perplexity44.8Unverified
9Neural cache model (size = 2,000)Test perplexity40.8Unverified
10GPT-2 SmallTest perplexity37.5Unverified
#ModelMetricClaimedVerifiedStatus
1TCNTest perplexity108.47Unverified
2Seq-U-NetTest perplexity107.95Unverified
3GRU (Bai et al., 2018)Test perplexity92.48Unverified
4R-TransformerTest perplexity84.38Unverified
5Zaremba et al. (2014) - LSTM (medium)Test perplexity82.7Unverified
6Gal & Ghahramani (2016) - Variational LSTM (medium)Test perplexity79.7Unverified
7LSTM (Bai et al., 2018)Test perplexity78.93Unverified
8Zaremba et al. (2014) - LSTM (large)Test perplexity78.4Unverified
9Gal & Ghahramani (2016) - Variational LSTM (large)Test perplexity75.2Unverified
10Inan et al. (2016) - Variational RHNTest perplexity66Unverified
#ModelMetricClaimedVerifiedStatus
1LSTM (7 layers)Bit per Character (BPC)1.67Unverified
2HypernetworksBit per Character (BPC)1.34Unverified
3SHA-LSTM (4 layers, h=1024, no attention head)Bit per Character (BPC)1.33Unverified
4LN HM-LSTMBit per Character (BPC)1.32Unverified
5ByteNetBit per Character (BPC)1.31Unverified
6Recurrent Highway NetworksBit per Character (BPC)1.27Unverified
7Large FS-LSTM-4Bit per Character (BPC)1.25Unverified
8Large mLSTMBit per Character (BPC)1.24Unverified
9AWD-LSTM (3 layers)Bit per Character (BPC)1.23Unverified
10Cluster-Former (#C=512)Bit per Character (BPC)1.22Unverified
#ModelMetricClaimedVerifiedStatus
1Smaller Transformer 126M (pre-trained)Test perplexity33Unverified
2OPT 125MTest perplexity32.26Unverified
3Larger Transformer 771M (pre-trained)Test perplexity28.1Unverified
4OPT 1.3BTest perplexity19.55Unverified
5GPT-Neo 125MTest perplexity17.83Unverified
6OPT 2.7BTest perplexity17.81Unverified
7Smaller Transformer 126M (fine-tuned)Test perplexity12Unverified
8GPT-Neo 1.3BTest perplexity11.46Unverified
9Transformer 125MTest perplexity10.7Unverified
10GPT-Neo 2.7BTest perplexity10.44Unverified