SOTAVerified

Language Modelling

A language model is a model of natural language. Language models are useful for a variety of tasks, including speech recognition, machine translation, natural language generation (generating more human-like text), optical character recognition, route optimization, handwriting recognition, grammar induction, and information retrieval.

Large language models (LLMs), currently their most advanced form, are predominantly based on transformers trained on larger datasets (frequently using words scraped from the public internet). They have superseded recurrent neural network-based models, which had previously superseded the purely statistical models, such as word n-gram language model.

Source: Wikipedia

Papers

Showing 1395114000 of 17610 papers

TitleStatusHype
XHate-999: Analyzing and Detecting Abusive Language Across Domains and Languages0
Unifying Input and Output Smoothing in Neural Machine Translation0
GPolS: A Contextual Graph-Based Language Model for Analyzing Parliamentary Debates and Political Cohesion0
Incremental Neural Lexical Coherence ModelingCode0
Increasing Learning Efficiency of Self-Attention Networks through Direct Position Interactions, Learnable Temperature, and Convoluted AttentionCode0
Intermediate Self-supervised Learning for Machine Translation Quality Estimation0
Homonym normalisation by word sense clustering: a case in Japanese0
Corpus-based Identification of Verbs Participating in Verb Alternations Using Classification and Manual Annotation0
A Co-Attentive Cross-Lingual Neural Model for Dialogue Breakdown DetectionCode0
Few-Shot Text Classification with Edge-Labeling Graph Neural Network-Based Prototypical Network0
Detecting Non-literal Translations by Fine-tuning Cross-lingual Pre-trained Language ModelsCode0
Go Simple and Pre-Train on Domain-Specific Corpora: On the Role of Training Data for Text Classification0
Distill and Replay for Continual Language Learning0
Domain Transfer based Data Augmentation for Neural Query Translation0
Arabizi Language Models for Sentiment Analysis0
Context-Aware Text Normalisation for Historical Dialects0
Enhancing Clinical BERT Embedding using a Biomedical Knowledge BaseCode1
Building Hierarchically Disentangled Language Models for Text Generation with Named Entities0
A Deep Generative Approach to Native Language Identification0
Ad Lingua: Text Classification Improves Symbolism Prediction in Image Advertisements0
Exploring the zero-shot limit of FewRelCode0
Automatic Assistance for Academic Word Usage0
A Neural Local Coherence Analysis Model for Clarity Text Scoring0
SentiX: A Sentiment-Aware Pre-Trained Model for Cross-Domain Sentiment AnalysisCode1
Modeling Local Contexts for Joint Dialogue Act Recognition and Sentiment Classification with Bi-channel Dynamic Convolutions0
Syntax-Aware Graph Attention Network for Aspect-Level Sentiment Classification0
TableGPT: Few-shot Table-to-Text Generation with Table Structure Reconstruction and Content MatchingCode1
Multi-Task Learning for Knowledge Graph Completion with Pre-trained Language ModelsCode1
Multi-Word Lexical SimplificationCode0
Try to Substitute: An Unsupervised Chinese Word Sense Disambiguation Method Based on HowNetCode1
Unsupervised Aspect-Level Sentiment Controllable Style Transfer0
UnihanLM: Coarse-to-Fine Chinese-Japanese Language Model Pretraining with the Unihan DatabaseCode0
DAPPER: Learning Domain-Adapted Persona Representation Using Pretrained BERT and External Memory0
Investigating Learning Dynamics of BERT Fine-Tuning0
SWAGex at SemEval-2020 Task 4: Commonsense Explanation as Next Event Prediction0
Smash at SemEval-2020 Task 7: Optimizing the Hyperparameters of ERNIE 2.0 for Humor Ranking and Rating0
LT3 at SemEval-2020 Task 7: Comparing Feature-Based and Transformer-Based Approaches to Detect Funny Headlines0
Masked Reasoner at SemEval-2020 Task 4: Fine-Tuning RoBERTa for Commonsense Reasoning0
TextLearner at SemEval-2020 Task 10: A Contextualized Ranking System in Solving Emphasis Selection in Text0
MineriaUNAM at SemEval-2020 Task 3: Predicting Contextual WordSimilarity Using a Centroid Based Approach and Word Embeddings0
Mxgra at SemEval-2020 Task 4: Common Sense Making with Next Token Prediction0
JUST at SemEval-2020 Task 11: Detecting Propaganda Techniques Using BERT Pre-trained Model0
Kungfupanda at SemEval-2020 Task 12: BERT-Based Multi-TaskLearning for Offensive Language DetectionCode1
YNU-HPCC at SemEval-2020 Task 10: Using a Multi-granularity Ordinal Classification of the BiLSTM Model for Emphasis Selection0
TR at SemEval-2020 Task 4: Exploring the Limits of Language-model-based Common Sense Validation0
YNUtaoxin at SemEval-2020 Task 11: Identification Fragments of Propaganda Technique by Neural Sequence Labeling Models with Different Tagging Schemes and Pre-trained Language Model0
UI at SemEval-2020 Task 8: Text-Image Fusion for Sentiment Classification0
HIT-SCIR at SemEval-2020 Task 5: Training Pre-trained Language Model with Pseudo-labeling Data for Counterfactuals Detection0
ELMo-NB at SemEval-2020 Task 7: Assessing Sense of Humor in EditedNews Headlines Using ELMo and NB0
ETHAN at SemEval-2020 Task 5: Modelling Causal Reasoning in Language Using Neuro-symbolic Cloud Computing0
Show:102550
← PrevPage 280 of 353Next →

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1Decay RNNValidation perplexity76.67Unverified
2GRUValidation perplexity53.78Unverified
3LSTMValidation perplexity52.73Unverified
4LSTMTest perplexity48.7Unverified
5Temporal CNNTest perplexity45.2Unverified
6TCNTest perplexity45.19Unverified
7GCNN-8Test perplexity44.9Unverified
8Neural cache model (size = 100)Test perplexity44.8Unverified
9Neural cache model (size = 2,000)Test perplexity40.8Unverified
10GPT-2 SmallTest perplexity37.5Unverified
#ModelMetricClaimedVerifiedStatus
1TCNTest perplexity108.47Unverified
2Seq-U-NetTest perplexity107.95Unverified
3GRU (Bai et al., 2018)Test perplexity92.48Unverified
4R-TransformerTest perplexity84.38Unverified
5Zaremba et al. (2014) - LSTM (medium)Test perplexity82.7Unverified
6Gal & Ghahramani (2016) - Variational LSTM (medium)Test perplexity79.7Unverified
7LSTM (Bai et al., 2018)Test perplexity78.93Unverified
8Zaremba et al. (2014) - LSTM (large)Test perplexity78.4Unverified
9Gal & Ghahramani (2016) - Variational LSTM (large)Test perplexity75.2Unverified
10Inan et al. (2016) - Variational RHNTest perplexity66Unverified
#ModelMetricClaimedVerifiedStatus
1LSTM (7 layers)Bit per Character (BPC)1.67Unverified
2HypernetworksBit per Character (BPC)1.34Unverified
3SHA-LSTM (4 layers, h=1024, no attention head)Bit per Character (BPC)1.33Unverified
4LN HM-LSTMBit per Character (BPC)1.32Unverified
5ByteNetBit per Character (BPC)1.31Unverified
6Recurrent Highway NetworksBit per Character (BPC)1.27Unverified
7Large FS-LSTM-4Bit per Character (BPC)1.25Unverified
8Large mLSTMBit per Character (BPC)1.24Unverified
9AWD-LSTM (3 layers)Bit per Character (BPC)1.23Unverified
10Cluster-Former (#C=512)Bit per Character (BPC)1.22Unverified
#ModelMetricClaimedVerifiedStatus
1Smaller Transformer 126M (pre-trained)Test perplexity33Unverified
2OPT 125MTest perplexity32.26Unverified
3Larger Transformer 771M (pre-trained)Test perplexity28.1Unverified
4OPT 1.3BTest perplexity19.55Unverified
5GPT-Neo 125MTest perplexity17.83Unverified
6OPT 2.7BTest perplexity17.81Unverified
7Smaller Transformer 126M (fine-tuned)Test perplexity12Unverified
8GPT-Neo 1.3BTest perplexity11.46Unverified
9Transformer 125MTest perplexity10.7Unverified
10GPT-Neo 2.7BTest perplexity10.44Unverified