SOTAVerified

Language Modelling

A language model is a model of natural language. Language models are useful for a variety of tasks, including speech recognition, machine translation, natural language generation (generating more human-like text), optical character recognition, route optimization, handwriting recognition, grammar induction, and information retrieval.

Large language models (LLMs), currently their most advanced form, are predominantly based on transformers trained on larger datasets (frequently using words scraped from the public internet). They have superseded recurrent neural network-based models, which had previously superseded the purely statistical models, such as word n-gram language model.

Source: Wikipedia

Papers

Showing 1265112700 of 17610 papers

TitleStatusHype
The Acceptability Delta Criterion: Testing Knowledge of Language using the Gradience of Sentence Acceptability0
ProSPer: Probing Human and Neural Network Language Model Understanding of Spatial PerspectiveCode0
How Length Prediction Influence the Performance of Non-Autoregressive Translation?0
What BERT Based Language Model Learns in Spoken Transcripts: An Empirical Study0
Unsupervised Adverbial Identification in Modern Chinese Literature0
Multilingual Sequence Labeling Approach to solve Lexical Normalization0
Intrinsic evaluation of language models for code-switchingCode0
A Pilot Study for BERT Language Modelling and Morphological Analysis for Ancient and Medieval GreekCode1
Can Character-based Language Models Improve Downstream Task Performances In Low-Resource And Noisy Language Scenarios?0
BART for Post-Correction of OCR Newspaper Text0
Effective Use of Graph Convolution Network and Contextual Sub-Tree for Commodity News Event ExtractionCode1
Cryptocurrency Day Trading and Framing Prediction in Microblog Discourse0
Corporate Bankruptcy Prediction with BERT Model0
Learning Cross-lingual Representations for Event Coreference Resolution with Multi-view Alignment and Optimal Transport0
Small Data? No Problem! Exploring the Viability of Pretrained Multilingual Language Models for Low-resourced LanguagesCode1
Paraphrasing Compound Nominalizations0
OpenFraming: Open-sourced Tool for Computational Framing Analysis of Multilingual DataCode0
What’s in Your Head? Emergent Behaviour in Multi-Task Transformer Models0
Unsupervised Multi-View Post-OCR Error Correction With Language Models0
Distilling Relation Embeddings from Pretrained Language Models0
Effective Fine-Tuning Methods for Cross-lingual Adaptation0
BPM_MT: Enhanced Backchannel Prediction Model using Multi-Task Learning0
APIRecX: Cross-Library API Recommendation via Pre-Trained Language Model0
DILBERT: Customized Pre-Training for Domain Adaptation with Category Shift, with an Application to Aspect ExtractionCode1
A Language Model-based Generative Classifier for Sentence-level Discourse Parsing0
IGA: An Intent-Guided Authoring Assistant0
Chinese WPLC: A Chinese Dataset for Evaluating Pretrained Language Models on Word Prediction Given Long-Range Context0
AESOP: Paraphrase Generation with Adaptive Syntactic ControlCode1
AttentionRank: Unsupervised Keyphrase Extraction using Self and Cross AttentionsCode1
The World of an Octopus: How Reporting Bias Influences a Language Model’s Perception of ColorCode0
“It doesn’t look good for a date”: Transforming Critiques into Preferences for Conversational Recommendation SystemsCode0
Segment, Mask, and Predict: Augmenting Chinese Word Segmentation with Self-Supervision0
Less is More: Pretrain a Strong Siamese Encoder for Dense Text Retrieval Using a Weak DecoderCode1
Label-Enhanced Hierarchical Contextualized Representation for Sequential Metaphor Identification0
Multi-task Learning in Argument Mining for Persuasive Online Discussions0
SpellBERT: A Lightweight Pretrained Model for Chinese Spelling Check0
What Can a Generative Language Model Answer About a Passage?0
TSDAE: Using Transformer-based Sequential Denoising Auto-Encoderfor Unsupervised Sentence Embedding LearningCode1
UnClE: Explicitly Leveraging Semantic Similarity to Reduce the Parameters of Word Embeddings0
A Model of Cross-Lingual Knowledge-Grounded Response Generation for Open-Domain Dialogue SystemsCode1
APGN: Adversarial and Parameter Generation Networks for Multi-Source Cross-Domain Dependency Parsing0
Distilling Word Meaning in Context from Pre-trained Language ModelsCode0
Casting the Same Sentiment Classification ProblemCode0
HOTTER: Hierarchical Optimal Topic Transport with Explanatory Context RepresentationsCode0
ArabicTransformer: Efficient Large Arabic Language Model with Funnel Transformer and ELECTRA ObjectiveCode0
Exploring Multitask Learning for Low-Resource Abstractive SummarizationCode0
ODIST: Open World Classification via Distributionally Shifted Instances0
KLMo: Knowledge Graph Enhanced Pretrained Language Model with Fine-Grained RelationshipsCode0
Novel Natural Language Summarization of Program Code via Leveraging Multiple Input Representations0
Language Clustering for Multilingual Named Entity Recognition0
Show:102550
← PrevPage 254 of 353Next →

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1Decay RNNValidation perplexity76.67Unverified
2GRUValidation perplexity53.78Unverified
3LSTMValidation perplexity52.73Unverified
4LSTMTest perplexity48.7Unverified
5Temporal CNNTest perplexity45.2Unverified
6TCNTest perplexity45.19Unverified
7GCNN-8Test perplexity44.9Unverified
8Neural cache model (size = 100)Test perplexity44.8Unverified
9Neural cache model (size = 2,000)Test perplexity40.8Unverified
10GPT-2 SmallTest perplexity37.5Unverified
#ModelMetricClaimedVerifiedStatus
1TCNTest perplexity108.47Unverified
2Seq-U-NetTest perplexity107.95Unverified
3GRU (Bai et al., 2018)Test perplexity92.48Unverified
4R-TransformerTest perplexity84.38Unverified
5Zaremba et al. (2014) - LSTM (medium)Test perplexity82.7Unverified
6Gal & Ghahramani (2016) - Variational LSTM (medium)Test perplexity79.7Unverified
7LSTM (Bai et al., 2018)Test perplexity78.93Unverified
8Zaremba et al. (2014) - LSTM (large)Test perplexity78.4Unverified
9Gal & Ghahramani (2016) - Variational LSTM (large)Test perplexity75.2Unverified
10Inan et al. (2016) - Variational RHNTest perplexity66Unverified
#ModelMetricClaimedVerifiedStatus
1LSTM (7 layers)Bit per Character (BPC)1.67Unverified
2HypernetworksBit per Character (BPC)1.34Unverified
3SHA-LSTM (4 layers, h=1024, no attention head)Bit per Character (BPC)1.33Unverified
4LN HM-LSTMBit per Character (BPC)1.32Unverified
5ByteNetBit per Character (BPC)1.31Unverified
6Recurrent Highway NetworksBit per Character (BPC)1.27Unverified
7Large FS-LSTM-4Bit per Character (BPC)1.25Unverified
8Large mLSTMBit per Character (BPC)1.24Unverified
9AWD-LSTM (3 layers)Bit per Character (BPC)1.23Unverified
10Cluster-Former (#C=512)Bit per Character (BPC)1.22Unverified
#ModelMetricClaimedVerifiedStatus
1Smaller Transformer 126M (pre-trained)Test perplexity33Unverified
2OPT 125MTest perplexity32.26Unverified
3Larger Transformer 771M (pre-trained)Test perplexity28.1Unverified
4OPT 1.3BTest perplexity19.55Unverified
5GPT-Neo 125MTest perplexity17.83Unverified
6OPT 2.7BTest perplexity17.81Unverified
7Smaller Transformer 126M (fine-tuned)Test perplexity12Unverified
8GPT-Neo 1.3BTest perplexity11.46Unverified
9Transformer 125MTest perplexity10.7Unverified
10GPT-Neo 2.7BTest perplexity10.44Unverified