SOTAVerified

Language Modelling

A language model is a model of natural language. Language models are useful for a variety of tasks, including speech recognition, machine translation, natural language generation (generating more human-like text), optical character recognition, route optimization, handwriting recognition, grammar induction, and information retrieval.

Large language models (LLMs), currently their most advanced form, are predominantly based on transformers trained on larger datasets (frequently using words scraped from the public internet). They have superseded recurrent neural network-based models, which had previously superseded the purely statistical models, such as word n-gram language model.

Source: Wikipedia

Papers

Showing 1405114100 of 17610 papers

TitleStatusHype
Building a Swedish Open-Domain Conversational Language ModelCode0
Estimating Subjective Crowd-Evaluations as an Additional Objective to Improve Natural Language Generation0
Comparing the Benefit of Synthetic Training Data for Various Automatic Speech Recognition Architectures0
Non-autoregressive Transformer-based End-to-end ASR using BERT0
Lookup-Table Recurrent Language Models for Long Tail Speech Recognition0
Language model fusion for streaming end to end speech recognition0
Efficient Large-Scale Language Model Training on GPU Clusters Using Megatron-LMCode0
FIBER: Fill-in-the-Blanks as a Challenging Video Understanding Evaluation FrameworkCode0
Extended Parallel Corpus for Amharic-English Machine Translation0
Nutribullets Hybrid: Multi-document Health SummarizationCode0
Pushing the Limits of Non-Autoregressive Speech Recognition0
Interpreting A Pre-trained Model Is A Key For Model Architecture Optimization: A Case Study On Wav2Vec 2.00
Communication-Efficient Agnostic Federated Averaging0
An Empirical Evaluation of Word Embedding Models for Subjectivity Analysis TasksCode0
LT-LM: a novel non-autoregressive language model for single-shot lattice rescoringCode0
Towards Automated Psychotherapy via Language Modeling0
Semantic Distance: A New Metric for ASR Performance Analysis Towards Spoken Language Understanding0
SpeechStew: Simply Mix All Available Speech Recognition Data to Train One Large Neural Network0
COVID-19 sentiment analysis via deep learning during the rise of novel cases0
Contextualized Streaming End-to-End Speech Recognition with Trie-Based Deep Biasing and Shallow Fusion0
IndT5: A Text-to-Text Transformer for 10 Indigenous Languages0
ReCAM@IITK at SemEval-2021 Task 4: BERT and ALBERT based Ensemble for Abstract Word PredictionCode0
TransfoRNN: Capturing the Sequential Information in Self-Attention Representations for Language Modeling0
On-the-Fly Aligned Data Augmentation for Sequence-to-Sequence ASRCode0
UC2: Universal Cross-lingual Cross-modal Vision-and-Language Pre-training0
Multilingual Slavic Named Entity Recognition0
Quranic Verses Semantic Relatedness Using AraBERT0
Maoqin @ DravidianLangTech-EACL2021: The Application of Transformer-Based Model0
Low-Resource Language Modelling of South African LanguagesCode0
Maximal Multiverse Learning for Promoting Cross-Task Generalization of Fine-Tuned Language Models0
On the Computational Modelling of Michif Verbal Morphology0
Story Centaur: Large Language Model Few Shot Learning as a Creative Writing Tool0
Structural Encoding and Pre-training Matter: Adapting BERT for Table-Based Fact Verification0
Keep Learning: Self-supervised Meta-learning for Learning from Inference0
PunKtuator: A Multilingual Punctuation Restoration System for Spoken and Written TextCode0
ONE: Toward ONE model, ONE algorithm, ONE corpus dedicated to sentiment analysis of Arabic/Arabizi and its dialects0
MUCS@LT-EDI-EACL2021:CoHope-Hope Speech Detection for Equality, Diversity, and Inclusion in Code-Mixed Texts0
Pseudo-Label Guided Unsupervised Domain Adaptation of Contextual Embeddings0
TEAM HUB@LT-EDI-EACL2021: Hope Speech Detection Based On Pre-trained Language Model0
What does BERT Learn from Arabic Machine Reading Comprehension Datasets?0
Is Supervised Syntactic Parsing Beneficial for Language Understanding Tasks? An Empirical Investigation0
Efficient Unsupervised NMT for Related Languages with Cross-Lingual Language Models and Fidelity Objectives0
Globalizing BERT-based Transformer Architectures for Long Document Summarization0
CURIE: An Iterative Querying Approach for Reasoning About SituationsCode0
Expressive Text-to-Speech using Style Tag0
Canonical and Surface Morphological Segmentation for Nguni LanguagesCode0
DLRG@DravidianLangTech-EACL2021: Transformer based approachfor Offensive Language Identification on Code-Mixed Tamil0
Introducing A large Tunisian Arabizi Dialectal Dataset for Sentiment Analysis0
Benchmarking Pre-trained Language Models for Multilingual NER: TraSpaS at the BSNLP2021 Shared TaskCode0
Arabic Compact Language Modelling for Resource Limited Devices0
Show:102550
← PrevPage 282 of 353Next →

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1Decay RNNValidation perplexity76.67Unverified
2GRUValidation perplexity53.78Unverified
3LSTMValidation perplexity52.73Unverified
4LSTMTest perplexity48.7Unverified
5Temporal CNNTest perplexity45.2Unverified
6TCNTest perplexity45.19Unverified
7GCNN-8Test perplexity44.9Unverified
8Neural cache model (size = 100)Test perplexity44.8Unverified
9Neural cache model (size = 2,000)Test perplexity40.8Unverified
10GPT-2 SmallTest perplexity37.5Unverified
#ModelMetricClaimedVerifiedStatus
1TCNTest perplexity108.47Unverified
2Seq-U-NetTest perplexity107.95Unverified
3GRU (Bai et al., 2018)Test perplexity92.48Unverified
4R-TransformerTest perplexity84.38Unverified
5Zaremba et al. (2014) - LSTM (medium)Test perplexity82.7Unverified
6Gal & Ghahramani (2016) - Variational LSTM (medium)Test perplexity79.7Unverified
7LSTM (Bai et al., 2018)Test perplexity78.93Unverified
8Zaremba et al. (2014) - LSTM (large)Test perplexity78.4Unverified
9Gal & Ghahramani (2016) - Variational LSTM (large)Test perplexity75.2Unverified
10Inan et al. (2016) - Variational RHNTest perplexity66Unverified
#ModelMetricClaimedVerifiedStatus
1LSTM (7 layers)Bit per Character (BPC)1.67Unverified
2HypernetworksBit per Character (BPC)1.34Unverified
3SHA-LSTM (4 layers, h=1024, no attention head)Bit per Character (BPC)1.33Unverified
4LN HM-LSTMBit per Character (BPC)1.32Unverified
5ByteNetBit per Character (BPC)1.31Unverified
6Recurrent Highway NetworksBit per Character (BPC)1.27Unverified
7Large FS-LSTM-4Bit per Character (BPC)1.25Unverified
8Large mLSTMBit per Character (BPC)1.24Unverified
9AWD-LSTM (3 layers)Bit per Character (BPC)1.23Unverified
10Cluster-Former (#C=512)Bit per Character (BPC)1.22Unverified
#ModelMetricClaimedVerifiedStatus
1Smaller Transformer 126M (pre-trained)Test perplexity33Unverified
2OPT 125MTest perplexity32.26Unverified
3Larger Transformer 771M (pre-trained)Test perplexity28.1Unverified
4OPT 1.3BTest perplexity19.55Unverified
5GPT-Neo 125MTest perplexity17.83Unverified
6OPT 2.7BTest perplexity17.81Unverified
7Smaller Transformer 126M (fine-tuned)Test perplexity12Unverified
8GPT-Neo 1.3BTest perplexity11.46Unverified
9Transformer 125MTest perplexity10.7Unverified
10GPT-Neo 2.7BTest perplexity10.44Unverified