SOTAVerified

Language Modelling

A language model is a model of natural language. Language models are useful for a variety of tasks, including speech recognition, machine translation, natural language generation (generating more human-like text), optical character recognition, route optimization, handwriting recognition, grammar induction, and information retrieval.

Large language models (LLMs), currently their most advanced form, are predominantly based on transformers trained on larger datasets (frequently using words scraped from the public internet). They have superseded recurrent neural network-based models, which had previously superseded the purely statistical models, such as word n-gram language model.

Source: Wikipedia

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 12501–12550 of 17610 papers

TitleStatusHype
A Hazard Analysis Framework for Code Synthesis Large Language Models—0
A Transformer-based Neural Language Model that Synthesizes Brain Activation Maps from Free-Form Text Queries—0
Chunk-aware Alignment and Lexical Constraint for Visual Entailment with Natural Language ExplanationsCode0
PanGu-Coder: Program Synthesis with Function-Level Language ModelingCode0
Language models of protein sequences at the scale of evolution enable accurate structure prediction—0
The Birth of Bias: A case study on the evolution of gender bias in an English language modelCode0
Integrating Linguistic Theory and Neural Language ModelsCode0
Word Play for Playing Othello (Reverses)—0
Towards the Human Global Context: Does the Vision-Language Model Really Judge Like a Human Being?—0
STT: Soft Template Tuning for Few-Shot Adaptation—0
Training Large-Vocabulary Neural Language Models by Private Federated Learning for Resource-Constrained Devices—0
Natural language processing for clusterization of genes according to their functions—0
ELECTRA is a Zero-Shot Learner, TooCode0
An Overview of Distant Supervision for Relation Extraction with a Focus on Denoising and Pre-training Methods—0
Multimodal Dialog Systems with Dual Knowledge-enhanced Generative Pretrained Language Model—0
Learning Flexible Translation between Robot Actions and Language Descriptions—0
A No-Code Low-Code Paradigm for Authoring Business Automations Using Natural Language—0
Beware the Rationalization Trap! When Language Model Explainability Diverges from our Mental Models of Language—0
BERTIN: Efficient Pre-Training of a Spanish Language Model using Perplexity Sampling—0
Combing for Credentials: Active Pattern Extraction from Smart Reply—0
Confident Adaptive Language Modeling—0
TRIE++: Towards End-to-End Information Extraction from Visually Rich Documents—0
Layout-Aware Information Extraction for Document-Grounded Dialogue: Dataset, Method and Demonstration—0
Language models show human-like content effects on reasoning tasksCode0
Neural Data-to-Text Generation Based on Small Datasets: Comparing the Added Value of Two Semi-Supervised Learning Approaches on Top of a Large Language Model—0
Multilinguals at SemEval-2022 Task 11: Complex NER in Semantically Ambiguous Settings for Low Resource LanguagesCode0
Text-driven Emotional Style Control and Cross-speaker Style Transfer in Neural TTS—0
A Transfer Learning Based Model for Text Readability Assessment in German—0
A Novel DeBERTa-based Model for Financial Question Answering Task—0
Internal Language Model Estimation based Language Model Fusion for Cross-Domain Code-Switching Speech Recognition—0
Hidden Schema Networks—0
Meta-Learning the Difference: Preparing Large Language Models for Efficient AdaptationCode0
Neural Language Models are not Born Equal to Fit Brain Data, but Training Helps—0
Text Enriched Sparse Hyperbolic Graph Convolutional Networks—0
SimLM: Pre-training with Representation Bottleneck for Dense Passage Retrieval—0
Aspect-Based Sentiment Analysis using Local Context Focus Mechanism with DeBERTa—0
Gender Biases and Where to Find Them: Exploring Gender Bias in Pre-Trained Transformer-based Language Models Using Movement PruningCode0
ASR-Generated Text for Language Model Pre-training Applied to Speech Tasks—0
Cross-Lingual QA as a Stepping Stone for Monolingual Open QA in Icelandic—0
MIA 2022 Shared Task Submission: Leveraging Entity Representations, Dense-Sparse Hybrids, and Fusion-in-Decoder for Cross-Lingual Question Answering—0
Robustness Analysis of Video-Language Models Against Visual and Language PerturbationsCode0
BERT, can HE predict contrastive focus? Predicting and controlling prominence in neural TTS using a language model—0
Generating Repetitions with Appropriate Repeated WordsCode0
FRAME: Evaluating Rationale-Label Consistency Metrics for Free-Text Rationales—0
UserLibri: A Dataset for ASR Personalization Using Only Text—0
You Don’t Know My Favorite Color: Preventing Dialogue Representations from Revealing Speakers’ Private PersonasCode0
Unsupervised Paraphrasability Prediction for Compound Nominalizations—0
ValCAT: Variable-Length Contextualized Adversarial Transformations Using Encoder-Decoder Language ModelCode0
Zuo Zhuan Ancient Chinese Dataset for Word Sense Disambiguation—0
niksss at SemEval-2022 Task 6: Are Traditionally Pre-Trained Contextual Embeddings Enough for Detecting Intended Sarcasm ?—0
Show:102550
← PrevPage 251 of 353Next →
All datasetsWikiText-103Penn Treebank (Word Level)enwik8The PileWikiText-2LAMBADAOne Billion WordText8Penn Treebank (Character Level)Hutter PrizeOpenWebTextSALMon

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1Decay RNNValidation perplexity76.67—Unverified
2GRUValidation perplexity53.78—Unverified
3LSTMValidation perplexity52.73—Unverified
4LSTMTest perplexity48.7—Unverified
5Temporal CNNTest perplexity45.2—Unverified
6TCNTest perplexity45.19—Unverified
7GCNN-8Test perplexity44.9—Unverified
8Neural cache model (size = 100)Test perplexity44.8—Unverified
9Neural cache model (size = 2,000)Test perplexity40.8—Unverified
10GPT-2 SmallTest perplexity37.5—Unverified
#ModelMetricClaimedVerifiedStatus
1TCNTest perplexity108.47—Unverified
2Seq-U-NetTest perplexity107.95—Unverified
3GRU (Bai et al., 2018)Test perplexity92.48—Unverified
4R-TransformerTest perplexity84.38—Unverified
5Zaremba et al. (2014) - LSTM (medium)Test perplexity82.7—Unverified
6Gal & Ghahramani (2016) - Variational LSTM (medium)Test perplexity79.7—Unverified
7LSTM (Bai et al., 2018)Test perplexity78.93—Unverified
8Zaremba et al. (2014) - LSTM (large)Test perplexity78.4—Unverified
9Gal & Ghahramani (2016) - Variational LSTM (large)Test perplexity75.2—Unverified
10Inan et al. (2016) - Variational RHNTest perplexity66—Unverified
#ModelMetricClaimedVerifiedStatus
1LSTM (7 layers)Bit per Character (BPC)1.67—Unverified
2HypernetworksBit per Character (BPC)1.34—Unverified
3SHA-LSTM (4 layers, h=1024, no attention head)Bit per Character (BPC)1.33—Unverified
4LN HM-LSTMBit per Character (BPC)1.32—Unverified
5ByteNetBit per Character (BPC)1.31—Unverified
6Recurrent Highway NetworksBit per Character (BPC)1.27—Unverified
7Large FS-LSTM-4Bit per Character (BPC)1.25—Unverified
8Large mLSTMBit per Character (BPC)1.24—Unverified
9AWD-LSTM (3 layers)Bit per Character (BPC)1.23—Unverified
10Cluster-Former (#C=512)Bit per Character (BPC)1.22—Unverified
#ModelMetricClaimedVerifiedStatus
1Smaller Transformer 126M (pre-trained)Test perplexity33—Unverified
2OPT 125MTest perplexity32.26—Unverified
3Larger Transformer 771M (pre-trained)Test perplexity28.1—Unverified
4OPT 1.3BTest perplexity19.55—Unverified
5GPT-Neo 125MTest perplexity17.83—Unverified
6OPT 2.7BTest perplexity17.81—Unverified
7Smaller Transformer 126M (fine-tuned)Test perplexity12—Unverified
8GPT-Neo 1.3BTest perplexity11.46—Unverified
9Transformer 125MTest perplexity10.7—Unverified
10GPT-Neo 2.7BTest perplexity10.44—Unverified