Language Modelling

A language model is a model of natural language. Language models are useful for a variety of tasks, including speech recognition, machine translation, natural language generation (generating more human-like text), optical character recognition, route optimization, handwriting recognition, grammar induction, and information retrieval.

Large language models (LLMs), currently their most advanced form, are predominantly based on transformers trained on larger datasets (frequently using words scraped from the public internet). They have superseded recurrent neural network-based models, which had previously superseded the purely statistical models, such as word n-gram language model.

Source: Wikipedia

Papers

Recently Added Most Hyped Most Active Needs Verification Most Verified

Showing 4076–4100 of 17610 papers

Title	Date	Tasks	Status	Score
MILL: Mutual Verification with Large Language Models for Zero-Shot Query Expansion	Oct 29, 2023	Information RetrievalLanguage Modelling	CodeCode Available	5
Minimizing PLM-Based Few-Shot Intent Detectors	Jul 13, 2024	Data AugmentationKnowledge Distillation	CodeCode Available	5
Logit Separability-Driven Samples and Multiple Class-Related Words Selection for Advancing In-Context Learning	Jun 16, 2024	In-Context LearningLanguage Modeling	CodeCode Available	5
A Dutch Financial Large Language Model	Oct 3, 2024	Language ModelingLanguage Modelling	CodeCode Available	5
Planning with Multi-Constraints via Collaborative Language Agents	May 26, 2024	Language ModellingLarge Language Model	CodeCode Available	5
AnchiBERT: A Pre-Trained Model for Ancient ChineseLanguage Understanding and Generation	Sep 24, 2020	Cultural Vocal Bursts Intensity PredictionLanguage Modeling	CodeCode Available	5
Barack's Wife Hillary: Using Knowledge Graphs for Fact-Aware Language Modeling	Jul 1, 2019	Knowledge GraphsLanguage Modeling	CodeCode Available	5
A Dual Encoder Sequence to Sequence Model for Open-Domain Dialogue Modeling	Oct 28, 2017	Language ModelingLanguage Modelling	CodeCode Available	5
Barack's Wife Hillary: Using Knowledge-Graphs for Fact-Aware Language Modeling	Jun 17, 2019	Knowledge GraphsLanguage Modeling	CodeCode Available	5
MAMA: Meta-optimized Angular Margin Contrastive Framework for Video-Language Representation Learning	Jul 4, 2024	Language ModelingLanguage Modelling	CodeCode Available	5
Meta-Learning the Difference: Preparing Large Language Models for Efficient Adaptation	Jul 7, 2022	Abstractive Text SummarizationLanguage Modeling	CodeCode Available	5
MetaSC: Test-Time Safety Specification Optimization for Language Models	Feb 11, 2025	Language ModelingLanguage Modelling	CodeCode Available	5
Debiasing Pre-Trained Language Models via Efficient Fine-Tuning	May 1, 2022	Language ModelingLanguage Modelling	CodeCode Available	5
Meta-Context Transformers for Domain-Specific Response Generation	Oct 12, 2020	Dialogue GenerationLanguage Modeling	CodeCode Available	5
MetaFill: Text Infilling for Meta-Path Generation on Heterogeneous Information Networks	Oct 14, 2022	Graph EmbeddingLanguage Modelling	CodeCode Available	5
An Effective Deployment of Diffusion LM for Data Augmentation in Low-Resource Sentiment Classification	Sep 5, 2024	Data AugmentationDiversity	CodeCode Available	5
Bangla Grammatical Error Detection Using T5 Transformer Model	Mar 19, 2023	Grammatical Error DetectionLanguage Modeling	CodeCode Available	5
Adsorb-Agent: Autonomous Identification of Stable Adsorption Configurations via Large Language Model Agent	Oct 22, 2024	Language ModelingLanguage Modelling	CodeCode Available	5
MetaCoCo: A New Few-Shot Classification Benchmark with Spurious Correlation	Apr 30, 2024	Cross-Domain Few-ShotLanguage Modelling	CodeCode Available	5
Meta Fine-Tuning Neural Language Models for Multi-Domain Text Mining	Mar 29, 2020	Few-Shot LearningLanguage Modeling	CodeCode Available	5
MEND: Meta dEmonstratioN Distillation for Efficient and Effective In-Context Learning	Mar 11, 2024	DecoderIn-Context Learning	CodeCode Available	5
Anatomy of Neural Language Models	Jan 8, 2024	AnatomyLanguage Modeling	CodeCode Available	5
Memory-Efficient Adaptive Optimization	Jan 30, 2019	Language ModelingLanguage Modelling	CodeCode Available	5
Memory-efficient Stochastic methods for Memory-based Transformers	Nov 14, 2023	Language ModelingLanguage Modelling	CodeCode Available	5
Memory Transformer	Jun 20, 2020	Language ModelingLanguage Modelling	CodeCode Available	5

Show:10 25 50

← PrevPage 164 of 705Next →

All datasets WikiText-103 Penn Treebank (Word Level)enwik8 The Pile WikiText-2 LAMBADA One Billion Word Text8 Penn Treebank (Character Level)Hutter Prize OpenWebText SALMon

Benchmark Results

#	Model	Metric	Claimed	Verified	Status
1	Decay RNN	Validation perplexity	76.67	—	Unverified
2	GRU	Validation perplexity	53.78	—	Unverified
3	LSTM	Validation perplexity	52.73	—	Unverified
4	LSTM	Test perplexity	48.7	—	Unverified
5	Temporal CNN	Test perplexity	45.2	—	Unverified
6	TCN	Test perplexity	45.19	—	Unverified
7	GCNN-8	Test perplexity	44.9	—	Unverified
8	Neural cache model (size = 100)	Test perplexity	44.8	—	Unverified
9	Neural cache model (size = 2,000)	Test perplexity	40.8	—	Unverified
10	GPT-2 Small	Test perplexity	37.5	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TCN	Test perplexity	108.47	—	Unverified
2	Seq-U-Net	Test perplexity	107.95	—	Unverified
3	GRU (Bai et al., 2018)	Test perplexity	92.48	—	Unverified
4	R-Transformer	Test perplexity	84.38	—	Unverified
5	Zaremba et al. (2014) - LSTM (medium)	Test perplexity	82.7	—	Unverified
6	Gal & Ghahramani (2016) - Variational LSTM (medium)	Test perplexity	79.7	—	Unverified
7	LSTM (Bai et al., 2018)	Test perplexity	78.93	—	Unverified
8	Zaremba et al. (2014) - LSTM (large)	Test perplexity	78.4	—	Unverified
9	Gal & Ghahramani (2016) - Variational LSTM (large)	Test perplexity	75.2	—	Unverified
10	Inan et al. (2016) - Variational RHN	Test perplexity	66	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	LSTM (7 layers)	Bit per Character (BPC)	1.67	—	Unverified
2	Hypernetworks	Bit per Character (BPC)	1.34	—	Unverified
3	SHA-LSTM (4 layers, h=1024, no attention head)	Bit per Character (BPC)	1.33	—	Unverified
4	LN HM-LSTM	Bit per Character (BPC)	1.32	—	Unverified
5	ByteNet	Bit per Character (BPC)	1.31	—	Unverified
6	Recurrent Highway Networks	Bit per Character (BPC)	1.27	—	Unverified
7	Large FS-LSTM-4	Bit per Character (BPC)	1.25	—	Unverified
8	Large mLSTM	Bit per Character (BPC)	1.24	—	Unverified
9	AWD-LSTM (3 layers)	Bit per Character (BPC)	1.23	—	Unverified
10	Cluster-Former (#C=512)	Bit per Character (BPC)	1.22	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Smaller Transformer 126M (pre-trained)	Test perplexity	33	—	Unverified
2	OPT 125M	Test perplexity	32.26	—	Unverified
3	Larger Transformer 771M (pre-trained)	Test perplexity	28.1	—	Unverified
4	OPT 1.3B	Test perplexity	19.55	—	Unverified
5	GPT-Neo 125M	Test perplexity	17.83	—	Unverified
6	OPT 2.7B	Test perplexity	17.81	—	Unverified
7	Smaller Transformer 126M (fine-tuned)	Test perplexity	12	—	Unverified
8	GPT-Neo 1.3B	Test perplexity	11.46	—	Unverified
9	Transformer 125M	Test perplexity	10.7	—	Unverified
10	GPT-Neo 2.7B	Test perplexity	10.44	—	Unverified