Language Modelling

A language model is a model of natural language. Language models are useful for a variety of tasks, including speech recognition, machine translation, natural language generation (generating more human-like text), optical character recognition, route optimization, handwriting recognition, grammar induction, and information retrieval.

Large language models (LLMs), currently their most advanced form, are predominantly based on transformers trained on larger datasets (frequently using words scraped from the public internet). They have superseded recurrent neural network-based models, which had previously superseded the purely statistical models, such as word n-gram language model.

Source: Wikipedia

Papers

Recently Added Most Hyped Most Active Needs Verification Most Verified

Showing 9251–9275 of 17610 papers

Title	Date	Tasks	Status	Hype
Fast Quantum Algorithm for Attention Computation	Jul 16, 2023	Language ModelingLanguage Modelling	—Unverified	0
Planting a SEED of Vision in Large Language Model	Jul 16, 2023	Image GenerationImage to text	CodeCode Available	2
Language Conditioned Traffic Generation	Jul 16, 2023	DecoderLanguage Modeling	CodeCode Available	1
The Potential and Pitfalls of using a Large Language Model such as ChatGPT or GPT-4 as a Clinical Assistant	Jul 16, 2023	DiagnosticLanguage Modeling	—Unverified	0
Disco-Bench: A Discourse-Aware Evaluation Benchmark for Language Modelling	Jul 16, 2023	DiagnosticLanguage Modelling	CodeCode Available	2
Intuitive Access to Smartphone Settings Using Relevance Model Trained by Contrastive Learning	Jul 15, 2023	Contrastive LearningKnowledge Distillation	—Unverified	0
Think-on-Graph: Deep and Responsible Reasoning of Large Language Model on Knowledge Graph	Jul 15, 2023	HallucinationKnowledge Graphs	CodeCode Available	2
Transformers are Universal Predictors	Jul 15, 2023	Language ModelingLanguage Modelling	—Unverified	0
Coupling Large Language Models with Logic Programming for Robust and General Reasoning from Text	Jul 15, 2023	Language ModelingLanguage Modelling	CodeCode Available	1
Drive Like a Human: Rethinking Autonomous Driving with Large Language Models	Jul 14, 2023	Autonomous DrivingCommon Sense Reasoning	CodeCode Available	2
HYTREL: Hypergraph-enhanced Tabular Data Representation Learning	Jul 14, 2023	Language ModelingLanguage Modelling	CodeCode Available	1
Gloss Attention for Gloss-free Sign Language Translation	Jul 14, 2023	Gloss-free Sign Language TranslationLanguage Modeling	CodeCode Available	1
Improving BERT with Hybrid Pooling Network and Drop Mask	Jul 14, 2023	Language ModelingLanguage Modelling	—Unverified	0
Mega-TTS 2: Boosting Prompting Mechanisms for Zero-Shot Speech Synthesis	Jul 14, 2023	In-Context LearningLanguage Modelling	—Unverified	0
Population Expansion for Training Language Models with Private Federated Learning	Jul 14, 2023	Domain AdaptationFederated Learning	—Unverified	0
MorphPiece : A Linguistic Tokenizer for Large Language Models	Jul 14, 2023	Language ModelingLanguage Modelling	—Unverified	0
Making the Most Out of the Limited Context Length: Predictive Power Varies with Clinical Note Type and Note Section	Jul 13, 2023	Language ModelingLanguage Modelling	—Unverified	0
Does Collaborative Human-LM Dialogue Generation Help Information Extraction from Human Dialogues?	Jul 13, 2023	Dialogue GenerationDialogue State Tracking	—Unverified	0
Generating Benchmarks for Factuality Evaluation of Language Models	Jul 13, 2023	Language ModelingLanguage Modelling	CodeCode Available	2
Copy Is All You Need	Jul 13, 2023	AllDomain Adaptation	CodeCode Available	1
In-context Autoencoder for Context Compression in a Large Language Model	Jul 13, 2023	GPULanguage Modeling	CodeCode Available	1
Electoral Agitation Data Set: The Use Case of the Polish Election	Jul 13, 2023	Language ModelingLanguage Modelling	CodeCode Available	0
Instruction Mining: Instruction Data Selection for Tuning Large Language Models	Jul 12, 2023	Instruction FollowingLanguage Modeling	—Unverified	0
VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Models	Jul 12, 2023	FormLanguage Modelling	CodeCode Available	2
Transformers in Reinforcement Learning: A Survey	Jul 12, 2023	Cloud ComputingCombinatorial Optimization	—Unverified	0

Show:10 25 50

← PrevPage 371 of 705Next →

All datasets WikiText-103 Penn Treebank (Word Level)enwik8 The Pile WikiText-2 LAMBADA One Billion Word Text8 Penn Treebank (Character Level)Hutter Prize OpenWebText SALMon

Benchmark Results

#	Model	Metric	Claimed	Verified	Status
1	Decay RNN	Validation perplexity	76.67	—	Unverified
2	GRU	Validation perplexity	53.78	—	Unverified
3	LSTM	Validation perplexity	52.73	—	Unverified
4	LSTM	Test perplexity	48.7	—	Unverified
5	Temporal CNN	Test perplexity	45.2	—	Unverified
6	TCN	Test perplexity	45.19	—	Unverified
7	GCNN-8	Test perplexity	44.9	—	Unverified
8	Neural cache model (size = 100)	Test perplexity	44.8	—	Unverified
9	Neural cache model (size = 2,000)	Test perplexity	40.8	—	Unverified
10	GPT-2 Small	Test perplexity	37.5	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TCN	Test perplexity	108.47	—	Unverified
2	Seq-U-Net	Test perplexity	107.95	—	Unverified
3	GRU (Bai et al., 2018)	Test perplexity	92.48	—	Unverified
4	R-Transformer	Test perplexity	84.38	—	Unverified
5	Zaremba et al. (2014) - LSTM (medium)	Test perplexity	82.7	—	Unverified
6	Gal & Ghahramani (2016) - Variational LSTM (medium)	Test perplexity	79.7	—	Unverified
7	LSTM (Bai et al., 2018)	Test perplexity	78.93	—	Unverified
8	Zaremba et al. (2014) - LSTM (large)	Test perplexity	78.4	—	Unverified
9	Gal & Ghahramani (2016) - Variational LSTM (large)	Test perplexity	75.2	—	Unverified
10	Inan et al. (2016) - Variational RHN	Test perplexity	66	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	LSTM (7 layers)	Bit per Character (BPC)	1.67	—	Unverified
2	Hypernetworks	Bit per Character (BPC)	1.34	—	Unverified
3	SHA-LSTM (4 layers, h=1024, no attention head)	Bit per Character (BPC)	1.33	—	Unverified
4	LN HM-LSTM	Bit per Character (BPC)	1.32	—	Unverified
5	ByteNet	Bit per Character (BPC)	1.31	—	Unverified
6	Recurrent Highway Networks	Bit per Character (BPC)	1.27	—	Unverified
7	Large FS-LSTM-4	Bit per Character (BPC)	1.25	—	Unverified
8	Large mLSTM	Bit per Character (BPC)	1.24	—	Unverified
9	AWD-LSTM (3 layers)	Bit per Character (BPC)	1.23	—	Unverified
10	Cluster-Former (#C=512)	Bit per Character (BPC)	1.22	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Smaller Transformer 126M (pre-trained)	Test perplexity	33	—	Unverified
2	OPT 125M	Test perplexity	32.26	—	Unverified
3	Larger Transformer 771M (pre-trained)	Test perplexity	28.1	—	Unverified
4	OPT 1.3B	Test perplexity	19.55	—	Unverified
5	GPT-Neo 125M	Test perplexity	17.83	—	Unverified
6	OPT 2.7B	Test perplexity	17.81	—	Unverified
7	Smaller Transformer 126M (fine-tuned)	Test perplexity	12	—	Unverified
8	GPT-Neo 1.3B	Test perplexity	11.46	—	Unverified
9	Transformer 125M	Test perplexity	10.7	—	Unverified
10	GPT-Neo 2.7B	Test perplexity	10.44	—	Unverified