Language Modelling

A language model is a model of natural language. Language models are useful for a variety of tasks, including speech recognition, machine translation, natural language generation (generating more human-like text), optical character recognition, route optimization, handwriting recognition, grammar induction, and information retrieval.

Large language models (LLMs), currently their most advanced form, are predominantly based on transformers trained on larger datasets (frequently using words scraped from the public internet). They have superseded recurrent neural network-based models, which had previously superseded the purely statistical models, such as word n-gram language model.

Source: Wikipedia

Papers

Recently Added Most Hyped Most Active Needs Verification Most Verified

Showing 8001–8025 of 17610 papers

Title	Date	Tasks	Status	Hype
A Survey of Confidence Estimation and Calibration in Large Language Models	Nov 14, 2023	Language Modelling	—Unverified	0
LLatrieval: LLM-Verified Retrieval for Verifiable Generation	Nov 14, 2023	Language ModelingLanguage Modelling	CodeCode Available	1
Zero-shot audio captioning with audio-language model guidance and audio context keywords	Nov 14, 2023	Audio captioningDescriptive	CodeCode Available	1
Just Ask One More Time! Self-Agreement Improves Reasoning of Language Models in (Almost) All Scenarios	Nov 14, 2023	AllDecoder	—Unverified	0
MechAgents: Large language model multi-agent collaborations can solve mechanics problems, generate new data, and integrate knowledge	Nov 14, 2023	Language ModelingLanguage Modelling	CodeCode Available	1
Anti-LM Decoding for Zero-shot In-context Machine Translation	Nov 14, 2023	In-Context LearningLanguage Modeling	CodeCode Available	0
Activity Sparsity Complements Weight Sparsity for Efficient RNN Inference	Nov 13, 2023	Deep LearningLanguage Modeling	—Unverified	0
Pretrain like Your Inference: Masked Tuning Improves Zero-Shot Composed Image Retrieval	Nov 13, 2023	Contrastive LearningImage Retrieval	CodeCode Available	0
Reducing the Need for Backpropagation and Discovering Better Optima With Explicit Optimizations of Neural Networks	Nov 13, 2023	Language Modelling	—Unverified	0
Controlled Text Generation for Black-box Language Models via Score-based Progressive Editor	Nov 13, 2023	Language ModelingLanguage Modelling	CodeCode Available	0
Language Model-In-The-Loop: Data Optimal Approach to Learn-To-Recommend Actions in Text Games	Nov 13, 2023	Language ModelingLanguage Modelling	—Unverified	0
AuthentiGPT: Detecting Machine-Generated Text via Black-Box Language Models Denoising	Nov 13, 2023	DenoisingLanguage Modeling	—Unverified	0
Gen-Z: Generative Zero-Shot Text Classification with Contextualized Label Descriptions	Nov 13, 2023	ClassificationLanguage Modeling	—Unverified	0
Teach me with a Whisper: Enhancing Large Language Models for Analyzing Spoken Transcripts using Speech Embeddings	Nov 13, 2023	Knowledge DistillationLanguage Modeling	—Unverified	0
On Elastic Language Models	Nov 13, 2023	Information RetrievalKnowledge Distillation	—Unverified	0
On The Truthfulness of 'Surprisingly Likely' Responses of Large Language Models	Nov 13, 2023	Language ModelingLanguage Modelling	—Unverified	0
In Search of the Long-Tail: Systematic Generation of Long-Tail Inferential Knowledge via Logical Rule Guided Search	Nov 13, 2023	Language ModellingNatural Language Inference	CodeCode Available	0
Semi-automatic Data Enhancement for Document-Level Relation Extraction with Distant Supervision from Large Language Models	Nov 13, 2023	Document-level Relation ExtractionIn-Context Learning	CodeCode Available	1
The Impact of Large Language Models on Scientific Discovery: a Preliminary Study using GPT-4	Nov 13, 2023	Computational chemistryDrug Discovery	—Unverified	0
On the Discussion of Large Language Models: Symmetry of Agents and Interplay with Prompts	Nov 13, 2023	Language ModelingLanguage Modelling	—Unverified	0
Towards the Law of Capacity Gap in Distilling Language Models	Nov 13, 2023	Language Modelling	CodeCode Available	1
ViLMA: A Zero-Shot Benchmark for Linguistic and Temporal Grounding in Video-Language Models	Nov 13, 2023	counterfactualLanguage Modeling	—Unverified	0
Explanation-aware Soft Ensemble Empowers Large Language Model In-context Learning	Nov 13, 2023	In-Context LearningLanguage Modeling	—Unverified	0
An Analysis and Mitigation of the Reversal Curse	Nov 13, 2023	DenoisingLanguage Modelling	CodeCode Available	1
SPHINX: The Joint Mixing of Weights, Tasks, and Visual Embeddings for Multi-modal Large Language Models	Nov 13, 2023	Described Object DetectionLanguage Modeling	CodeCode Available	4

Show:10 25 50

← PrevPage 321 of 705Next →

All datasets WikiText-103 Penn Treebank (Word Level)enwik8 The Pile WikiText-2 LAMBADA One Billion Word Text8 Penn Treebank (Character Level)Hutter Prize OpenWebText SALMon

Benchmark Results

#	Model	Metric	Claimed	Verified	Status
1	Decay RNN	Validation perplexity	76.67	—	Unverified
2	GRU	Validation perplexity	53.78	—	Unverified
3	LSTM	Validation perplexity	52.73	—	Unverified
4	LSTM	Test perplexity	48.7	—	Unverified
5	Temporal CNN	Test perplexity	45.2	—	Unverified
6	TCN	Test perplexity	45.19	—	Unverified
7	GCNN-8	Test perplexity	44.9	—	Unverified
8	Neural cache model (size = 100)	Test perplexity	44.8	—	Unverified
9	Neural cache model (size = 2,000)	Test perplexity	40.8	—	Unverified
10	GPT-2 Small	Test perplexity	37.5	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TCN	Test perplexity	108.47	—	Unverified
2	Seq-U-Net	Test perplexity	107.95	—	Unverified
3	GRU (Bai et al., 2018)	Test perplexity	92.48	—	Unverified
4	R-Transformer	Test perplexity	84.38	—	Unverified
5	Zaremba et al. (2014) - LSTM (medium)	Test perplexity	82.7	—	Unverified
6	Gal & Ghahramani (2016) - Variational LSTM (medium)	Test perplexity	79.7	—	Unverified
7	LSTM (Bai et al., 2018)	Test perplexity	78.93	—	Unverified
8	Zaremba et al. (2014) - LSTM (large)	Test perplexity	78.4	—	Unverified
9	Gal & Ghahramani (2016) - Variational LSTM (large)	Test perplexity	75.2	—	Unverified
10	Inan et al. (2016) - Variational RHN	Test perplexity	66	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	LSTM (7 layers)	Bit per Character (BPC)	1.67	—	Unverified
2	Hypernetworks	Bit per Character (BPC)	1.34	—	Unverified
3	SHA-LSTM (4 layers, h=1024, no attention head)	Bit per Character (BPC)	1.33	—	Unverified
4	LN HM-LSTM	Bit per Character (BPC)	1.32	—	Unverified
5	ByteNet	Bit per Character (BPC)	1.31	—	Unverified
6	Recurrent Highway Networks	Bit per Character (BPC)	1.27	—	Unverified
7	Large FS-LSTM-4	Bit per Character (BPC)	1.25	—	Unverified
8	Large mLSTM	Bit per Character (BPC)	1.24	—	Unverified
9	AWD-LSTM (3 layers)	Bit per Character (BPC)	1.23	—	Unverified
10	Cluster-Former (#C=512)	Bit per Character (BPC)	1.22	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Smaller Transformer 126M (pre-trained)	Test perplexity	33	—	Unverified
2	OPT 125M	Test perplexity	32.26	—	Unverified
3	Larger Transformer 771M (pre-trained)	Test perplexity	28.1	—	Unverified
4	OPT 1.3B	Test perplexity	19.55	—	Unverified
5	GPT-Neo 125M	Test perplexity	17.83	—	Unverified
6	OPT 2.7B	Test perplexity	17.81	—	Unverified
7	Smaller Transformer 126M (fine-tuned)	Test perplexity	12	—	Unverified
8	GPT-Neo 1.3B	Test perplexity	11.46	—	Unverified
9	Transformer 125M	Test perplexity	10.7	—	Unverified
10	GPT-Neo 2.7B	Test perplexity	10.44	—	Unverified