Language Modelling

A language model is a model of natural language. Language models are useful for a variety of tasks, including speech recognition, machine translation, natural language generation (generating more human-like text), optical character recognition, route optimization, handwriting recognition, grammar induction, and information retrieval.

Large language models (LLMs), currently their most advanced form, are predominantly based on transformers trained on larger datasets (frequently using words scraped from the public internet). They have superseded recurrent neural network-based models, which had previously superseded the purely statistical models, such as word n-gram language model.

Source: Wikipedia

Papers

Recently Added Most Hyped Most Active Needs Verification Most Verified

Showing 9326–9350 of 17610 papers

Title	Date	Tasks	Status	Hype
On Conditional and Compositional Language Model Differentiable Prompting	Jul 4, 2023	Few-Shot LearningLanguage Modeling	—Unverified	0
Robots That Ask For Help: Uncertainty Alignment for Large Language Model Planners	Jul 4, 2023	Conformal PredictionLanguage Modeling	—Unverified	0
Prompt Tuning Pushes Farther, Contrastive Learning Pulls Closer: A Two-Stage Approach to Mitigate Social Biases	Jul 4, 2023	Contrastive Learningcounterfactual	—Unverified	0
ALBERTI, a Multilingual Domain Specific Language Model for Poetry Analysis	Jul 3, 2023	Language ModelingLanguage Modelling	—Unverified	0
ChatGPT vs. Google: A Comparative Study of Search Performance and User Experience	Jul 3, 2023	ChatbotFact Checking	—Unverified	0
Exploring the In-context Learning Ability of Large Language Model for Biomedical Concept Linking	Jul 3, 2023	In-Context LearningInformation Retrieval	—Unverified	0
Trainable Transformer in Transformer	Jul 3, 2023	AttributeIn-Context Learning	CodeCode Available	1
Large Language Models Enable Few-Shot Clustering	Jul 2, 2023	ClusteringLanguage Modeling	CodeCode Available	1
GenRec: Large Language Model for Generative Recommendation	Jul 2, 2023	Language ModelingLanguage Modelling	CodeCode Available	1
PatternGPT :A Pattern-Driven Framework for Large Language Model Text Generation	Jul 2, 2023	Federated LearningLanguage Modeling	—Unverified	0
Conformer LLMs -- Convolution Augmented Large Language Models	Jul 2, 2023	Automatic Speech RecognitionLanguage Modeling	—Unverified	0
MedCPT: Contrastive Pre-trained Transformers with Large-scale PubMed Search Logs for Zero-shot Biomedical Information Retrieval	Jul 2, 2023	Biomedical Information RetrievalContrastive Learning	CodeCode Available	2
CephGPT-4: An Interactive Multimodal Cephalometric Measurement and Diagnostic System with Visual Large Language Model	Jul 1, 2023	DiagnosticLanguage Modeling	—Unverified	0
BatGPT: A Bidirectional Autoregessive Talker from Generative Pre-trained Transformer	Jul 1, 2023	Language ModelingLanguage Modelling	CodeCode Available	2
How far is Language Model from 100% Few-shot Named Entity Recognition in Medical Domain	Jul 1, 2023	few-shot-nerFew-shot NER	CodeCode Available	1
DoReMi: Grounding Language Model by Detecting and Recovering from Plan-Execution Misalignment	Jul 1, 2023	Language ModelingLanguage Modelling	—Unverified	0
InstructEval: Systematic Evaluation of Instruction Selection Methods	Jul 1, 2023	BenchmarkingIn-Context Learning	—Unverified	0
THUIR2 at NTCIR-16 Session Search (SS) Task	Jul 1, 2023	Language ModelingLanguage Modelling	—Unverified	0
Provable Robust Watermarking for AI-Generated Text	Jun 30, 2023	Language Modelling	CodeCode Available	2
Biomedical Language Models are Robust to Sub-optimal Tokenization	Jun 30, 2023	Entity LinkingLanguage Modeling	CodeCode Available	0
Ticket-BERT: Labeling Incident Management Tickets with Language Models	Jun 30, 2023	Active LearningLanguage Modeling	—Unverified	0
LMBot: Distilling Graph Knowledge into Language Model for Graph-less Deployment in Twitter Bot Detection	Jun 30, 2023	Domain AdaptationLanguage Modeling	CodeCode Available	1
Stay on topic with Classifier-Free Guidance	Jun 30, 2023	Code GenerationCommon Sense Reasoning	—Unverified	0
Towards Open-Domain Topic Classification	Jun 29, 2023	ClassificationLanguage Modeling	—Unverified	0
Could Small Language Models Serve as Recommenders? Towards Data-centric Cold-start Recommendations	Jun 29, 2023	In-Context LearningLanguage Modeling	CodeCode Available	1

Show:10 25 50

← PrevPage 374 of 705Next →

All datasets WikiText-103 Penn Treebank (Word Level)enwik8 The Pile WikiText-2 LAMBADA One Billion Word Text8 Penn Treebank (Character Level)Hutter Prize OpenWebText SALMon

Benchmark Results

#	Model	Metric	Claimed	Verified	Status
1	Decay RNN	Validation perplexity	76.67	—	Unverified
2	GRU	Validation perplexity	53.78	—	Unverified
3	LSTM	Validation perplexity	52.73	—	Unverified
4	LSTM	Test perplexity	48.7	—	Unverified
5	Temporal CNN	Test perplexity	45.2	—	Unverified
6	TCN	Test perplexity	45.19	—	Unverified
7	GCNN-8	Test perplexity	44.9	—	Unverified
8	Neural cache model (size = 100)	Test perplexity	44.8	—	Unverified
9	Neural cache model (size = 2,000)	Test perplexity	40.8	—	Unverified
10	GPT-2 Small	Test perplexity	37.5	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TCN	Test perplexity	108.47	—	Unverified
2	Seq-U-Net	Test perplexity	107.95	—	Unverified
3	GRU (Bai et al., 2018)	Test perplexity	92.48	—	Unverified
4	R-Transformer	Test perplexity	84.38	—	Unverified
5	Zaremba et al. (2014) - LSTM (medium)	Test perplexity	82.7	—	Unverified
6	Gal & Ghahramani (2016) - Variational LSTM (medium)	Test perplexity	79.7	—	Unverified
7	LSTM (Bai et al., 2018)	Test perplexity	78.93	—	Unverified
8	Zaremba et al. (2014) - LSTM (large)	Test perplexity	78.4	—	Unverified
9	Gal & Ghahramani (2016) - Variational LSTM (large)	Test perplexity	75.2	—	Unverified
10	Inan et al. (2016) - Variational RHN	Test perplexity	66	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	LSTM (7 layers)	Bit per Character (BPC)	1.67	—	Unverified
2	Hypernetworks	Bit per Character (BPC)	1.34	—	Unverified
3	SHA-LSTM (4 layers, h=1024, no attention head)	Bit per Character (BPC)	1.33	—	Unverified
4	LN HM-LSTM	Bit per Character (BPC)	1.32	—	Unverified
5	ByteNet	Bit per Character (BPC)	1.31	—	Unverified
6	Recurrent Highway Networks	Bit per Character (BPC)	1.27	—	Unverified
7	Large FS-LSTM-4	Bit per Character (BPC)	1.25	—	Unverified
8	Large mLSTM	Bit per Character (BPC)	1.24	—	Unverified
9	AWD-LSTM (3 layers)	Bit per Character (BPC)	1.23	—	Unverified
10	Cluster-Former (#C=512)	Bit per Character (BPC)	1.22	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Smaller Transformer 126M (pre-trained)	Test perplexity	33	—	Unverified
2	OPT 125M	Test perplexity	32.26	—	Unverified
3	Larger Transformer 771M (pre-trained)	Test perplexity	28.1	—	Unverified
4	OPT 1.3B	Test perplexity	19.55	—	Unverified
5	GPT-Neo 125M	Test perplexity	17.83	—	Unverified
6	OPT 2.7B	Test perplexity	17.81	—	Unverified
7	Smaller Transformer 126M (fine-tuned)	Test perplexity	12	—	Unverified
8	GPT-Neo 1.3B	Test perplexity	11.46	—	Unverified
9	Transformer 125M	Test perplexity	10.7	—	Unverified
10	GPT-Neo 2.7B	Test perplexity	10.44	—	Unverified