Language Modelling

A language model is a model of natural language. Language models are useful for a variety of tasks, including speech recognition, machine translation, natural language generation (generating more human-like text), optical character recognition, route optimization, handwriting recognition, grammar induction, and information retrieval.

Large language models (LLMs), currently their most advanced form, are predominantly based on transformers trained on larger datasets (frequently using words scraped from the public internet). They have superseded recurrent neural network-based models, which had previously superseded the purely statistical models, such as word n-gram language model.

Source: Wikipedia

Papers

Recently Added Most Hyped Most Active Needs Verification Most Verified

Showing 10326–10350 of 17610 papers

Title	Date	Tasks	Status	Hype
FlexGen: High-Throughput Generative Inference of Large Language Models with a Single GPU	Mar 13, 2023	CPUGPU	CodeCode Available	5
ODIN: On-demand Data Formulation to Mitigate Dataset Lock-in	Mar 13, 2023	DiversityLanguage Modeling	—Unverified	0
Model-tuning Via Prompts Makes NLP Models Adversarially Robust	Mar 13, 2023	Adversarial RobustnessLanguage Modelling	CodeCode Available	0
A comprehensive evaluation of ChatGPT's zero-shot Text-to-SQL capability	Mar 12, 2023	Code GenerationLanguage Modeling	CodeCode Available	1
Accommodating Audio Modality in CLIP for Multimodal Processing	Mar 12, 2023	AudioCapsContrastive Learning	CodeCode Available	0
Consistency Analysis of ChatGPT	Mar 11, 2023	Few-Shot LearningLanguage Modelling	—Unverified	0
Stabilizing Transformer Training by Preventing Attention Entropy Collapse	Mar 11, 2023	Automatic Speech Recognitionimage-classification	CodeCode Available	2
Learning Combinatorial Prompts for Universal Controllable Image Captioning	Mar 11, 2023	controllable image captioningImage Captioning	—Unverified	0
Towards MoE Deployment: Mitigating Inefficiencies in Mixture-of-Expert (MoE) Inference	Mar 10, 2023	CPUDecoder	—Unverified	0
Susceptibility to Influence of Large Language Models	Mar 10, 2023	AttributeLanguage Modelling	—Unverified	0
Algorithmic Ghost in the Research Shell: Large Language Models and Academic Knowledge Creation in Management Research	Mar 10, 2023	ArticlesLanguage Modeling	—Unverified	0
An Overview on Language Models: Recent Developments and Outlook	Mar 10, 2023	Language ModelingLanguage Modelling	—Unverified	0
Open-Ended Medical Visual Question Answering Through Prefix Tuning of Language Models	Mar 10, 2023	Language ModelingLanguage Modelling	CodeCode Available	1
Rewarding Chatbots for Real-World Engagement with Millions of Users	Mar 10, 2023	ChatbotLanguage Modelling	—Unverified	0
Tag2Text: Guiding Vision-Language Model via Image Tagging	Mar 10, 2023	Language ModelingLanguage Modelling	CodeCode Available	4
Iterative Few-shot Semantic Segmentation from Image Label Text	Mar 10, 2023	Few-Shot Semantic SegmentationLanguage Modeling	CodeCode Available	1
Refined Vision-Language Modeling for Fine-grained Multi-modal Pre-training	Mar 9, 2023	Image-text matchingLanguage Modeling	—Unverified	0
Planning with Large Language Models for Code Generation	Mar 9, 2023	Code GenerationLanguage Modelling	—Unverified	0
Knowledge-augmented Few-shot Visual Relation Detection	Mar 9, 2023	DiversityFew-Shot Learning	—Unverified	0
Can a Frozen Pretrained Language Model be used for Zero-shot Neural Retrieval on Entity-centric Questions?	Mar 9, 2023	Domain GeneralizationLanguage Modeling	—Unverified	0
Weakly-Supervised HOI Detection from Interaction Labels Only and Language/Vision-Language Priors	Mar 9, 2023	Human-Object Interaction DetectionLanguage Modeling	—Unverified	0
Magnushammer: A Transformer-Based Approach to Premise Selection	Mar 8, 2023	Automated Theorem ProvingLanguage Modeling	—Unverified	0
Extending the Pre-Training of BLOOM for Improved Support of Traditional Chinese: Models, Methods and Results	Mar 8, 2023	ArticlesLanguage Modeling	—Unverified	0
Cost-Effective Hyperparameter Optimization for Large Language Model Generation Inference	Mar 8, 2023	Hyperparameter OptimizationLanguage Modeling	CodeCode Available	4
German BERT Model for Legal Named Entity Recognition	Mar 7, 2023	Language Modellingmodel	—Unverified	0

Show:10 25 50

← PrevPage 414 of 705Next →

All datasets WikiText-103 Penn Treebank (Word Level)enwik8 The Pile WikiText-2 LAMBADA One Billion Word Text8 Penn Treebank (Character Level)Hutter Prize OpenWebText SALMon

Benchmark Results

#	Model	Metric	Claimed	Verified	Status
1	Decay RNN	Validation perplexity	76.67	—	Unverified
2	GRU	Validation perplexity	53.78	—	Unverified
3	LSTM	Validation perplexity	52.73	—	Unverified
4	LSTM	Test perplexity	48.7	—	Unverified
5	Temporal CNN	Test perplexity	45.2	—	Unverified
6	TCN	Test perplexity	45.19	—	Unverified
7	GCNN-8	Test perplexity	44.9	—	Unverified
8	Neural cache model (size = 100)	Test perplexity	44.8	—	Unverified
9	Neural cache model (size = 2,000)	Test perplexity	40.8	—	Unverified
10	GPT-2 Small	Test perplexity	37.5	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TCN	Test perplexity	108.47	—	Unverified
2	Seq-U-Net	Test perplexity	107.95	—	Unverified
3	GRU (Bai et al., 2018)	Test perplexity	92.48	—	Unverified
4	R-Transformer	Test perplexity	84.38	—	Unverified
5	Zaremba et al. (2014) - LSTM (medium)	Test perplexity	82.7	—	Unverified
6	Gal & Ghahramani (2016) - Variational LSTM (medium)	Test perplexity	79.7	—	Unverified
7	LSTM (Bai et al., 2018)	Test perplexity	78.93	—	Unverified
8	Zaremba et al. (2014) - LSTM (large)	Test perplexity	78.4	—	Unverified
9	Gal & Ghahramani (2016) - Variational LSTM (large)	Test perplexity	75.2	—	Unverified
10	Inan et al. (2016) - Variational RHN	Test perplexity	66	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	LSTM (7 layers)	Bit per Character (BPC)	1.67	—	Unverified
2	Hypernetworks	Bit per Character (BPC)	1.34	—	Unverified
3	SHA-LSTM (4 layers, h=1024, no attention head)	Bit per Character (BPC)	1.33	—	Unverified
4	LN HM-LSTM	Bit per Character (BPC)	1.32	—	Unverified
5	ByteNet	Bit per Character (BPC)	1.31	—	Unverified
6	Recurrent Highway Networks	Bit per Character (BPC)	1.27	—	Unverified
7	Large FS-LSTM-4	Bit per Character (BPC)	1.25	—	Unverified
8	Large mLSTM	Bit per Character (BPC)	1.24	—	Unverified
9	AWD-LSTM (3 layers)	Bit per Character (BPC)	1.23	—	Unverified
10	Cluster-Former (#C=512)	Bit per Character (BPC)	1.22	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Smaller Transformer 126M (pre-trained)	Test perplexity	33	—	Unverified
2	OPT 125M	Test perplexity	32.26	—	Unverified
3	Larger Transformer 771M (pre-trained)	Test perplexity	28.1	—	Unverified
4	OPT 1.3B	Test perplexity	19.55	—	Unverified
5	GPT-Neo 125M	Test perplexity	17.83	—	Unverified
6	OPT 2.7B	Test perplexity	17.81	—	Unverified
7	Smaller Transformer 126M (fine-tuned)	Test perplexity	12	—	Unverified
8	GPT-Neo 1.3B	Test perplexity	11.46	—	Unverified
9	Transformer 125M	Test perplexity	10.7	—	Unverified
10	GPT-Neo 2.7B	Test perplexity	10.44	—	Unverified