Language Modelling

A language model is a model of natural language. Language models are useful for a variety of tasks, including speech recognition, machine translation, natural language generation (generating more human-like text), optical character recognition, route optimization, handwriting recognition, grammar induction, and information retrieval.

Large language models (LLMs), currently their most advanced form, are predominantly based on transformers trained on larger datasets (frequently using words scraped from the public internet). They have superseded recurrent neural network-based models, which had previously superseded the purely statistical models, such as word n-gram language model.

Source: Wikipedia

Papers

Recently Added Most Hyped Most Active Needs Verification Most Verified

Showing 6251–6275 of 17610 papers

Title	Date	Tasks	Status	Hype
Evaluating GPT-4 with Vision on Detection of Radiological Findings on Chest Radiographs	Mar 22, 2024	DiagnosticLanguage Modeling	—Unverified	0
Unifying Large Language Model and Deep Reinforcement Learning for Human-in-Loop Interactive Socially-aware Navigation	Mar 22, 2024	BenchmarkingDeep Reinforcement Learning	—Unverified	0
Language-Based Depth Hints for Monocular Depth Estimation	Mar 22, 2024	Depth EstimationInstance Segmentation	—Unverified	0
Measuring Gender and Racial Biases in Large Language Models	Mar 22, 2024	Decision MakingLanguage Modelling	—Unverified	0
Controlled Training Data Generation with Diffusion Models	Mar 22, 2024	Language ModelingLanguage Modelling	—Unverified	0
Text Clustering with Large Language Model Embeddings	Mar 22, 2024	ClusteringDimensionality Reduction	—Unverified	0
CoLLEGe: Concept Embedding Generation for Large Language Models	Mar 22, 2024	Language ModelingLanguage Modelling	—Unverified	0
LLaVA-PruMerge: Adaptive Token Reduction for Efficient Large Multimodal Models	Mar 22, 2024	Language ModellingLarge Language Model	CodeCode Available	2
Long-CLIP: Unlocking the Long-Text Capability of CLIP	Mar 22, 2024	Image GenerationImage Retrieval	CodeCode Available	4
Investigating the Performance of Language Models for Completing Code in Functional Programming Languages: a Haskell Case Study	Mar 22, 2024	Code CompletionHumanEval	CodeCode Available	0
Make VLM Recognize Visual Hallucination on Cartoon Character Image with Pose Information	Mar 22, 2024	3D ReconstructionHallucination	—Unverified	0
Bioinformatics and Biomedical Informatics with ChatGPT: Year One Review	Mar 22, 2024	ChatbotDrug Discovery	—Unverified	0
An Exploratory Investigation into Code License Infringements in Large Language Model Training Datasets	Mar 22, 2024	Language ModelingLanguage Modelling	CodeCode Available	0
Stance Reasoner: Zero-Shot Stance Detection on Social Media with Explicit Reasoning	Mar 22, 2024	Few-Shot Stance DetectionIn-Context Learning	CodeCode Available	0
Sequence-to-Sequence Language Models for Character and Emotion Detection in Dream Narratives	Mar 21, 2024	In-Context LearningLanguage Modeling	—Unverified	0
Sequential Decision-Making for Inline Text Autocomplete	Mar 21, 2024	Decision MakingLanguage Modelling	—Unverified	0
PE-GPT: A Physics-Informed Interactive Large Language Model for Power Converter Modulation Design	Mar 21, 2024	In-Context LearningLanguage Modeling	—Unverified	0
VidLA: Video-Language Alignment at Scale	Mar 21, 2024	Language ModellingVisual Grounding	—Unverified	0
A Multimodal Approach to Device-Directed Speech Detection with Large Language Models	Mar 21, 2024	Automatic Speech RecognitionAutomatic Speech Recognition (ASR)	—Unverified	0
MMIDR: Teaching Large Language Model to Interpret Multimodal Misinformation via Knowledge Distillation	Mar 21, 2024	Data AugmentationDecision Making	CodeCode Available	1
Leveraging Large Language Model-based Room-Object Relationships Knowledge for Enhancing Multimodal-Input Object Goal Navigation	Mar 21, 2024	Common Sense ReasoningLanguage Modeling	—Unverified	0
Recourse for reclamation: Chatting with generative language models	Mar 21, 2024	Information RetrievalLanguage Modeling	—Unverified	0
Context Quality Matters in Training Fusion-in-Decoder for Extractive Open-Domain Question Answering	Mar 21, 2024	DecoderLanguage Modeling	—Unverified	0
Cobra: Extending Mamba to Multi-Modal Large Language Model for Efficient Inference	Mar 21, 2024	Language ModelingLanguage Modelling	CodeCode Available	3
Regularized Adaptive Momentum Dual Averaging with an Efficient Inexact Subproblem Solver for Training Structured Neural Network	Mar 21, 2024	Language ModelingLanguage Modelling	CodeCode Available	0

Show:10 25 50

← PrevPage 251 of 705Next →

All datasets WikiText-103 Penn Treebank (Word Level)enwik8 The Pile WikiText-2 LAMBADA One Billion Word Text8 Penn Treebank (Character Level)Hutter Prize OpenWebText SALMon

Benchmark Results

#	Model	Metric	Claimed	Verified	Status
1	Decay RNN	Validation perplexity	76.67	—	Unverified
2	GRU	Validation perplexity	53.78	—	Unverified
3	LSTM	Validation perplexity	52.73	—	Unverified
4	LSTM	Test perplexity	48.7	—	Unverified
5	Temporal CNN	Test perplexity	45.2	—	Unverified
6	TCN	Test perplexity	45.19	—	Unverified
7	GCNN-8	Test perplexity	44.9	—	Unverified
8	Neural cache model (size = 100)	Test perplexity	44.8	—	Unverified
9	Neural cache model (size = 2,000)	Test perplexity	40.8	—	Unverified
10	GPT-2 Small	Test perplexity	37.5	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TCN	Test perplexity	108.47	—	Unverified
2	Seq-U-Net	Test perplexity	107.95	—	Unverified
3	GRU (Bai et al., 2018)	Test perplexity	92.48	—	Unverified
4	R-Transformer	Test perplexity	84.38	—	Unverified
5	Zaremba et al. (2014) - LSTM (medium)	Test perplexity	82.7	—	Unverified
6	Gal & Ghahramani (2016) - Variational LSTM (medium)	Test perplexity	79.7	—	Unverified
7	LSTM (Bai et al., 2018)	Test perplexity	78.93	—	Unverified
8	Zaremba et al. (2014) - LSTM (large)	Test perplexity	78.4	—	Unverified
9	Gal & Ghahramani (2016) - Variational LSTM (large)	Test perplexity	75.2	—	Unverified
10	Inan et al. (2016) - Variational RHN	Test perplexity	66	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	LSTM (7 layers)	Bit per Character (BPC)	1.67	—	Unverified
2	Hypernetworks	Bit per Character (BPC)	1.34	—	Unverified
3	SHA-LSTM (4 layers, h=1024, no attention head)	Bit per Character (BPC)	1.33	—	Unverified
4	LN HM-LSTM	Bit per Character (BPC)	1.32	—	Unverified
5	ByteNet	Bit per Character (BPC)	1.31	—	Unverified
6	Recurrent Highway Networks	Bit per Character (BPC)	1.27	—	Unverified
7	Large FS-LSTM-4	Bit per Character (BPC)	1.25	—	Unverified
8	Large mLSTM	Bit per Character (BPC)	1.24	—	Unverified
9	AWD-LSTM (3 layers)	Bit per Character (BPC)	1.23	—	Unverified
10	Cluster-Former (#C=512)	Bit per Character (BPC)	1.22	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Smaller Transformer 126M (pre-trained)	Test perplexity	33	—	Unverified
2	OPT 125M	Test perplexity	32.26	—	Unverified
3	Larger Transformer 771M (pre-trained)	Test perplexity	28.1	—	Unverified
4	OPT 1.3B	Test perplexity	19.55	—	Unverified
5	GPT-Neo 125M	Test perplexity	17.83	—	Unverified
6	OPT 2.7B	Test perplexity	17.81	—	Unverified
7	Smaller Transformer 126M (fine-tuned)	Test perplexity	12	—	Unverified
8	GPT-Neo 1.3B	Test perplexity	11.46	—	Unverified
9	Transformer 125M	Test perplexity	10.7	—	Unverified
10	GPT-Neo 2.7B	Test perplexity	10.44	—	Unverified