Language Modelling

A language model is a model of natural language. Language models are useful for a variety of tasks, including speech recognition, machine translation, natural language generation (generating more human-like text), optical character recognition, route optimization, handwriting recognition, grammar induction, and information retrieval.

Large language models (LLMs), currently their most advanced form, are predominantly based on transformers trained on larger datasets (frequently using words scraped from the public internet). They have superseded recurrent neural network-based models, which had previously superseded the purely statistical models, such as word n-gram language model.

Source: Wikipedia

Papers

Recently Added Most Hyped Most Active Needs Verification Most Verified

Showing 6551–6575 of 17610 papers

Title	Date	Tasks	Status	Score
The Knowledge Alignment Problem: Bridging Human and External Knowledge for Large Language Models	May 23, 2023	HallucinationLanguage Modeling	CodeCode Available	5
UFIN: Universal Feature Interaction Network for Multi-Domain Click-Through Rate Prediction	Nov 27, 2023	Click-Through Rate PredictionKnowledge Distillation	CodeCode Available	5
Lipsum-FT: Robust Fine-Tuning of Zero-Shot Models Using Random Text Guidance	Apr 1, 2024	image-classificationImage Classification	CodeCode Available	5
Structural Self-Supervised Objectives for Transformers	Sep 15, 2023	Fact VerificationLanguage Modeling	CodeCode Available	5
Leveraging Online Data to Enhance Medical Knowledge in a Small Persian Language Model	May 21, 2025	Language ModelingLanguage Modelling	CodeCode Available	5
Self-supervised Preference Optimization: Enhance Your Language Model with Preference Degree Awareness	Sep 26, 2024	Language ModelingLanguage Modelling	CodeCode Available	5
On the Complementary Nature of Knowledge Graph Embedding, Fine Grain Entity Types, and Language Modeling	Oct 12, 2020	Graph EmbeddingKnowledge Graph Embedding	CodeCode Available	5
Self-Supervised Knowledge Assimilation for Expert-Layman Text Style Transfer	Oct 6, 2021	Language ModellingSelf-Supervised Learning	CodeCode Available	5
KRLS: Improving End-to-End Response Generation in Task Oriented Dialog with Reinforced Keywords Learning	Nov 30, 2022	Language ModelingLanguage Modelling	CodeCode Available	5
LecEval: An Automated Metric for Multimodal Knowledge Acquisition in Multimedia Learning	May 4, 2025	Language ModelingLanguage Modelling	CodeCode Available	5
Phone-ing it in: Towards Flexible Multi-Modal Language Model Training by Phonetic Representations of Data	May 1, 2022	Language ModelingLanguage Modelling	CodeCode Available	5
Self-Supervised Contrastive Learning with Adversarial Perturbations for Defending Word Substitution-based Attacks	Jul 15, 2021	Adversarial AttackContrastive Learning	CodeCode Available	5
Structured Content Preservation for Unsupervised Text Style Transfer	Oct 15, 2018	Language ModelingLanguage Modelling	CodeCode Available	5
Leveraging Multimodal LLM for Inspirational User Interface Search	Jan 29, 2025	Language ModelingLanguage Modelling	CodeCode Available	5
Structured Dialogue System for Mental Health: An LLM Chatbot Leveraging the PM+ Guidelines	Nov 16, 2024	ChatbotLanguage Modeling	CodeCode Available	5
Regulation of Language Models With Interpretability Will Likely Result In A Performance Trade-Off	Dec 12, 2024	Language ModelingLanguage Modelling	CodeCode Available	5
Mitigating Reversal Curse in Large Language Models via Semantic-aware Permutation Training	Mar 1, 2024	Language Modelling	CodeCode Available	5
Mitigating Test-Time Bias for Fair Image Retrieval	May 23, 2023	Image RetrievalLanguage Modeling	CodeCode Available	5
Structured Like a Language Model: Analysing AI as an Automated Subject	Dec 8, 2022	Language ModelingLanguage Modelling	CodeCode Available	5
Mitigating the Bias of Large Language Model Evaluation	Sep 25, 2024	Instruction FollowingLanguage Model Evaluation	CodeCode Available	5
Phonemic Transcription of Low-Resource Tonal Languages	Dec 1, 2017	Acoustic ModellingLanguage Modeling	CodeCode Available	5
Recurrent Neural Networks Hardware Implementation on FPGA	Nov 17, 2015	CPULanguage Modeling	CodeCode Available	5
Mitigating the Impact of Outlier Channels for Language Model Quantization with Activation Regularization	Apr 4, 2024	GPULanguage Modeling	CodeCode Available	5
Structured Sequence Modeling with Graph Convolutional Recurrent Networks	Dec 22, 2016	Language ModelingLanguage Modelling	CodeCode Available	5
Self-Refined Large Language Model as Automated Reward Function Designer for Deep Reinforcement Learning in Robotics	Sep 13, 2023	Common Sense ReasoningDeep Reinforcement Learning	CodeCode Available	5

Show:10 25 50

← PrevPage 263 of 705Next →

All datasets WikiText-103 Penn Treebank (Word Level)enwik8 The Pile WikiText-2 LAMBADA One Billion Word Text8 Penn Treebank (Character Level)Hutter Prize OpenWebText SALMon

Benchmark Results

#	Model	Metric	Claimed	Verified	Status
1	Decay RNN	Validation perplexity	76.67	—	Unverified
2	GRU	Validation perplexity	53.78	—	Unverified
3	LSTM	Validation perplexity	52.73	—	Unverified
4	LSTM	Test perplexity	48.7	—	Unverified
5	Temporal CNN	Test perplexity	45.2	—	Unverified
6	TCN	Test perplexity	45.19	—	Unverified
7	GCNN-8	Test perplexity	44.9	—	Unverified
8	Neural cache model (size = 100)	Test perplexity	44.8	—	Unverified
9	Neural cache model (size = 2,000)	Test perplexity	40.8	—	Unverified
10	GPT-2 Small	Test perplexity	37.5	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TCN	Test perplexity	108.47	—	Unverified
2	Seq-U-Net	Test perplexity	107.95	—	Unverified
3	GRU (Bai et al., 2018)	Test perplexity	92.48	—	Unverified
4	R-Transformer	Test perplexity	84.38	—	Unverified
5	Zaremba et al. (2014) - LSTM (medium)	Test perplexity	82.7	—	Unverified
6	Gal & Ghahramani (2016) - Variational LSTM (medium)	Test perplexity	79.7	—	Unverified
7	LSTM (Bai et al., 2018)	Test perplexity	78.93	—	Unverified
8	Zaremba et al. (2014) - LSTM (large)	Test perplexity	78.4	—	Unverified
9	Gal & Ghahramani (2016) - Variational LSTM (large)	Test perplexity	75.2	—	Unverified
10	Inan et al. (2016) - Variational RHN	Test perplexity	66	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	LSTM (7 layers)	Bit per Character (BPC)	1.67	—	Unverified
2	Hypernetworks	Bit per Character (BPC)	1.34	—	Unverified
3	SHA-LSTM (4 layers, h=1024, no attention head)	Bit per Character (BPC)	1.33	—	Unverified
4	LN HM-LSTM	Bit per Character (BPC)	1.32	—	Unverified
5	ByteNet	Bit per Character (BPC)	1.31	—	Unverified
6	Recurrent Highway Networks	Bit per Character (BPC)	1.27	—	Unverified
7	Large FS-LSTM-4	Bit per Character (BPC)	1.25	—	Unverified
8	Large mLSTM	Bit per Character (BPC)	1.24	—	Unverified
9	AWD-LSTM (3 layers)	Bit per Character (BPC)	1.23	—	Unverified
10	Cluster-Former (#C=512)	Bit per Character (BPC)	1.22	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Smaller Transformer 126M (pre-trained)	Test perplexity	33	—	Unverified
2	OPT 125M	Test perplexity	32.26	—	Unverified
3	Larger Transformer 771M (pre-trained)	Test perplexity	28.1	—	Unverified
4	OPT 1.3B	Test perplexity	19.55	—	Unverified
5	GPT-Neo 125M	Test perplexity	17.83	—	Unverified
6	OPT 2.7B	Test perplexity	17.81	—	Unverified
7	Smaller Transformer 126M (fine-tuned)	Test perplexity	12	—	Unverified
8	GPT-Neo 1.3B	Test perplexity	11.46	—	Unverified
9	Transformer 125M	Test perplexity	10.7	—	Unverified
10	GPT-Neo 2.7B	Test perplexity	10.44	—	Unverified