SOTAVerified

Language Modelling

A language model is a model of natural language. Language models are useful for a variety of tasks, including speech recognition, machine translation, natural language generation (generating more human-like text), optical character recognition, route optimization, handwriting recognition, grammar induction, and information retrieval.

Large language models (LLMs), currently their most advanced form, are predominantly based on transformers trained on larger datasets (frequently using words scraped from the public internet). They have superseded recurrent neural network-based models, which had previously superseded the purely statistical models, such as word n-gram language model.

Source: Wikipedia

Papers

Showing 60016050 of 17610 papers

TitleStatusHype
Learning to Generate Compositional Color DescriptionsCode0
Understanding Architectures Learnt by Cell-based Neural Architecture SearchCode0
Multimodal data matters: language model pre-training over structured and unstructured electronic health recordsCode0
Narrative Shift Detection: A Hybrid Approach of Dynamic Topic Models and Large Language ModelsCode0
RILQ: Rank-Insensitive LoRA-based Quantization Error Compensation for Boosting 2-bit Large Language Model AccuracyCode0
Learning to Solve Voxel Building Embodied Tasks from Pixels and Natural Language InstructionsCode0
MANGO: A Benchmark for Evaluating Mapping and Navigation Abilities of Large Language ModelsCode0
Product Information Extraction using ChatGPTCode0
Riemannian Normalizing Flow on Variational Wasserstein Autoencoder for Text ModelingCode0
RIDE: Enhancing Large Language Model Alignment through Restyled In-Context Learning Demonstration ExemplarsCode0
MAMUT: A Novel Framework for Modifying Mathematical Formulas for the Generation of Specialized Datasets for Language Model TrainingCode0
Professor Forcing: A New Algorithm for Training Recurrent NetworksCode0
LLM-QFL: Distilling Large Language Model for Quantum Federated LearningCode0
Profiling Patient Transcript Using Large Language Model Reasoning Augmentation for Alzheimer's Disease DetectionCode0
Quite Good, but Not Enough: Nationality Bias in Large Language Models -- A Case Study of ChatGPTCode0
Musketeer: Joint Training for Multi-task Vision Language Model with Task Explanation PromptsCode0
The Quantization Model of Neural ScalingCode0
Multiplicative LSTM for sequence modellingCode0
LOGIN: A Large Language Model Consulted Graph Neural Network Training FrameworkCode0
Just What You Desire: Constrained Timeline Summarization with Self-Reflection for Enhanced RelevanceCode0
KatzBot: Revolutionizing Academic Chatbot for Enhanced CommunicationCode0
The Referential Reader: A Recurrent Entity Network for Anaphora ResolutionCode0
MSDT: Masked Language Model Scoring Defense in Text DomainCode0
Progressive Class Semantic Matching for Semi-supervised Text ClassificationCode0
Progressive Distillation Based on Masked Generation Feature Method for Knowledge Graph CompletionCode0
LEP-AD: Language Embedding of Proteins and Attention to Drugs predicts drug target interactionsCode0
Training Neural Networks as Recognizers of Formal LanguagesCode0
Training Neural Response Selection for Task-Oriented Dialogue SystemsCode0
Large Language Model-Guided Prediction Toward Quantum Materials SynthesisCode0
LLM-RankFusion: Mitigating Intrinsic Inconsistency in LLM-based RankingCode0
Projective Methods for Mitigating Gender Bias in Pre-trained Language ModelsCode0
Project SHADOW: Symbolic Higher-order Associative Deductive reasoning On Wikidata using LM probingCode0
Neural machine translation system for Lezgian, Russian and Azerbaijani languagesCode0
ProMap: Effective Bilingual Lexicon Induction via Language Model PromptingCode0
Length Optimization in Conformal PredictionCode0
Prometheus Chatbot: Knowledge Graph Collaborative Large Language Model for Computer Components RecommendationCode0
Revisiting Topic-Guided Language ModelsCode0
Revisiting The Classics: A Study on Identifying and Rectifying Gender Stereotypes in Rhymes and PoemsCode0
LLM Reading Tea Leaves: Automatically Evaluating Topic Models with Large Language ModelsCode0
NarrowBERT: Accelerating Masked Language Model Pretraining and InferenceCode0
LEMON: Language-Based Environment Manipulation via Execution-Guided Pre-trainingCode0
Promoting Exploration in Memory-Augmented Adam using Critical MomentaCode0
Promoting Open-domain Dialogue Generation through Learning Pattern Information between Contexts and ResponsesCode0
Neural Machine Translation of Clinical Text: An Empirical Investigation into Multilingual Pre-Trained Language Models and Transfer-LearningCode0
Noise Augmented Fine Tuning for Mitigating Hallucinations in Large Language ModelsCode0
KidLM: Advancing Language Models for Children -- Early Insights and Future DirectionsCode0
MST5 -- Multilingual Question Answering over Knowledge GraphsCode0
Language Detoxification with Attribute-Discriminative Latent SpaceCode0
Linguistic Frameworks Go Toe-to-Toe at Neuro-Symbolic Language ModelingCode0
Oracle performance for visual captioningCode0
Show:102550
← PrevPage 121 of 353Next →

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1Decay RNNValidation perplexity76.67Unverified
2GRUValidation perplexity53.78Unverified
3LSTMValidation perplexity52.73Unverified
4LSTMTest perplexity48.7Unverified
5Temporal CNNTest perplexity45.2Unverified
6TCNTest perplexity45.19Unverified
7GCNN-8Test perplexity44.9Unverified
8Neural cache model (size = 100)Test perplexity44.8Unverified
9Neural cache model (size = 2,000)Test perplexity40.8Unverified
10GPT-2 SmallTest perplexity37.5Unverified
#ModelMetricClaimedVerifiedStatus
1TCNTest perplexity108.47Unverified
2Seq-U-NetTest perplexity107.95Unverified
3GRU (Bai et al., 2018)Test perplexity92.48Unverified
4R-TransformerTest perplexity84.38Unverified
5Zaremba et al. (2014) - LSTM (medium)Test perplexity82.7Unverified
6Gal & Ghahramani (2016) - Variational LSTM (medium)Test perplexity79.7Unverified
7LSTM (Bai et al., 2018)Test perplexity78.93Unverified
8Zaremba et al. (2014) - LSTM (large)Test perplexity78.4Unverified
9Gal & Ghahramani (2016) - Variational LSTM (large)Test perplexity75.2Unverified
10Inan et al. (2016) - Variational RHNTest perplexity66Unverified
#ModelMetricClaimedVerifiedStatus
1LSTM (7 layers)Bit per Character (BPC)1.67Unverified
2HypernetworksBit per Character (BPC)1.34Unverified
3SHA-LSTM (4 layers, h=1024, no attention head)Bit per Character (BPC)1.33Unverified
4LN HM-LSTMBit per Character (BPC)1.32Unverified
5ByteNetBit per Character (BPC)1.31Unverified
6Recurrent Highway NetworksBit per Character (BPC)1.27Unverified
7Large FS-LSTM-4Bit per Character (BPC)1.25Unverified
8Large mLSTMBit per Character (BPC)1.24Unverified
9AWD-LSTM (3 layers)Bit per Character (BPC)1.23Unverified
10Cluster-Former (#C=512)Bit per Character (BPC)1.22Unverified
#ModelMetricClaimedVerifiedStatus
1Smaller Transformer 126M (pre-trained)Test perplexity33Unverified
2OPT 125MTest perplexity32.26Unverified
3Larger Transformer 771M (pre-trained)Test perplexity28.1Unverified
4OPT 1.3BTest perplexity19.55Unverified
5GPT-Neo 125MTest perplexity17.83Unverified
6OPT 2.7BTest perplexity17.81Unverified
7Smaller Transformer 126M (fine-tuned)Test perplexity12Unverified
8GPT-Neo 1.3BTest perplexity11.46Unverified
9Transformer 125MTest perplexity10.7Unverified
10GPT-Neo 2.7BTest perplexity10.44Unverified