SOTAVerified

Language Modelling

A language model is a model of natural language. Language models are useful for a variety of tasks, including speech recognition, machine translation, natural language generation (generating more human-like text), optical character recognition, route optimization, handwriting recognition, grammar induction, and information retrieval.

Large language models (LLMs), currently their most advanced form, are predominantly based on transformers trained on larger datasets (frequently using words scraped from the public internet). They have superseded recurrent neural network-based models, which had previously superseded the purely statistical models, such as word n-gram language model.

Source: Wikipedia

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 10051–10100 of 17610 papers

TitleStatusHype
UFIN: Universal Feature Interaction Network for Multi-Domain Click-Through Rate PredictionCode0
Learning to Skip for Language Modeling—0
Uncertainty-aware Language Modeling for Selective Question Answering—0
Benchmarking Large Language Model Volatility—0
GPT4Video: A Unified Multimodal Large Language Model for lnstruction-Followed Understanding and Safety-Aware Generation—0
Enhancing Sentiment Analysis Results through Outlier Detection Optimization—0
LANS: A Layout-Aware Neural Solver for Plane Geometry Problem—0
Multilingual self-supervised speech representations improve the speech recognition of low-resource African languages with codeswitching—0
Solving the Right Problem is Key for Translational NLP: A Case Study in UMLS Vocabulary InsertionCode0
Tracing Influence at Scale: A Contrastive Learning Approach to Linking Public Comments and Regulator Responses—0
ÚFAL CorPipe at CRAC 2023: Larger Context Improves Multilingual Coreference ResolutionCode0
CMed-GPT: Prompt Tuning for Entity-Aware Chinese Medical Dialogue Generation—0
GATGPT: A Pre-trained Large Language Model with Graph Attention Network for Spatiotemporal Imputation—0
GPT-4V Takes the Wheel: Promises and Challenges for Pedestrian Behavior Prediction—0
DaG LLM ver 1.0: Pioneering Instruction-Tuned Language Modeling for Korean NLP—0
Controlling Large Language Model-based Agents for Large-Scale Decision-Making: An Actor-Critic Approach—0
A Cross Attention Approach to Diagnostic Explainability using Clinical Practice Guidelines for DepressionCode0
A Multi-solution Study on GDPR AI-enabled Completeness Checking of DPAs—0
Lego: Learning to Disentangle and Invert Personalized Concepts Beyond Object Appearance in Text-to-Image Diffusion Models—0
Understanding the Vulnerability of CLIP to Image CompressionCode0
PrivateLoRA For Efficient Privacy Preserving LLM—0
MAIRA-1: A specialised large multimodal model for radiology report generation—0
Towards Detecting, Recognizing, and Parsing the Address Information from Bangla Signboard: A Deep Learning-based Approach—0
Perceptual Structure in the Absence of Grounding for LLMs: The Impact of Abstractedness and Subjectivity in Color Language—0
EA-KD: Entropy-based Adaptive Knowledge Distillation—0
LM-Cocktail: Resilient Tuning of Language Models via Model Merging—0
Mitigating Large Language Model Hallucinations via Autonomous Knowledge Graph-based Retrofitting—0
Nova: Generative Language Models for Assembly Code with Hierarchical Attention and Contrastive Learning—0
Current Topological and Machine Learning Applications for Bias Detection in Text—0
Large Language Model-Enhanced Algorithm Selection: Towards Comprehensive Algorithm RepresentationCode0
Detecting out-of-distribution text using topological features of transformer-based language modelsCode0
Towards Responsible Generative AI: A Reference Architecture for Designing Foundation Model based Agents—0
Vamos: Versatile Action Models for Video UnderstandingCode0
Enhancing Visual Grounding and Generalization: A Multi-Task Cycle Training Approach for Vision-Language ModelsCode0
PhayaThaiBERT: Enhancing a Pretrained Thai Language Model with Unassimilated LoanwordsCode0
Systematic word meta-sense extensionCode0
Enabling On-Device Large Language Model Personalization with Self-Supervised Data Selection and Synthesis—0
Beyond Turing: A Comparative Analysis of Approaches for Detecting Machine-Generated Text—0
ATLANTIC: Structure-Aware Retrieval-Augmented Language Model for Interdisciplinary Science—0
Attribution and Alignment: Effects of Local Context Repetition on Utterance Production and Comprehension in Dialogue—0
From Concept to Manufacturing: Evaluating Vision-Language Models for Engineering Design—0
DocPedia: Unleashing the Power of Large Multimodal Model in the Frequency Domain for Versatile Document Understanding—0
Fingerspelling PoseNet: Enhancing Fingerspelling Translation with Pose-Based Transformer ModelsCode0
Clarity ChatGPT: An Interactive and Adaptive Processing System for Image Restoration and Enhancement—0
GPT-4V(ision) for Robotics: Multimodal Task Planning from Human Demonstration—0
Adapt in Contexts: Retrieval-Augmented Domain Adaptation via In-Context Learning—0
Unifying Corroborative and Contributive Attributions in Large Language Models—0
KBioXLM: A Knowledge-anchored Biomedical Multilingual Pretrained Language ModelCode0
Fine-Tuning Adaptive Stochastic Optimizers: Determining the Optimal Hyperparameter ε via Gradient Magnitude Histogram Analysis—0
Refactoring Programs Using Large Language Models with Few-Shot Examples—0
Show:102550
← PrevPage 202 of 353Next →
All datasetsWikiText-103Penn Treebank (Word Level)enwik8The PileWikiText-2LAMBADAOne Billion WordText8Penn Treebank (Character Level)Hutter PrizeOpenWebTextSALMon

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1Decay RNNValidation perplexity76.67—Unverified
2GRUValidation perplexity53.78—Unverified
3LSTMValidation perplexity52.73—Unverified
4LSTMTest perplexity48.7—Unverified
5Temporal CNNTest perplexity45.2—Unverified
6TCNTest perplexity45.19—Unverified
7GCNN-8Test perplexity44.9—Unverified
8Neural cache model (size = 100)Test perplexity44.8—Unverified
9Neural cache model (size = 2,000)Test perplexity40.8—Unverified
10GPT-2 SmallTest perplexity37.5—Unverified
#ModelMetricClaimedVerifiedStatus
1TCNTest perplexity108.47—Unverified
2Seq-U-NetTest perplexity107.95—Unverified
3GRU (Bai et al., 2018)Test perplexity92.48—Unverified
4R-TransformerTest perplexity84.38—Unverified
5Zaremba et al. (2014) - LSTM (medium)Test perplexity82.7—Unverified
6Gal & Ghahramani (2016) - Variational LSTM (medium)Test perplexity79.7—Unverified
7LSTM (Bai et al., 2018)Test perplexity78.93—Unverified
8Zaremba et al. (2014) - LSTM (large)Test perplexity78.4—Unverified
9Gal & Ghahramani (2016) - Variational LSTM (large)Test perplexity75.2—Unverified
10Inan et al. (2016) - Variational RHNTest perplexity66—Unverified
#ModelMetricClaimedVerifiedStatus
1LSTM (7 layers)Bit per Character (BPC)1.67—Unverified
2HypernetworksBit per Character (BPC)1.34—Unverified
3SHA-LSTM (4 layers, h=1024, no attention head)Bit per Character (BPC)1.33—Unverified
4LN HM-LSTMBit per Character (BPC)1.32—Unverified
5ByteNetBit per Character (BPC)1.31—Unverified
6Recurrent Highway NetworksBit per Character (BPC)1.27—Unverified
7Large FS-LSTM-4Bit per Character (BPC)1.25—Unverified
8Large mLSTMBit per Character (BPC)1.24—Unverified
9AWD-LSTM (3 layers)Bit per Character (BPC)1.23—Unverified
10Cluster-Former (#C=512)Bit per Character (BPC)1.22—Unverified
#ModelMetricClaimedVerifiedStatus
1Smaller Transformer 126M (pre-trained)Test perplexity33—Unverified
2OPT 125MTest perplexity32.26—Unverified
3Larger Transformer 771M (pre-trained)Test perplexity28.1—Unverified
4OPT 1.3BTest perplexity19.55—Unverified
5GPT-Neo 125MTest perplexity17.83—Unverified
6OPT 2.7BTest perplexity17.81—Unverified
7Smaller Transformer 126M (fine-tuned)Test perplexity12—Unverified
8GPT-Neo 1.3BTest perplexity11.46—Unverified
9Transformer 125MTest perplexity10.7—Unverified
10GPT-Neo 2.7BTest perplexity10.44—Unverified