SOTAVerified

Language Modelling

A language model is a model of natural language. Language models are useful for a variety of tasks, including speech recognition, machine translation, natural language generation (generating more human-like text), optical character recognition, route optimization, handwriting recognition, grammar induction, and information retrieval.

Large language models (LLMs), currently their most advanced form, are predominantly based on transformers trained on larger datasets (frequently using words scraped from the public internet). They have superseded recurrent neural network-based models, which had previously superseded the purely statistical models, such as word n-gram language model.

Source: Wikipedia

Papers

Showing 78517900 of 17610 papers

TitleStatusHype
Human Preferences for Constructive Interactions in Language Model Alignment0
Human Preferences in Large Language Model Latent Space: A Technical Analysis on the Reliability of Synthetic Data in Voting Outcome Prediction0
HumanVLM: Foundation for Human-Scene Vision-Language Model0
Humpty Dumpty: Controlling Word Meanings via Corpus Poisoning0
HUNYUANPROVER: A Scalable Data Synthesis Framework and Guided Tree Search for Automated Theorem Proving0
HW-TSC's Submission to the CCMT 2024 Machine Translation Tasks0
Hyacinth6B: A large language model for Traditional Chinese0
Hybrid Agents for Image Restoration0
Hybrid AI for Responsive Multi-Turn Online Conversations with Novel Dynamic Routing and Feedback Adaptation0
Hybrid Attention-based Encoder-decoder Model for Efficient Language Model Adaptation0
Hybrid Autoregressive Transducer (hat)0
Hybrid CTC-Attention based End-to-End Speech Recognition using Subword Units0
Hybrid Emoji-Based Masked Language Models for Zero-Shot Abusive Language Detection0
HybridGen: VLM-Guided Hybrid Planning for Scalable Data Generation of Imitation Learning0
Hybrid Morphological Segmentation for Phrase-Based Machine Translation0
HybriDNA: A Hybrid Transformer-Mamba2 Long-Range DNA Language Model0
Hybrid Offline-online Scheduling Method for Large Language Model Inference Optimization0
Unified Preference Optimization: Language Model Alignment Beyond the Preference Frontier0
Hybrid Retrieval and Multi-stage Text Ranking Solution at TREC 2022 Deep Learning Track0
Hybrid-RACA: Hybrid Retrieval-Augmented Composition Assistance for Real-time Text Prediction0
Hybrid Selection of Language Model Training Data Using Linguistic Information and Perplexity0
Hybrid-SQuAD: Hybrid Scholarly Question Answering Dataset0
Hybrid Student-Teacher Large Language Model Refinement for Cancer Toxicity Symptom Extraction0
Hybrid Transducer and Attention based Encoder-Decoder Modeling for Speech-to-Text Tasks0
HyCIR: Boosting Zero-Shot Composed Image Retrieval with Synthetic Labels0
HypER: Literature-grounded Hypothesis Generation and Distillation with Provenance0
HYPERmotion: Learning Hybrid Behavior Planning for Autonomous Loco-manipulation0
Hypernymization of named entity-rich captions for grounding-based multi-modal pretraining0
Hyperparameter Optimization for Large Language Model Instruction-Tuning0
HyperPELT: Unified Parameter-Efficient Language Model Tuning for Both Language and Vision-and-Language Tasks0
Hyperspherical Query Likelihood Models with Word Embeddings0
HyperTuning: Toward Adapting Large Language Models without Back-propagation0
Hypothesis-only Biases in Large Language Model-Elicited Natural Language Inference0
Hypothesis Testing for Quantifying LLM-Human Misalignment in Multiple Choice Settings0
I2MVFormer: Large Language Model Generated Multi-View Document Supervision for Zero-Shot Image Classification0
IAPT: Instruction-Aware Prompt Tuning for Large Language Models0
IBM MNLP IE at CASE 2021 Task 1: Multigranular and Multilingual Event Detection on Protest News0
I Can Has Cheezburger? A Nonparanormal Approach to Combining Textual and Visual Information for Predicting and Generating Popular Meme Descriptions0
ICCV23 Visual-Dialog Emotion Explanation Challenge: SEU_309 Team Technical Report0
ICE-SEARCH: A Language Model-Driven Feature Selection Approach0
ICH-Qwen: A Large Language Model Towards Chinese Intangible Cultural Heritage0
iCompass at SemEval-2020 Task 12: From a Syntax-ignorant N-gram Embeddings Model to a Deep Bidirectional Language Model0
ICONS: Influence Consensus for Vision-Language Data Selection0
ID-Agnostic User Behavior Pre-training for Sequential Recommendation0
ID-centric Pre-training for Recommendation0
IDEAL: Data Equilibrium Adaptation for Multi-Capability Language Model Alignment0
IDEA Prune: An Integrated Enlarge-and-Prune Pipeline in Generative Language Model Pretraining0
Identification de profil clinique du patient: Une approche de classification de séquences utilisant des modèles de langage français contextualisés (Identification of patient clinical profiles : A sequence classification approach using contextualised French language models )0
Identification of Enzymatic Active Sites with Unsupervised Language Modeling0
Identifying Adversarial Attacks on Text Classifiers0
Show:102550
← PrevPage 158 of 353Next →

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1Decay RNNValidation perplexity76.67Unverified
2GRUValidation perplexity53.78Unverified
3LSTMValidation perplexity52.73Unverified
4LSTMTest perplexity48.7Unverified
5Temporal CNNTest perplexity45.2Unverified
6TCNTest perplexity45.19Unverified
7GCNN-8Test perplexity44.9Unverified
8Neural cache model (size = 100)Test perplexity44.8Unverified
9Neural cache model (size = 2,000)Test perplexity40.8Unverified
10GPT-2 SmallTest perplexity37.5Unverified
#ModelMetricClaimedVerifiedStatus
1TCNTest perplexity108.47Unverified
2Seq-U-NetTest perplexity107.95Unverified
3GRU (Bai et al., 2018)Test perplexity92.48Unverified
4R-TransformerTest perplexity84.38Unverified
5Zaremba et al. (2014) - LSTM (medium)Test perplexity82.7Unverified
6Gal & Ghahramani (2016) - Variational LSTM (medium)Test perplexity79.7Unverified
7LSTM (Bai et al., 2018)Test perplexity78.93Unverified
8Zaremba et al. (2014) - LSTM (large)Test perplexity78.4Unverified
9Gal & Ghahramani (2016) - Variational LSTM (large)Test perplexity75.2Unverified
10Inan et al. (2016) - Variational RHNTest perplexity66Unverified
#ModelMetricClaimedVerifiedStatus
1LSTM (7 layers)Bit per Character (BPC)1.67Unverified
2HypernetworksBit per Character (BPC)1.34Unverified
3SHA-LSTM (4 layers, h=1024, no attention head)Bit per Character (BPC)1.33Unverified
4LN HM-LSTMBit per Character (BPC)1.32Unverified
5ByteNetBit per Character (BPC)1.31Unverified
6Recurrent Highway NetworksBit per Character (BPC)1.27Unverified
7Large FS-LSTM-4Bit per Character (BPC)1.25Unverified
8Large mLSTMBit per Character (BPC)1.24Unverified
9AWD-LSTM (3 layers)Bit per Character (BPC)1.23Unverified
10Cluster-Former (#C=512)Bit per Character (BPC)1.22Unverified
#ModelMetricClaimedVerifiedStatus
1Smaller Transformer 126M (pre-trained)Test perplexity33Unverified
2OPT 125MTest perplexity32.26Unverified
3Larger Transformer 771M (pre-trained)Test perplexity28.1Unverified
4OPT 1.3BTest perplexity19.55Unverified
5GPT-Neo 125MTest perplexity17.83Unverified
6OPT 2.7BTest perplexity17.81Unverified
7Smaller Transformer 126M (fine-tuned)Test perplexity12Unverified
8GPT-Neo 1.3BTest perplexity11.46Unverified
9Transformer 125MTest perplexity10.7Unverified
10GPT-Neo 2.7BTest perplexity10.44Unverified