SOTAVerified

Language Modelling

A language model is a model of natural language. Language models are useful for a variety of tasks, including speech recognition, machine translation, natural language generation (generating more human-like text), optical character recognition, route optimization, handwriting recognition, grammar induction, and information retrieval.

Large language models (LLMs), currently their most advanced form, are predominantly based on transformers trained on larger datasets (frequently using words scraped from the public internet). They have superseded recurrent neural network-based models, which had previously superseded the purely statistical models, such as word n-gram language model.

Source: Wikipedia

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 8201–8250 of 17610 papers

TitleStatusHype
Information Density as a Factor for Variation in the Embedding of Relative Clauses—0
Information Extraction from Historical Well Records Using A Large Language Model—0
Information Flow Control in Machine Learning through Modular Model Architecture—0
Information fusion strategy integrating pre-trained language model and contrastive learning for materials knowledge mining—0
Information Suppression in Large Language Models: Auditing, Quantifying, and Characterizing Censorship in DeepSeek—0
Infrared and Visible Image Fusion with Hierarchical Human Perception—0
Infrrd.ai at SemEval-2022 Task 11: A system for named entity recognition using data augmentation, transformer-based sequence labeling model, and EnsembleCRF—0
Infusing Future Information into Monotonic Attention Through Language Models—0
In Generative AI we Trust: Can Chatbots Effectively Verify Political Information?—0
Using an LLM to Help With Code Understanding—0
Initial Decoding with Minimally Augmented Language Model for Improved Lattice Rescoring in Low Resource ASR—0
Injecting Text in Self-Supervised Speech Pretraining—0
InkubaLM: A small language model for low-resource African languages—0
Input Seed Features for Guiding the Generation Process: A Statistical Approach for Spanish—0
Input-Tuning: Adapting Unfamiliar Inputs to Frozen Pretrained Models—0
INSIGHT: Enhancing Autonomous Driving Safety through Vision-Language Models on Context-Aware Hazard Detection and Edge Case Evaluation—0
Inspire creativity with ORIBA: Transform Artists' Original Characters into Chatbots through Large Language Model—0
Inspire the Large Language Model by External Knowledge on BioMedical Named Entity Recognition—0
Instance Selection Improves Cross-Lingual Model Training for Fine-Grained Sentiment Analysis—0
Institutional Platform for Secure Self-Service Large Language Model Exploration—0
InstructEval: Systematic Evaluation of Instruction Selection Methods—0
Instruction Diversity Drives Generalization To Unseen Tasks—0
Instruction-Driven Game Engine: A Poker Case Study—0
Instruction Mining: Instruction Data Selection for Tuning Large Language Models—0
InstructMPC: A Human-LLM-in-the-Loop Framework for Context-Aware Control—0
InstructP2P: Learning to Edit 3D Point Clouds with Text Instructions—0
InstructPipe: Generating Visual Blocks Pipelines with Human Instructions and LLMs—0
Integrating an Unsupervised Transliteration Model into Statistical Machine Translation—0
Integrating Dictionaries into an Unsupervised Model for Myanmar Word Segmentation—0
Integrating Dictionary and Web N-grams for Chinese Spell Checking—0
Integrating Domain Knowledge into Large Language Models for Enhanced Fashion Recommendations—0
Integrating Encyclopedic Knowledge into Neural Language Models—0
Integrating Knowledge in End-to-End Automatic Speech Recognition for Mandarin-English Code-Switching—0
Integrating Language and Vision to Generate Natural Language Descriptions of Videos in the Wild—0
Integrating Language Models into Direct Speech Translation: An Inference-Time Solution to Control Gender Inflection—0
Enhancing Traffic Incident Management with Large Language Models: A Hybrid Machine Learning Approach for Severity Classification—0
Integrating LLMs for Explainable Fault Diagnosis in Complex Systems—0
Integrating Morphological Desegmentation into Phrase-based Decoding—0
Integrating morpho-syntactic features in English-Arabic statistical machine translation—0
Integrating Object Detection Modality into Visual Language Model for Enhanced Autonomous Driving Agent—0
Integrating Optical Character Recognition and Machine Translation of Historical Documents—0
Integrating Order Information and Event Relation for Script Event Prediction—0
Integrating Pause Information with Word Embeddings in Language Models for Alzheimer's Disease Detection from Spontaneous Speech—0
Integrating Phrase-based Reordering Features into a Chart-based Decoder for Machine Translation—0
Integrating Pretrained Language Model for Dialogue Policy Learning—0
Integrating Pre-trained Language Model into Neural Machine Translation—0
Integrating Protein Sequence and Expression Level to Analysis Molecular Characterization of Breast Cancer Subtypes—0
Integrating remote sensing data assimilation, deep learning and large language model for interactive wheat breeding yield prediction—0
Integrating Summarization and Retrieval for Enhanced Personalization via Large Language Models—0
Integrating Text-to-Music Models with Language Models: Composing Long Structured Music Pieces—0
Show:102550
← PrevPage 165 of 353Next →
All datasetsWikiText-103Penn Treebank (Word Level)enwik8The PileWikiText-2LAMBADAOne Billion WordText8Penn Treebank (Character Level)Hutter PrizeOpenWebTextSALMon

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1Decay RNNValidation perplexity76.67—Unverified
2GRUValidation perplexity53.78—Unverified
3LSTMValidation perplexity52.73—Unverified
4LSTMTest perplexity48.7—Unverified
5Temporal CNNTest perplexity45.2—Unverified
6TCNTest perplexity45.19—Unverified
7GCNN-8Test perplexity44.9—Unverified
8Neural cache model (size = 100)Test perplexity44.8—Unverified
9Neural cache model (size = 2,000)Test perplexity40.8—Unverified
10GPT-2 SmallTest perplexity37.5—Unverified
#ModelMetricClaimedVerifiedStatus
1TCNTest perplexity108.47—Unverified
2Seq-U-NetTest perplexity107.95—Unverified
3GRU (Bai et al., 2018)Test perplexity92.48—Unverified
4R-TransformerTest perplexity84.38—Unverified
5Zaremba et al. (2014) - LSTM (medium)Test perplexity82.7—Unverified
6Gal & Ghahramani (2016) - Variational LSTM (medium)Test perplexity79.7—Unverified
7LSTM (Bai et al., 2018)Test perplexity78.93—Unverified
8Zaremba et al. (2014) - LSTM (large)Test perplexity78.4—Unverified
9Gal & Ghahramani (2016) - Variational LSTM (large)Test perplexity75.2—Unverified
10Inan et al. (2016) - Variational RHNTest perplexity66—Unverified
#ModelMetricClaimedVerifiedStatus
1LSTM (7 layers)Bit per Character (BPC)1.67—Unverified
2HypernetworksBit per Character (BPC)1.34—Unverified
3SHA-LSTM (4 layers, h=1024, no attention head)Bit per Character (BPC)1.33—Unverified
4LN HM-LSTMBit per Character (BPC)1.32—Unverified
5ByteNetBit per Character (BPC)1.31—Unverified
6Recurrent Highway NetworksBit per Character (BPC)1.27—Unverified
7Large FS-LSTM-4Bit per Character (BPC)1.25—Unverified
8Large mLSTMBit per Character (BPC)1.24—Unverified
9AWD-LSTM (3 layers)Bit per Character (BPC)1.23—Unverified
10Cluster-Former (#C=512)Bit per Character (BPC)1.22—Unverified
#ModelMetricClaimedVerifiedStatus
1Smaller Transformer 126M (pre-trained)Test perplexity33—Unverified
2OPT 125MTest perplexity32.26—Unverified
3Larger Transformer 771M (pre-trained)Test perplexity28.1—Unverified
4OPT 1.3BTest perplexity19.55—Unverified
5GPT-Neo 125MTest perplexity17.83—Unverified
6OPT 2.7BTest perplexity17.81—Unverified
7Smaller Transformer 126M (fine-tuned)Test perplexity12—Unverified
8GPT-Neo 1.3BTest perplexity11.46—Unverified
9Transformer 125MTest perplexity10.7—Unverified
10GPT-Neo 2.7BTest perplexity10.44—Unverified