SOTAVerified

Language Modelling

A language model is a model of natural language. Language models are useful for a variety of tasks, including speech recognition, machine translation, natural language generation (generating more human-like text), optical character recognition, route optimization, handwriting recognition, grammar induction, and information retrieval.

Large language models (LLMs), currently their most advanced form, are predominantly based on transformers trained on larger datasets (frequently using words scraped from the public internet). They have superseded recurrent neural network-based models, which had previously superseded the purely statistical models, such as word n-gram language model.

Source: Wikipedia

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 8551–8600 of 17610 papers

TitleStatusHype
ED-SAM: An Efficient Diffusion Sampling Approach to Domain Generalization in Vision-Language Foundation Models—0
會議語音辨識使用語者資訊之語言模型調適技術 (On the Use of Speaker-Aware Language Model Adaptation Techniques for Meeting Speech Recognition ) [In Chinese]—0
EEG2TEXT-CN: An Exploratory Study of Open-Vocabulary Chinese Text-EEG Alignment via Large Language Model and Contrastive Learning on ChineseEEG—0
EEG-Language Modeling for Pathology Detection—0
EE-MLLM: A Data-Efficient and Compute-Efficient Multimodal Large Language Model—0
Effect and Analysis of Large-scale Language Model Rescoring on Competitive ASR Systems—0
Effective Black-Box Multi-Faceted Attacks Breach Vision Large Language Model Guardrails—0
Effective Decoder Masking for Transformer Based End-to-End Speech Recognition—0
Effective faking of verbal deception detection with target-aligned adversarial attacks—0
Effective FAQ Retrieval and Question Matching With Unsupervised Knowledge Injection—0
Effective Fine-Tuning Methods for Cross-lingual Adaptation—0
Effective internal language model training and fusion for factorized transducer model—0
Effective Large Language Model Adaptation for Improved Grounding and Citation Generation—0
Effective Large Language Model Debugging with Best-first Tree Search—0
Effectively Prompting Small-sized Language Models for Cross-lingual Tasks via Winning Tickets—0
Effectiveness of Character Language Model for Vietnamese Named Entity Recognition—0
Effectiveness of Deep Networks in NLP using BiDAF as an example architecture—0
Effective SAM Combination for Open-Vocabulary Semantic Segmentation—0
Effective Selection of Translation Model Training Data—0
Effective Sentence Scoring Method using Bidirectional Language Model for Speech Recognition—0
Effective Text Adaptation for LLM-based ASR through Soft Prompt Fine-Tuning—0
Effect of Language and Error Models on Efficiency of Finite-State Spell-Checking and Correction—0
Effect of Selection Format on LLM Performance—0
Effects of Communicative Pressures on Novice L2 Learners' Use of Optional Formal Devices—0
Effects of Number of Filters of Convolutional Layers on Speech Recognition Model Accuracy—0
Effects of Stop Words Elimination for Arabic Information Retrieval: A Comparative Study—0
Efficient and Comprehensive Feature Extraction in Large Vision-Language Model for Pathology Analysis—0
Efficient and Context-Aware Label Propagation for Zero-/Few-Shot Training-Free Adaptation of Vision-Language Model—0
Efficient and Direct Duplex Modeling for Speech-to-Speech Language Model—0
Efficient Heterogeneous Large Language Model Decoding with Model-Attention Disaggregation—0
Efficient and effective training of language and graph neural network models—0
Efficient and Interpretable Neural Models for Entity Tracking—0
Efficient and Reliable Overlay Networks for Decentralized Federated Learning—0
Efficient Avoidance of Vulnerabilities in Auto-completed Smart Contract Code Using Vulnerability-constrained Decoding—0
EfficientCLIP: Efficient Cross-Modal Pre-training by Ensemble Confident Learning and Language Modeling—0
Efficient Contextual Representation Learning Without Softmax Layer—0
Efficient Contextual Representation Learning With Continuous Outputs—0
Efficient Distributed Retrieval-Augmented Generation for Enhancing Language Model Performance—0
Efficient Domain Adaptation of Language Models via Adaptive Tokenization—0
Efficient Domain-adaptive Continual Pretraining for the Process Industry in the German Language—0
Efficient Dynamic WFST Decoding for Personalized Language Models—0
Efficient Fine-Tuning of Large Language Models for Automated Medical Documentation—0
Self-Supervised Contrastive Learning for Code Retrieval and Summarization via Semantic-Preserving Transformations—0
Efficient GPT Model Pre-training using Tensor Train Matrix Representation—0
Efficient Hierarchical Domain Adaptation for Pretrained Language Models—0
Efficient Human-AI Coordination via Preparatory Language-based Convention—0
Efficient Hybrid Language Model Compression through Group-Aware SSM Pruning—0
Efficient Knowledge Distillation via Curriculum Extraction—0
Efficient Language Model Architectures for Differentially Private Federated Learning—0
Efficient Language Modeling with Automatic Relevance Determination in Recurrent Neural Networks—0
Show:102550
← PrevPage 172 of 353Next →
All datasetsWikiText-103Penn Treebank (Word Level)enwik8The PileWikiText-2LAMBADAOne Billion WordText8Penn Treebank (Character Level)Hutter PrizeOpenWebTextSALMon

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1Decay RNNValidation perplexity76.67—Unverified
2GRUValidation perplexity53.78—Unverified
3LSTMValidation perplexity52.73—Unverified
4LSTMTest perplexity48.7—Unverified
5Temporal CNNTest perplexity45.2—Unverified
6TCNTest perplexity45.19—Unverified
7GCNN-8Test perplexity44.9—Unverified
8Neural cache model (size = 100)Test perplexity44.8—Unverified
9Neural cache model (size = 2,000)Test perplexity40.8—Unverified
10GPT-2 SmallTest perplexity37.5—Unverified
#ModelMetricClaimedVerifiedStatus
1TCNTest perplexity108.47—Unverified
2Seq-U-NetTest perplexity107.95—Unverified
3GRU (Bai et al., 2018)Test perplexity92.48—Unverified
4R-TransformerTest perplexity84.38—Unverified
5Zaremba et al. (2014) - LSTM (medium)Test perplexity82.7—Unverified
6Gal & Ghahramani (2016) - Variational LSTM (medium)Test perplexity79.7—Unverified
7LSTM (Bai et al., 2018)Test perplexity78.93—Unverified
8Zaremba et al. (2014) - LSTM (large)Test perplexity78.4—Unverified
9Gal & Ghahramani (2016) - Variational LSTM (large)Test perplexity75.2—Unverified
10Inan et al. (2016) - Variational RHNTest perplexity66—Unverified
#ModelMetricClaimedVerifiedStatus
1LSTM (7 layers)Bit per Character (BPC)1.67—Unverified
2HypernetworksBit per Character (BPC)1.34—Unverified
3SHA-LSTM (4 layers, h=1024, no attention head)Bit per Character (BPC)1.33—Unverified
4LN HM-LSTMBit per Character (BPC)1.32—Unverified
5ByteNetBit per Character (BPC)1.31—Unverified
6Recurrent Highway NetworksBit per Character (BPC)1.27—Unverified
7Large FS-LSTM-4Bit per Character (BPC)1.25—Unverified
8Large mLSTMBit per Character (BPC)1.24—Unverified
9AWD-LSTM (3 layers)Bit per Character (BPC)1.23—Unverified
10Cluster-Former (#C=512)Bit per Character (BPC)1.22—Unverified
#ModelMetricClaimedVerifiedStatus
1Smaller Transformer 126M (pre-trained)Test perplexity33—Unverified
2OPT 125MTest perplexity32.26—Unverified
3Larger Transformer 771M (pre-trained)Test perplexity28.1—Unverified
4OPT 1.3BTest perplexity19.55—Unverified
5GPT-Neo 125MTest perplexity17.83—Unverified
6OPT 2.7BTest perplexity17.81—Unverified
7Smaller Transformer 126M (fine-tuned)Test perplexity12—Unverified
8GPT-Neo 1.3BTest perplexity11.46—Unverified
9Transformer 125MTest perplexity10.7—Unverified
10GPT-Neo 2.7BTest perplexity10.44—Unverified