SOTAVerified

Language Modelling

A language model is a model of natural language. Language models are useful for a variety of tasks, including speech recognition, machine translation, natural language generation (generating more human-like text), optical character recognition, route optimization, handwriting recognition, grammar induction, and information retrieval.

Large language models (LLMs), currently their most advanced form, are predominantly based on transformers trained on larger datasets (frequently using words scraped from the public internet). They have superseded recurrent neural network-based models, which had previously superseded the purely statistical models, such as word n-gram language model.

Source: Wikipedia

Papers

Showing 1015110200 of 17610 papers

TitleStatusHype
Multi-Fidelity Methods for Optimization: A Survey0
Multi-Grained Chinese Word Segmentation0
Multi-Granularity Prediction with Learnable Fusion for Scene Text Recognition0
Multi-Granularity Semantic Revision for Large Language Model Distillation0
Multi-Graph Decoding for Code-Switching ASR0
Multi-group Uncertainty Quantification for Long-form Text Generation0
Multi-Head State Space Model for Speech Recognition0
Multi-label natural language processing to identify diagnosis and procedure codes from MIMIC-III inpatient notes0
Multi-level Distillation of Semantic Knowledge for Pre-training Multilingual Language Model0
Multi-Level Modeling Units for End-to-End Mandarin Speech Recognition0
Multilevel Text Normalization with Sequence-to-Sequence Networks and Multisource Learning0
Multilingual and Fully Non-Autoregressive ASR with Large Language Model Fusion: A Comprehensive Study0
Multilingual and Multimodal Learning for Brazilian Portuguese0
Multilingual BERT has an accent: Evaluating English influences on fluency in multilingual models0
Multilingual BERT Post-Pretraining Alignment0
Neural Approaches to Multilingual Information Retrieval0
Multilingual Comparative Analysis of Deep-Learning Dependency Parsing Results Using Parallel Corpora0
Multilingual End-to-End Speech Recognition with A Single Transformer on Low-Resource Languages0
Multilingual Few-Shot Learning via Language Model Retrieval0
Multilingual Language Model Pretraining using Machine-translated Data0
Multilingual Language Models for Named Entity Recognition in German and English0
Multilingual Large Language Model: A Survey of Resources, Taxonomy and Frontiers0
Multilingual prediction of Alzheimer's disease through domain adaptation and concept-based language modelling0
Multilingual Pre-training with Universal Dependency Learning0
Multilingual Reliability and ``Semantic'' Structure of Continuous Word Spaces0
Multilingual self-supervised speech representations improve the speech recognition of low-resource African languages with codeswitching0
Multilingual Sequence Labeling Approach to solve Lexical Normalization0
Multilingual sequence-to-sequence speech recognition: architecture, transfer learning, and language modeling0
Multilingual Slavic Named Entity Recognition0
Multilingual Speech Recognition With A Single End-To-End Model0
Multilingual Substitution-based Word Sense Induction0
Multilingual Syntax-aware Language Modeling through Dependency Tree Conversion0
Multilingual Syntax-aware Language Modeling through Dependency Tree Conversion0
Multilingual Test Sets for Machine Translation of Search Queries for Cross-Lingual Information Retrieval in the Medical Domain0
Multilingual Tourist Assistance using ChatGPT: Comparing Capabilities in Hindi, Telugu, and Kannada0
Multilingual Transfer Learning for Code-Switched Language and Speech Neural Modeling0
Multilingual Transformer Language Model for Speech Recognition in Low-resource Languages0
Multilingual unsupervised sequence segmentation transfers to extremely low-resource languages0
Multi-lingual Wikipedia Summarization and Title Generation On Low Resource Corpus0
Multilingual Zero Resource Speech Recognition Base on Self-Supervise Pre-Trained Acoustic Models0
Multi-megabase scale genome interpretation with genetic language models0
Multi-modal Agent Tuning: Building a VLM-Driven Agent for Efficient Tool Usage0
Multi-modal Causal Structure Learning and Root Cause Analysis0
Multimodal Classification of Teaching Activities from University Lecture Recordings0
Multi-modal clothing recommendation model based on large model and VAE enhancement0
Multimodal Comparable Corpora as Resources for Extracting Parallel Data: Parallel Phrases Extraction0
Multimodal Conditionality for Natural Language Generation0
The Context of Crash Occurrence: A Complexity-Infused Approach Integrating Semantic, Contextual, and Kinematic Features0
Multimodal Crop Type Classification Fusing Multi-Spectral Satellite Time Series with Farmers Crop Rotations and Local Crop Distribution0
Multimodal Data and Resource Efficient Device-Directed Speech Detection with Large Foundation Models0
Show:102550
← PrevPage 204 of 353Next →

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1Decay RNNValidation perplexity76.67Unverified
2GRUValidation perplexity53.78Unverified
3LSTMValidation perplexity52.73Unverified
4LSTMTest perplexity48.7Unverified
5Temporal CNNTest perplexity45.2Unverified
6TCNTest perplexity45.19Unverified
7GCNN-8Test perplexity44.9Unverified
8Neural cache model (size = 100)Test perplexity44.8Unverified
9Neural cache model (size = 2,000)Test perplexity40.8Unverified
10GPT-2 SmallTest perplexity37.5Unverified
#ModelMetricClaimedVerifiedStatus
1TCNTest perplexity108.47Unverified
2Seq-U-NetTest perplexity107.95Unverified
3GRU (Bai et al., 2018)Test perplexity92.48Unverified
4R-TransformerTest perplexity84.38Unverified
5Zaremba et al. (2014) - LSTM (medium)Test perplexity82.7Unverified
6Gal & Ghahramani (2016) - Variational LSTM (medium)Test perplexity79.7Unverified
7LSTM (Bai et al., 2018)Test perplexity78.93Unverified
8Zaremba et al. (2014) - LSTM (large)Test perplexity78.4Unverified
9Gal & Ghahramani (2016) - Variational LSTM (large)Test perplexity75.2Unverified
10Inan et al. (2016) - Variational RHNTest perplexity66Unverified
#ModelMetricClaimedVerifiedStatus
1LSTM (7 layers)Bit per Character (BPC)1.67Unverified
2HypernetworksBit per Character (BPC)1.34Unverified
3SHA-LSTM (4 layers, h=1024, no attention head)Bit per Character (BPC)1.33Unverified
4LN HM-LSTMBit per Character (BPC)1.32Unverified
5ByteNetBit per Character (BPC)1.31Unverified
6Recurrent Highway NetworksBit per Character (BPC)1.27Unverified
7Large FS-LSTM-4Bit per Character (BPC)1.25Unverified
8Large mLSTMBit per Character (BPC)1.24Unverified
9AWD-LSTM (3 layers)Bit per Character (BPC)1.23Unverified
10Cluster-Former (#C=512)Bit per Character (BPC)1.22Unverified
#ModelMetricClaimedVerifiedStatus
1Smaller Transformer 126M (pre-trained)Test perplexity33Unverified
2OPT 125MTest perplexity32.26Unverified
3Larger Transformer 771M (pre-trained)Test perplexity28.1Unverified
4OPT 1.3BTest perplexity19.55Unverified
5GPT-Neo 125MTest perplexity17.83Unverified
6OPT 2.7BTest perplexity17.81Unverified
7Smaller Transformer 126M (fine-tuned)Test perplexity12Unverified
8GPT-Neo 1.3BTest perplexity11.46Unverified
9Transformer 125MTest perplexity10.7Unverified
10GPT-Neo 2.7BTest perplexity10.44Unverified