SOTAVerified

Language Modelling

A language model is a model of natural language. Language models are useful for a variety of tasks, including speech recognition, machine translation, natural language generation (generating more human-like text), optical character recognition, route optimization, handwriting recognition, grammar induction, and information retrieval.

Large language models (LLMs), currently their most advanced form, are predominantly based on transformers trained on larger datasets (frequently using words scraped from the public internet). They have superseded recurrent neural network-based models, which had previously superseded the purely statistical models, such as word n-gram language model.

Source: Wikipedia

Papers

Showing 1575115800 of 17610 papers

TitleStatusHype
Show and Guide: Instructional-Plan Grounded Vision and Language ModelCode0
On-the-Fly Aligned Data Augmentation for Sequence-to-Sequence ASRCode0
Learn Continually, Generalize Rapidly: Lifelong Knowledge Accumulation for Few-shot LearningCode0
LICHEE: Improving Language Model Pre-training with Multi-grained TokenizationCode0
On The Evaluation of Machine Translation Systems Trained With Back-TranslationCode0
Libra-Merging: Importance-redundancy and Pruning-merging Trade-off for Acceleration Plug-in in Large Vision-Language ModelCode0
The World of an Octopus: How Reporting Bias Influences a Language Model's Perception of ColorCode0
On the End-to-End Solution to Mandarin-English Code-switching Speech RecognitionCode0
Masked Language Models are Good Heterogeneous Graph GeneralizersCode0
Siamese-DETR for Generic Multi-Object TrackingCode0
On the Encoder-Decoder Incompatibility in Variational Text Modeling and BeyondCode0
On the Effect of (Near) Duplicate Subwords in Language ModellingCode0
On the Cross-lingual Transferability of Monolingual RepresentationsCode0
TAPER: Time-Aware Patient EHR RepresentationCode0
Masked Language Model Based Textual Adversarial Example DetectionCode0
On the Stability of a non-hyperbolic nonlinear map with non-bounded set of non-isolated fixed points with applications to Machine LearningCode0
Sig2text, a Vision-language model for Non-cooperative Radar Signal ParsingCode0
Sig-Networks Toolkit: Signature Networks for Longitudinal Language ModellingCode0
The World of an Octopus: How Reporting Bias Influences a Language Model’s Perception of ColorCode0
Masked Generative Story Transformer with Character Guidance and Caption AugmentationCode0
Masked Diffusion with Task-awareness for Procedure Planning in Instructional VideosCode0
Kyoto University Participation to WAT 2017Code0
SILC-EFSA: Self-aware In-context Learning Correction for Entity-level Financial Sentiment AnalysisCode0
On the Complementary Nature of Knowledge Graph Embedding, Fine Grain Entity Types, and Language ModelingCode0
On the adequacy of untuned warmup for adaptive optimizationCode0
LIBRA: Measuring Bias of Large Language Model from a Local ContextCode0
The Z-loss: a shift and scale invariant classification loss belonging to the Spherical FamilyCode0
SimCPSR: Simple Contrastive Learning for Paper Submission Recommendation SystemCode0
Language Modelling for Sound Event Detection with Teacher Forcing and Scheduled SamplingCode0
Thieves on Sesame Street! Model Extraction of BERT-based APIsCode0
Think Again Networks and the Delta LossCode0
LG-CAV: Train Any Concept Activation Vector with Language GuidanceCode0
Targeted Syntactic Evaluation of Language ModelsCode0
On Robustness of Finetuned Transformer-based NLP ModelsCode0
Towards Ontology-Enhanced Representation Learning for Large Language ModelsCode0
On Recovering Higher-order Interactions from Protein Language ModelsCode0
On Monotonic Aggregation for Open-domain QACode0
OnlySportsLM: Optimizing Sports-Domain Language Models with SOTA Performance under Billion ParametersCode0
TruthEval: A Dataset to Evaluate LLM Truthfulness and ReliabilityCode0
Learn from Failure: Fine-Tuning LLMs with Trial-and-Error Data for Intuitionistic Propositional Logic ProvingCode0
Target-specified Sequence Labeling with Multi-head Self-attention for Target-oriented Opinion Words ExtractionCode0
Masked and Permuted Implicit Context Learning for Scene Text RecognitionCode0
TransFool: An Adversarial Attack against Neural Machine Translation ModelsCode0
Learn from Downstream and Be Yourself in Multimodal Large Language Model Fine-TuningCode0
Thinking Outside of the Differential Privacy Box: A Case Study in Text Privatization with Language Model PromptingCode0
Simple Fusion: Return of the Language ModelCode0
Online Spatial Concept and Lexical Acquisition with Simultaneous Localization and MappingCode0
Language Model Knowledge Distillation for Efficient Question Answering in SpanishCode0
Kyoto-NMT: a Neural Machine Translation implementation in ChainerCode0
Online Normalization for Training Neural NetworksCode0
Show:102550
← PrevPage 316 of 353Next →

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1Decay RNNValidation perplexity76.67Unverified
2GRUValidation perplexity53.78Unverified
3LSTMValidation perplexity52.73Unverified
4LSTMTest perplexity48.7Unverified
5Temporal CNNTest perplexity45.2Unverified
6TCNTest perplexity45.19Unverified
7GCNN-8Test perplexity44.9Unverified
8Neural cache model (size = 100)Test perplexity44.8Unverified
9Neural cache model (size = 2,000)Test perplexity40.8Unverified
10GPT-2 SmallTest perplexity37.5Unverified
#ModelMetricClaimedVerifiedStatus
1TCNTest perplexity108.47Unverified
2Seq-U-NetTest perplexity107.95Unverified
3GRU (Bai et al., 2018)Test perplexity92.48Unverified
4R-TransformerTest perplexity84.38Unverified
5Zaremba et al. (2014) - LSTM (medium)Test perplexity82.7Unverified
6Gal & Ghahramani (2016) - Variational LSTM (medium)Test perplexity79.7Unverified
7LSTM (Bai et al., 2018)Test perplexity78.93Unverified
8Zaremba et al. (2014) - LSTM (large)Test perplexity78.4Unverified
9Gal & Ghahramani (2016) - Variational LSTM (large)Test perplexity75.2Unverified
10Inan et al. (2016) - Variational RHNTest perplexity66Unverified
#ModelMetricClaimedVerifiedStatus
1LSTM (7 layers)Bit per Character (BPC)1.67Unverified
2HypernetworksBit per Character (BPC)1.34Unverified
3SHA-LSTM (4 layers, h=1024, no attention head)Bit per Character (BPC)1.33Unverified
4LN HM-LSTMBit per Character (BPC)1.32Unverified
5ByteNetBit per Character (BPC)1.31Unverified
6Recurrent Highway NetworksBit per Character (BPC)1.27Unverified
7Large FS-LSTM-4Bit per Character (BPC)1.25Unverified
8Large mLSTMBit per Character (BPC)1.24Unverified
9AWD-LSTM (3 layers)Bit per Character (BPC)1.23Unverified
10Cluster-Former (#C=512)Bit per Character (BPC)1.22Unverified
#ModelMetricClaimedVerifiedStatus
1Smaller Transformer 126M (pre-trained)Test perplexity33Unverified
2OPT 125MTest perplexity32.26Unverified
3Larger Transformer 771M (pre-trained)Test perplexity28.1Unverified
4OPT 1.3BTest perplexity19.55Unverified
5GPT-Neo 125MTest perplexity17.83Unverified
6OPT 2.7BTest perplexity17.81Unverified
7Smaller Transformer 126M (fine-tuned)Test perplexity12Unverified
8GPT-Neo 1.3BTest perplexity11.46Unverified
9Transformer 125MTest perplexity10.7Unverified
10GPT-Neo 2.7BTest perplexity10.44Unverified