SOTAVerified

Language Modelling

A language model is a model of natural language. Language models are useful for a variety of tasks, including speech recognition, machine translation, natural language generation (generating more human-like text), optical character recognition, route optimization, handwriting recognition, grammar induction, and information retrieval.

Large language models (LLMs), currently their most advanced form, are predominantly based on transformers trained on larger datasets (frequently using words scraped from the public internet). They have superseded recurrent neural network-based models, which had previously superseded the purely statistical models, such as word n-gram language model.

Source: Wikipedia

Papers

Showing 1745117500 of 17610 papers

TitleStatusHype
Adaptively Truncating Backpropagation Through Time to Control Gradient BiasCode0
Invalidator: Automated Patch Correctness Assessment via Semantic and Syntactic ReasoningCode0
GPoeT-2: A GPT-2 Based Poem GeneratorCode0
Is Conversational XAI All You Need? Human-AI Decision Making With a Conversational XAI AssistantCode0
Private Memorization Editing: Turning Memorization into a Defense to Strengthen Data Privacy in Large Language ModelsCode0
Extreme Classification in Log Memory using Count-Min Sketch: A Case Study of Amazon Search with 50M ProductsCode0
A Japanese Language Model and Three New Evaluation Benchmarks for Pharmaceutical NLPCode0
GPT-3 Models are Poor Few-Shot Learners in the Biomedical DomainCode0
A Semi-Supervised Approach for Low-Resourced Text GenerationCode0
F5C-finder: An Explainable and Ensemble Biological Language Model for Predicting 5-Formylcytidine Modifications on mRNACode0
FaaF: Facts as a Function for the evaluation of generated textCode0
Adaptive Classifier-Free Guidance via Dynamic Low-Confidence MaskingCode0
FABLE: A Novel Data-Flow Analysis Benchmark on Procedural Text for Large Language Model EvaluationCode0
An Empirical Comparison of Generative Approaches for Product Attribute-Value IdentificationCode0
An Empirical Analysis of Uncertainty in Large Language Model EvaluationsCode0
An Embedded Deep Learning based Word PredictionCode0
Automatic Creation of Text Corpora for Low-Resource Languages from the Internet: The Case of Swiss GermanCode0
GPT4SGG: Synthesizing Scene Graphs from Holistic and Region-specific NarrativesCode0
Computational Assessment of Hyperpartisanship in News TitlesCode0
Automatic Classification of News Subjects in Broadcast News: Application to a Gender Bias Representation AnalysisCode0
An Embarrassingly Simple Approach for Transfer Learning from Pretrained Language ModelsCode0
An Effective Deployment of Diffusion LM for Data Augmentation in Low-Resource Sentiment ClassificationCode0
An EcoSage Assistant: Towards Building A Multimodal Plant Care Dialogue AssistantCode0
Spurious Feature Eraser: Stabilizing Test-Time Adaptation for Vision-Language Foundation ModelCode0
A Hybrid System for Systematic Generalization in Simple Arithmetic ProblemsCode0
Anchor Points: Benchmarking Models with Much Fewer ExamplesCode0
Automatic benchmarking of large multimodal models via iterative experiment programmingCode0
Compressing Sentence Representation for Semantic Retrieval via Homomorphic Projective DistillationCode0
Compressing Neural Language Models by Sparse Word RepresentationsCode0
Compositional Language Understanding with Text-based Relational ReasoningCode0
Improving Pre-trained Language Model Sensitivity via Mask Specific losses: A case study on Biomedical NERCode0
Compositional Generalization in Semantic Parsing: Pre-training vs. Specialized ArchitecturesCode0
GPT-based Generation for Classical Chinese PoetryCode0
Automated title and abstract screening for scoping reviews using the GPT-4 Large Language ModelCode0
Automated Source Code Generation and Auto-completion Using Deep Learning: Comparing and Discussing Current Language-Model-Related ApproachesCode0
ISR-LLM: Iterative Self-Refined Large Language Model for Long-Horizon Sequential Task PlanningCode0
Composing Structure-Aware Batches for Pairwise Sentence ClassificationCode0
Composing Byte-Pair Encodings for Morphological Sequence ClassificationCode0
Fair and Argumentative Language Modeling for Computational ArgumentationCode0
Composer Style Classification of Piano Sheet Music Images Using Language Model PretrainingCode0
Automated Privacy Information Annotation in Large Language Model InteractionsCode0
Hyperparameter Power Impact in Transformer Language Model TrainingCode0
Low-Resource Language Processing: An OCR-Driven Summarization and Translation PipelineCode0
Fair multilingual vandalism detection system for WikipediaCode0
Fairness-Aware Structured Pruning in TransformersCode0
Automated Generation and Tagging of Knowledge Components from Multiple-Choice QuestionsCode0
ComplexFormer: Disruptively Advancing Transformer Inference Ability via Head-Specific Complex Vector AttentionCode0
Automated Evaluation of Out-of-Context ErrorsCode0
Hyperpolyglot LLMs: Cross-Lingual Interpretability in Token EmbeddingsCode0
Comparing Text Representations: A Theory-Driven ApproachCode0
Show:102550
← PrevPage 350 of 353Next →

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1Decay RNNValidation perplexity76.67Unverified
2GRUValidation perplexity53.78Unverified
3LSTMValidation perplexity52.73Unverified
4LSTMTest perplexity48.7Unverified
5Temporal CNNTest perplexity45.2Unverified
6TCNTest perplexity45.19Unverified
7GCNN-8Test perplexity44.9Unverified
8Neural cache model (size = 100)Test perplexity44.8Unverified
9Neural cache model (size = 2,000)Test perplexity40.8Unverified
10GPT-2 SmallTest perplexity37.5Unverified
#ModelMetricClaimedVerifiedStatus
1TCNTest perplexity108.47Unverified
2Seq-U-NetTest perplexity107.95Unverified
3GRU (Bai et al., 2018)Test perplexity92.48Unverified
4R-TransformerTest perplexity84.38Unverified
5Zaremba et al. (2014) - LSTM (medium)Test perplexity82.7Unverified
6Gal & Ghahramani (2016) - Variational LSTM (medium)Test perplexity79.7Unverified
7LSTM (Bai et al., 2018)Test perplexity78.93Unverified
8Zaremba et al. (2014) - LSTM (large)Test perplexity78.4Unverified
9Gal & Ghahramani (2016) - Variational LSTM (large)Test perplexity75.2Unverified
10Inan et al. (2016) - Variational RHNTest perplexity66Unverified
#ModelMetricClaimedVerifiedStatus
1LSTM (7 layers)Bit per Character (BPC)1.67Unverified
2HypernetworksBit per Character (BPC)1.34Unverified
3SHA-LSTM (4 layers, h=1024, no attention head)Bit per Character (BPC)1.33Unverified
4LN HM-LSTMBit per Character (BPC)1.32Unverified
5ByteNetBit per Character (BPC)1.31Unverified
6Recurrent Highway NetworksBit per Character (BPC)1.27Unverified
7Large FS-LSTM-4Bit per Character (BPC)1.25Unverified
8Large mLSTMBit per Character (BPC)1.24Unverified
9AWD-LSTM (3 layers)Bit per Character (BPC)1.23Unverified
10Cluster-Former (#C=512)Bit per Character (BPC)1.22Unverified
#ModelMetricClaimedVerifiedStatus
1Smaller Transformer 126M (pre-trained)Test perplexity33Unverified
2OPT 125MTest perplexity32.26Unverified
3Larger Transformer 771M (pre-trained)Test perplexity28.1Unverified
4OPT 1.3BTest perplexity19.55Unverified
5GPT-Neo 125MTest perplexity17.83Unverified
6OPT 2.7BTest perplexity17.81Unverified
7Smaller Transformer 126M (fine-tuned)Test perplexity12Unverified
8GPT-Neo 1.3BTest perplexity11.46Unverified
9Transformer 125MTest perplexity10.7Unverified
10GPT-Neo 2.7BTest perplexity10.44Unverified