SOTAVerified

Language Modelling

A language model is a model of natural language. Language models are useful for a variety of tasks, including speech recognition, machine translation, natural language generation (generating more human-like text), optical character recognition, route optimization, handwriting recognition, grammar induction, and information retrieval.

Large language models (LLMs), currently their most advanced form, are predominantly based on transformers trained on larger datasets (frequently using words scraped from the public internet). They have superseded recurrent neural network-based models, which had previously superseded the purely statistical models, such as word n-gram language model.

Source: Wikipedia

Papers

Showing 1715117200 of 17610 papers

TitleStatusHype
Temperature-scaling surprisal estimates improve fit to human reading times -- but does it do so for the "right reasons"?Code0
How much complexity does an RNN architecture need to learn syntax-sensitive dependencies?Code0
Generation with Dynamic VocabularyCode0
Every Answer Matters: Evaluating Commonsense with Probabilistic MeasuresCode0
Controlling the Amount of Verbatim Copying in Abstractive SummarizationCode0
Evidence-backed Fact Checking using RAG and Few-Shot In-Context Learning with LLMsCode0
AlgebraNetsCode0
ANGOFA: Leveraging OFA Embedding Initialization and Synthetic Data for Angolan Language ModelCode0
Evidence Is All You Need: Ordering Imaging Studies via Language Model Alignment with the ACR Appropriateness CriteriaCode0
Generative adversarial networks vs large language models: a comparative study on synthetic tabular data generationCode0
Controlling Large Language Model with Latent ActionsCode0
Controlled Text Generation for Black-box Language Models via Score-based Progressive EditorCode0
Decoupled Sequence and Structure Generation for Realistic Antibody DesignCode0
How Personality Traits Influence Negotiation Outcomes? A Simulation based on Large Language ModelsCode0
Bidirectional Transformer Reranker for Grammatical Error CorrectionCode0
Controllable Neural Story Plot Generation via Reward ShapingCode0
How Phonotactics Affect Multilingual and Zero-shot ASR PerformanceCode0
Controllable Citation Sentence Generation with Language ModelsCode0
How Predictable Are Large Language Model Capabilities? A Case Study on BIG-benchCode0
Improving Generalization Performance by Switching from Adam to SGDCode0
How Robust Are Router-LLMs? Analysis of the Fragility of LLM Routing CapabilitiesCode0
Evolutionary Stochastic Gradient Descent for Optimization of Deep Neural NetworksCode0
Evolutionary Verbalizer Search for Prompt-based Few Shot Text ClassificationCode0
Evolution of ESG-focused DLT Research: An NLP Analysis of the LiteratureCode0
Improving Grammatical Error Correction with Machine Translation PairsCode0
A Common Pitfall of Margin-based Language Model Alignment: Gradient EntanglementCode0
A deep language model for software codeCode0
Evolving Assembly Code in an Adversarial EnvironmentCode0
A Deep Generative Model for Fragment-Based Molecule GenerationCode0
Improving In-Context Learning with Small Language Model EnsemblesCode0
Contrastive learning of T cell receptor representationsCode0
AlcLaM: Arabic Dialectal Language ModelCode0
Generative Prompt InternalizationCode0
Evolving Subnetwork Training for Large Language ModelsCode0
Improving Information Extraction on Business Documents with Specific Pre-Training TasksCode0
Is Training Data Quality or Quantity More Impactful to Small Language Model Performance?Code0
Bidirectional Attention as a Mixture of Continuous Word ExpertsCode0
How to Determine the Most Powerful Pre-trained Language Model without Brute Force Fine-tuning? An Empirical SurveyCode0
BiasKG: Adversarial Knowledge Graphs to Induce Bias in Large Language ModelsCode0
How to Determine the Preferred Image Distribution of a Black-Box Vision-Language Model?Code0
How To Evaluate Your Dialogue System: Probe Tasks as an Alternative for Token-level Evaluation MetricsCode0
Examining Language Modeling Assumptions Using an Annotated Literary Dialect CorpusCode0
An Eye on Clinical BERT: Investigating Language Model Generalization for Diabetic Eye Disease PhenotypingCode0
An Exploratory Study on Automatic Identification of Assumptions in the Development of Deep Learning FrameworksCode0
How to Leverage Demonstration Data in Alignment for Large Language Model? A Self-Imitation Learning PerspectiveCode0
How to Leverage Personal Textual Knowledge for Personalized Conversational Information RetrievalCode0
Contrastive Language Prompting to Ease False Positives in Medical Anomaly DetectionCode0
exBERT: A Visual Analysis Tool to Explore Learned Representations in Transformers ModelsCode0
Contrastive and Consistency Learning for Neural Noisy-Channel Model in Spoken Language UnderstandingCode0
Continuous Speech Tokenizer in Text To SpeechCode0
Show:102550
← PrevPage 344 of 353Next →

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1Decay RNNValidation perplexity76.67Unverified
2GRUValidation perplexity53.78Unverified
3LSTMValidation perplexity52.73Unverified
4LSTMTest perplexity48.7Unverified
5Temporal CNNTest perplexity45.2Unverified
6TCNTest perplexity45.19Unverified
7GCNN-8Test perplexity44.9Unverified
8Neural cache model (size = 100)Test perplexity44.8Unverified
9Neural cache model (size = 2,000)Test perplexity40.8Unverified
10GPT-2 SmallTest perplexity37.5Unverified
#ModelMetricClaimedVerifiedStatus
1TCNTest perplexity108.47Unverified
2Seq-U-NetTest perplexity107.95Unverified
3GRU (Bai et al., 2018)Test perplexity92.48Unverified
4R-TransformerTest perplexity84.38Unverified
5Zaremba et al. (2014) - LSTM (medium)Test perplexity82.7Unverified
6Gal & Ghahramani (2016) - Variational LSTM (medium)Test perplexity79.7Unverified
7LSTM (Bai et al., 2018)Test perplexity78.93Unverified
8Zaremba et al. (2014) - LSTM (large)Test perplexity78.4Unverified
9Gal & Ghahramani (2016) - Variational LSTM (large)Test perplexity75.2Unverified
10Inan et al. (2016) - Variational RHNTest perplexity66Unverified
#ModelMetricClaimedVerifiedStatus
1LSTM (7 layers)Bit per Character (BPC)1.67Unverified
2HypernetworksBit per Character (BPC)1.34Unverified
3SHA-LSTM (4 layers, h=1024, no attention head)Bit per Character (BPC)1.33Unverified
4LN HM-LSTMBit per Character (BPC)1.32Unverified
5ByteNetBit per Character (BPC)1.31Unverified
6Recurrent Highway NetworksBit per Character (BPC)1.27Unverified
7Large FS-LSTM-4Bit per Character (BPC)1.25Unverified
8Large mLSTMBit per Character (BPC)1.24Unverified
9AWD-LSTM (3 layers)Bit per Character (BPC)1.23Unverified
10Cluster-Former (#C=512)Bit per Character (BPC)1.22Unverified
#ModelMetricClaimedVerifiedStatus
1Smaller Transformer 126M (pre-trained)Test perplexity33Unverified
2OPT 125MTest perplexity32.26Unverified
3Larger Transformer 771M (pre-trained)Test perplexity28.1Unverified
4OPT 1.3BTest perplexity19.55Unverified
5GPT-Neo 125MTest perplexity17.83Unverified
6OPT 2.7BTest perplexity17.81Unverified
7Smaller Transformer 126M (fine-tuned)Test perplexity12Unverified
8GPT-Neo 1.3BTest perplexity11.46Unverified
9Transformer 125MTest perplexity10.7Unverified
10GPT-Neo 2.7BTest perplexity10.44Unverified