SOTAVerified

Language Modelling

A language model is a model of natural language. Language models are useful for a variety of tasks, including speech recognition, machine translation, natural language generation (generating more human-like text), optical character recognition, route optimization, handwriting recognition, grammar induction, and information retrieval.

Large language models (LLMs), currently their most advanced form, are predominantly based on transformers trained on larger datasets (frequently using words scraped from the public internet). They have superseded recurrent neural network-based models, which had previously superseded the purely statistical models, such as word n-gram language model.

Source: Wikipedia

Papers

Showing 1645116500 of 17610 papers

TitleStatusHype
Fingerprinting web servers through Transformer-encoded HTTP response headersCode0
A Resilient and Accessible Distribution-Preserving Watermark for Large Language ModelsCode0
Fingerspelling PoseNet: Enhancing Fingerspelling Translation with Pose-Based Transformer ModelsCode0
Character-Level Language Modeling with Deeper Self-AttentionCode0
Character-Level Incremental Speech Recognition with Recurrent Neural NetworksCode0
Modeling Disclosive Transparency in NLP Application DescriptionsCode0
Assessing the Reliability of Large Language Model KnowledgeCode0
Finnish resources for evaluating language model semanticsCode0
Effectiveness of Cross-linguistic Extraction of Genetic Information using Generative Large Language ModelsCode0
Characterizing Verbatim Short-Term Memory in Neural Language ModelsCode0
DigiCall: A Benchmark for Measuring the Maturity of Digital Strategy through Company Earning CallsCode0
Assessing the Promise and Pitfalls of ChatGPT for Automated Code GenerationCode0
Differentially Private Steering for Large Language Model AlignmentCode0
FinTree: Financial Dataset Pretrain Transformer Encoder for Relation ExtractionCode0
FinTruthQA: A Benchmark Dataset for Evaluating the Quality of Financial Information DisclosureCode0
Differentiable Random Access Memory using LatticesCode0
Effective Representation for Easy-First Dependency ParsingCode0
Characterizing Learning Curves During Language Model Pre-Training: Learning, Forgetting, and StabilityCode0
Effective Use of Bidirectional Language Modeling for Transfer Learning in Biomedical Named Entity RecognitionCode0
Differentiable Outlier Detection Enable Robust Deep Multimodal AnalysisCode0
Differentiable N-gram Objective on Abstractive SummarizationCode0
Fire Burns, Sword Cuts: Commonsense Inductive Bias for Exploration in Text-based GamesCode0
DiFair: A Benchmark for Disentangled Assessment of Gender Knowledge and BiasCode0
A Geometric Notion of Causal ProbingCode0
Effect of Visual Extensions on Natural Language Understanding in Vision-and-Language ModelsCode0
Characterizing and Understanding the Behavior of Quantized Models for Reliable DeploymentCode0
FIRE: Food Image to REcipe generationCode0
First Align, then Predict: Understanding the Cross-Lingual Ability of Multilingual BERTCode0
Efficacy of Language Model Self-Play in Non-Zero-Sum GamesCode0
Diet-ODIN: A Novel Framework for Opioid Misuse Detection with Interpretable Dietary PatternsCode0
First Automatic Fongbe Continuous Speech Recognition System: Development of Acoustic Models and Language ModelsCode0
Character-based Neural Networks for Sentence Pair ModelingCode0
First Heuristic Then Rational: Dynamic Use of Heuristics in Language Model ReasoningCode0
Assessing the Ability of LSTMs to Learn Syntax-Sensitive DependenciesCode0
First-Pass Large Vocabulary Continuous Speech Recognition using Bi-Directional Recurrent DNNsCode0
A Controlled Reevaluation of Coreference Resolution ModelsCode0
Efficient and Long-Tailed Generalization for Pre-trained Vision-Language ModelCode0
FIRST: Teach A Reliable Large Language Model Through Efficient Trustworthy DistillationCode0
A Context-based Approach for Dialogue Act Recognition using Simple Recurrent Neural NetworksCode0
HBTP: Heuristic Behavior Tree Planning with Large Language Model ReasoningCode0
Efficient Black-Box Adversarial Attacks on Neural Text DetectorsCode0
Assessing Generative Language Models in Classification Tasks: Performance and Self-Evaluation Capabilities in the Environmental and Climate Change DomainCode0
Dict-BERT: Enhancing Language Model Pre-training with DictionaryCode0
DIBERT: Dependency Injected Bidirectional Encoder Representations from TransformersCode0
Efficient Contextualized Representation: Language Model Pruning for Sequence LabelingCode0
Fisher Mask Nodes for Language Model MergingCode0
Chameleon: A Flexible Data-mixing Framework for Language Model Pretraining and FinetuningCode0
FiSSA at SemEval-2020 Task 9: Fine-tuned For FeelingsCode0
IPO: Your Language Model is Secretly a Preference ClassifierCode0
Image as a Foreign Language: BEiT Pretraining for All Vision and Vision-Language TasksCode0
Show:102550
← PrevPage 330 of 353Next →

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1Decay RNNValidation perplexity76.67Unverified
2GRUValidation perplexity53.78Unverified
3LSTMValidation perplexity52.73Unverified
4LSTMTest perplexity48.7Unverified
5Temporal CNNTest perplexity45.2Unverified
6TCNTest perplexity45.19Unverified
7GCNN-8Test perplexity44.9Unverified
8Neural cache model (size = 100)Test perplexity44.8Unverified
9Neural cache model (size = 2,000)Test perplexity40.8Unverified
10GPT-2 SmallTest perplexity37.5Unverified
#ModelMetricClaimedVerifiedStatus
1TCNTest perplexity108.47Unverified
2Seq-U-NetTest perplexity107.95Unverified
3GRU (Bai et al., 2018)Test perplexity92.48Unverified
4R-TransformerTest perplexity84.38Unverified
5Zaremba et al. (2014) - LSTM (medium)Test perplexity82.7Unverified
6Gal & Ghahramani (2016) - Variational LSTM (medium)Test perplexity79.7Unverified
7LSTM (Bai et al., 2018)Test perplexity78.93Unverified
8Zaremba et al. (2014) - LSTM (large)Test perplexity78.4Unverified
9Gal & Ghahramani (2016) - Variational LSTM (large)Test perplexity75.2Unverified
10Inan et al. (2016) - Variational RHNTest perplexity66Unverified
#ModelMetricClaimedVerifiedStatus
1LSTM (7 layers)Bit per Character (BPC)1.67Unverified
2HypernetworksBit per Character (BPC)1.34Unverified
3SHA-LSTM (4 layers, h=1024, no attention head)Bit per Character (BPC)1.33Unverified
4LN HM-LSTMBit per Character (BPC)1.32Unverified
5ByteNetBit per Character (BPC)1.31Unverified
6Recurrent Highway NetworksBit per Character (BPC)1.27Unverified
7Large FS-LSTM-4Bit per Character (BPC)1.25Unverified
8Large mLSTMBit per Character (BPC)1.24Unverified
9AWD-LSTM (3 layers)Bit per Character (BPC)1.23Unverified
10Cluster-Former (#C=512)Bit per Character (BPC)1.22Unverified
#ModelMetricClaimedVerifiedStatus
1Smaller Transformer 126M (pre-trained)Test perplexity33Unverified
2OPT 125MTest perplexity32.26Unverified
3Larger Transformer 771M (pre-trained)Test perplexity28.1Unverified
4OPT 1.3BTest perplexity19.55Unverified
5GPT-Neo 125MTest perplexity17.83Unverified
6OPT 2.7BTest perplexity17.81Unverified
7Smaller Transformer 126M (fine-tuned)Test perplexity12Unverified
8GPT-Neo 1.3BTest perplexity11.46Unverified
9Transformer 125MTest perplexity10.7Unverified
10GPT-Neo 2.7BTest perplexity10.44Unverified