SOTAVerified

Language Modelling

A language model is a model of natural language. Language models are useful for a variety of tasks, including speech recognition, machine translation, natural language generation (generating more human-like text), optical character recognition, route optimization, handwriting recognition, grammar induction, and information retrieval.

Large language models (LLMs), currently their most advanced form, are predominantly based on transformers trained on larger datasets (frequently using words scraped from the public internet). They have superseded recurrent neural network-based models, which had previously superseded the purely statistical models, such as word n-gram language model.

Source: Wikipedia

Papers

Showing 1490114950 of 17610 papers

TitleStatusHype
RecallM: An Adaptable Memory Mechanism with Temporal Understanding for Large Language ModelsCode0
LOGIN: A Large Language Model Consulted Graph Neural Network Training FrameworkCode0
Learn Your Tokens: Word-Pooled Tokenization for Language ModelingCode0
ReasoningLM: Enabling Structural Subgraph Reasoning in Pre-trained Language Models for Question Answering over Knowledge GraphCode0
Reasoning Large Language Model Errors Arise from Hallucinating Critical Problem FeaturesCode0
Reasoning-Grounded Natural Language Explanations for Language ModelsCode0
Towards Logically Sound Natural Language Reasoning with Logic-Enhanced Language Model AgentsCode0
Logical Implications for Visual Question Answering ConsistencyCode0
textTOvec: Deep Contextualized Neural Autoregressive Topic Models of Language with Distributed Compositional PriorCode0
Rethinking the Event Coding Pipeline with Prompt EntailmentCode0
Rethinking the Role of Proxy Rewards in Language Model AlignmentCode0
Reanalyzing L2 Preposition Learning with Bayesian Mixed Effects and a Pretrained Language ModelCode0
Location Name Extraction from Targeted Text Streams using Gazetteer-based Statistical Language ModelsCode0
REALM: RAG-Driven Enhancement of Multimodal Electronic Health Records Analysis via Large Language ModelsCode0
RealHarm: A Collection of Real-World Language Model Application FailuresCode0
Reading Between the Lines: A dataset and a study on why some texts are tougher than othersCode0
Topically Driven Neural Language ModelCode0
Textual Entailment for Effective Triple Validation in Object PredictionCode0
RCMHA: Relative Convolutional Multi-Head Attention for Natural Language ModellingCode0
Towards Interpretable Sequence Continuation: Analyzing Shared Circuits in Large Language ModelsCode0
Multi-task Learning of Negation and Speculation for Targeted Sentiment ClassificationCode0
Spell Once, Summon Anywhere: A Two-Level Open-Vocabulary Language ModelCode0
Spherical Latent Spaces for Stable Variational AutoencodersCode0
Multi-Task Deep Neural Networks for Natural Language UnderstandingCode0
Multi-system machine translation using online APIs for English-LatvianCode0
Knowledge-aware Collaborative Filtering with Pre-trained Language Model for Personalized Review-based Rating PredictionCode0
Retrieval Augmented Generation Systems: Automatic Dataset Creation, Evaluation and Boolean Agent SetupCode0
Retrieval-Augmented Language Model for Extreme Multi-Label Knowledge Graph Link PredictionCode0
Multistage Collaborative Knowledge Distillation from a Large Language Model for Semi-Supervised Sequence GenerationCode0
Multi-Programming Language Ensemble for Code Generation in Large Language ModelCode0
Multiplicative LSTM for sequence modellingCode0
RAVEN: In-Context Learning with Retrieval-Augmented Encoder-Decoder Language ModelsCode0
TF-LM: TensorFlow-based Language Modeling ToolkitCode0
Multiple-Source Domain Adaptation via Coordinated Domain Encoders and Paired ClassifiersCode0
Multi-objective Reinforcement learning from AI FeedbackCode0
Towards Unsupervised Recognition of Token-level Semantic Differences in Related DocumentsCode0
Rational RecurrencesCode0
Locally Differentially Private Document Generation Using Zero Shot PromptingCode0
RATE: Causal Explainability of Reward Models with Imperfect CounterfactualsCode0
Multi-Objective Large Language Model UnlearningCode0
RaTE: a Reproducible automatic Taxonomy Evaluation by Filling the GapCode0
RAS-Eval: A Comprehensive Benchmark for Security Evaluation of LLM Agents in Real-World EnvironmentsCode0
Robustness Analysis of Video-Language Models Against Visual and Language PerturbationsCode0
Localized Symbolic Knowledge Distillation for Visual Commonsense ModelsCode0
Rare Words: A Major Problem for Contextualized Embeddings And How to Fix it by Attentive MimickingCode0
Rapport-Driven Virtual Agent: Rapport Building Dialogue Strategy for Improving User Experience at First MeetingCode0
Retrieve to Explain: Evidence-driven Predictions with Language ModelsCode0
Multimodal Quantum Natural Language Processing: A Novel Framework for using Quantum Methods to Analyse Real DataCode0
Thai Wav2Vec2.0 with CommonVoice V8Code0
KatzBot: Revolutionizing Academic Chatbot for Enhanced CommunicationCode0
Show:102550
← PrevPage 299 of 353Next →

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1Decay RNNValidation perplexity76.67Unverified
2GRUValidation perplexity53.78Unverified
3LSTMValidation perplexity52.73Unverified
4LSTMTest perplexity48.7Unverified
5Temporal CNNTest perplexity45.2Unverified
6TCNTest perplexity45.19Unverified
7GCNN-8Test perplexity44.9Unverified
8Neural cache model (size = 100)Test perplexity44.8Unverified
9Neural cache model (size = 2,000)Test perplexity40.8Unverified
10GPT-2 SmallTest perplexity37.5Unverified
#ModelMetricClaimedVerifiedStatus
1TCNTest perplexity108.47Unverified
2Seq-U-NetTest perplexity107.95Unverified
3GRU (Bai et al., 2018)Test perplexity92.48Unverified
4R-TransformerTest perplexity84.38Unverified
5Zaremba et al. (2014) - LSTM (medium)Test perplexity82.7Unverified
6Gal & Ghahramani (2016) - Variational LSTM (medium)Test perplexity79.7Unverified
7LSTM (Bai et al., 2018)Test perplexity78.93Unverified
8Zaremba et al. (2014) - LSTM (large)Test perplexity78.4Unverified
9Gal & Ghahramani (2016) - Variational LSTM (large)Test perplexity75.2Unverified
10Inan et al. (2016) - Variational RHNTest perplexity66Unverified
#ModelMetricClaimedVerifiedStatus
1LSTM (7 layers)Bit per Character (BPC)1.67Unverified
2HypernetworksBit per Character (BPC)1.34Unverified
3SHA-LSTM (4 layers, h=1024, no attention head)Bit per Character (BPC)1.33Unverified
4LN HM-LSTMBit per Character (BPC)1.32Unverified
5ByteNetBit per Character (BPC)1.31Unverified
6Recurrent Highway NetworksBit per Character (BPC)1.27Unverified
7Large FS-LSTM-4Bit per Character (BPC)1.25Unverified
8Large mLSTMBit per Character (BPC)1.24Unverified
9AWD-LSTM (3 layers)Bit per Character (BPC)1.23Unverified
10Cluster-Former (#C=512)Bit per Character (BPC)1.22Unverified
#ModelMetricClaimedVerifiedStatus
1Smaller Transformer 126M (pre-trained)Test perplexity33Unverified
2OPT 125MTest perplexity32.26Unverified
3Larger Transformer 771M (pre-trained)Test perplexity28.1Unverified
4OPT 1.3BTest perplexity19.55Unverified
5GPT-Neo 125MTest perplexity17.83Unverified
6OPT 2.7BTest perplexity17.81Unverified
7Smaller Transformer 126M (fine-tuned)Test perplexity12Unverified
8GPT-Neo 1.3BTest perplexity11.46Unverified
9Transformer 125MTest perplexity10.7Unverified
10GPT-Neo 2.7BTest perplexity10.44Unverified