SOTAVerified

Language Modelling

A language model is a model of natural language. Language models are useful for a variety of tasks, including speech recognition, machine translation, natural language generation (generating more human-like text), optical character recognition, route optimization, handwriting recognition, grammar induction, and information retrieval.

Large language models (LLMs), currently their most advanced form, are predominantly based on transformers trained on larger datasets (frequently using words scraped from the public internet). They have superseded recurrent neural network-based models, which had previously superseded the purely statistical models, such as word n-gram language model.

Source: Wikipedia

Papers

Showing 1560115650 of 17610 papers

TitleStatusHype
SemStyle: Learning to Generate Stylised Image Captions using Unaligned TextCode0
Language Models Can Learn Exceptions to Syntactic RulesCode0
Learning Intrinsic Sparse Structures within Long Short-Term MemoryCode0
Towards Generating Query to Perform Query Focused Abstractive Summarization using Pre-trained ModelCode0
Syntactic realization with data-driven neural tree grammarsCode0
Sensei: Self-Supervised Sensor Name SegmentationCode0
LIMP: Large Language Model Enhanced Intent-aware Mobility PredictionCode0
LIMIT-BERT : Linguistics Informed Multi-Task BERTCode0
ORBIT: Cost-Effective Dataset Curation for Large Language Model Domain Adaptation with an Astronomy Case StudyCode0
Syntactic Substitutability as Unsupervised Dependency SyntaxCode0
Syntactic Surprisal From Neural Models Predicts, But Underestimates, Human Processing Difficulty From Syntactic AmbiguitiesCode0
Learning Instructions with Unlabeled Data for Zero-Shot Cross-Task GeneralizationCode0
TRIDENT: Enhancing Large Language Model Safety with Tri-Dimensional Diversified Red-Teaming Data SynthesisCode0
Oracle performance for visual captioningCode0
Sentence Encoders on STILTs: Supplementary Training on Intermediate Labeled-data TasksCode0
Towards Harnessing Large Language Models for Comprehension of Conversational GroundingCode0
Sentence-level Media Bias Analysis with Event Relation GraphCode0
Towards Hate Speech Detection at Large via Deep Generative ModelingCode0
Syntax-driven Data Augmentation for Named Entity RecognitionCode0
Language Models as Knowledge Bases?Code0
Lil-Bevo: Explorations of Strategies for Training Language Models in More Humanlike WaysCode0
TRAM: Bridging Trust Regions and Sharpness Aware MinimizationCode0
Language Models as Context-sensitive Word Search EnginesCode0
Language-Model Prior Overcomes Cold-Start ItemsCode0
Learning from Past Mistakes: Improving Automatic Speech Recognition Output via Noisy-Clean Phrase Context ModelingCode0
Learning Dynamic Contextualised Word Embeddings via Template-based Temporal AdaptationCode0
SentiLARE: Sentiment-Aware Language Representation Learning with Linguistic KnowledgeCode0
Linguistic Frameworks Go Toe-to-Toe at Neuro-Symbolic Language ModelingCode0
KG-BERT: BERT for Knowledge Graph CompletionCode0
Sentiment analysis in tweets: an assessment study from classical to modern text representation modelsCode0
Memory-Efficient Adaptive OptimizationCode0
Learning Dynamic Author Representations with Temporal Language ModelsCode0
Memory-Augmented Recurrent Neural Networks Can Learn Generalized Dyck LanguagesCode0
Likelihood as a Performance Gauge for Retrieval-Augmented GenerationCode0
Optimizing Retrieval-augmented Reader Models via Token EliminationCode0
Synthesizing Interpretable Control Policies through Large Language Model Guided SearchCode0
Sentiment-enhanced Graph-based Sarcasm Explanation in DialogueCode0
Optimizing Deep Neural Networks using Safety-Guided Self CompressionCode0
Transcending the Attention Paradigm: Representation Learning from Geospatial Social Media DataCode0
Memory and Knowledge Augmented Language Models for Inferring Salience in Long-Form StoriesCode0
Optimization of Armv9 architecture general large language model inference performance based on Llama.cppCode0
MeLT: Message-Level Transformer with Masked Document Representations as Pre-Training for Stance DetectionCode0
OpenTable-R1: A Reinforcement Learning Augmented Tool Agent for Open-Domain Table Question AnsweringCode0
Language Model Preference Evaluation with Multiple Weak EvaluatorsCode0
Synthetic Data Made to Order: The Case of ParsingCode0
Opening the Black Box: Analyzing Attention Weights and Hidden States in Pre-trained Language Models for Non-language TasksCode0
Towards Interpretable Hate Speech Detection using Large Language Model-extracted RationalesCode0
Learning Deterministic Weighted Automata with Queries and CounterexamplesCode0
Separating the Wheat from the Chaff with BREAD: An open-source benchmark and metrics to detect redundancy in textCode0
OpenFraming: Open-sourced Tool for Computational Framing Analysis of Multilingual DataCode0
Show:102550
← PrevPage 313 of 353Next →

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1Decay RNNValidation perplexity76.67Unverified
2GRUValidation perplexity53.78Unverified
3LSTMValidation perplexity52.73Unverified
4LSTMTest perplexity48.7Unverified
5Temporal CNNTest perplexity45.2Unverified
6TCNTest perplexity45.19Unverified
7GCNN-8Test perplexity44.9Unverified
8Neural cache model (size = 100)Test perplexity44.8Unverified
9Neural cache model (size = 2,000)Test perplexity40.8Unverified
10GPT-2 SmallTest perplexity37.5Unverified
#ModelMetricClaimedVerifiedStatus
1TCNTest perplexity108.47Unverified
2Seq-U-NetTest perplexity107.95Unverified
3GRU (Bai et al., 2018)Test perplexity92.48Unverified
4R-TransformerTest perplexity84.38Unverified
5Zaremba et al. (2014) - LSTM (medium)Test perplexity82.7Unverified
6Gal & Ghahramani (2016) - Variational LSTM (medium)Test perplexity79.7Unverified
7LSTM (Bai et al., 2018)Test perplexity78.93Unverified
8Zaremba et al. (2014) - LSTM (large)Test perplexity78.4Unverified
9Gal & Ghahramani (2016) - Variational LSTM (large)Test perplexity75.2Unverified
10Inan et al. (2016) - Variational RHNTest perplexity66Unverified
#ModelMetricClaimedVerifiedStatus
1LSTM (7 layers)Bit per Character (BPC)1.67Unverified
2HypernetworksBit per Character (BPC)1.34Unverified
3SHA-LSTM (4 layers, h=1024, no attention head)Bit per Character (BPC)1.33Unverified
4LN HM-LSTMBit per Character (BPC)1.32Unverified
5ByteNetBit per Character (BPC)1.31Unverified
6Recurrent Highway NetworksBit per Character (BPC)1.27Unverified
7Large FS-LSTM-4Bit per Character (BPC)1.25Unverified
8Large mLSTMBit per Character (BPC)1.24Unverified
9AWD-LSTM (3 layers)Bit per Character (BPC)1.23Unverified
10Cluster-Former (#C=512)Bit per Character (BPC)1.22Unverified
#ModelMetricClaimedVerifiedStatus
1Smaller Transformer 126M (pre-trained)Test perplexity33Unverified
2OPT 125MTest perplexity32.26Unverified
3Larger Transformer 771M (pre-trained)Test perplexity28.1Unverified
4OPT 1.3BTest perplexity19.55Unverified
5GPT-Neo 125MTest perplexity17.83Unverified
6OPT 2.7BTest perplexity17.81Unverified
7Smaller Transformer 126M (fine-tuned)Test perplexity12Unverified
8GPT-Neo 1.3BTest perplexity11.46Unverified
9Transformer 125MTest perplexity10.7Unverified
10GPT-Neo 2.7BTest perplexity10.44Unverified