SOTAVerified

Language Modelling

A language model is a model of natural language. Language models are useful for a variety of tasks, including speech recognition, machine translation, natural language generation (generating more human-like text), optical character recognition, route optimization, handwriting recognition, grammar induction, and information retrieval.

Large language models (LLMs), currently their most advanced form, are predominantly based on transformers trained on larger datasets (frequently using words scraped from the public internet). They have superseded recurrent neural network-based models, which had previously superseded the purely statistical models, such as word n-gram language model.

Source: Wikipedia

Papers

Showing 1720117250 of 17610 papers

TitleStatusHype
Continuous Language Model Interpolation for Dynamic and Controllable Text GenerationCode0
Improving Instruction Following in Language Models through Proxy-Based Uncertainty EstimationCode0
EXMODD: An EXplanatory Multimodal Open-Domain Dialogue datasetCode0
Generative Language Models on Nucleotide Sequences of Human GenesCode0
Exp4Fuse: A Rank Fusion Framework for Enhanced Sparse Retrieval using Large Language Model-based Query ExpansionCode0
How to Protect Copyright Data in Optimization of Large Language Models?Code0
TextKD-GAN: Text Generation using KnowledgeDistillation and Generative Adversarial NetworksCode0
Biased Importance Sampling for Deep Neural Network TrainingCode0
Improving Interpersonal Communication by Simulating Audiences with Language ModelsCode0
Expanding the Vocabulary of BERT for Knowledge Base ConstructionCode0
Expansion via Prediction of Importance with ContextualizationCode0
Beyond Reactive Safety: Risk-Aware LLM Alignment via Long-Horizon SimulationCode0
Beyond Perplexity: Multi-dimensional Safety Evaluation of LLM CompressionCode0
An Exploratory Investigation into Code License Infringements in Large Language Model Training DatasetsCode0
Beyond Ontology in Dialogue State Tracking for Goal-Oriented ChatbotCode0
Beyond Language: Learning Commonsense from Images for ReasoningCode0
Beyond In-Distribution Success: Scaling Curves of CoT Granularity for Language Model GeneralizationCode0
Beyond Distributional Hypothesis: Let Language Models Learn Meaning-Text CorrespondenceCode0
Continual Learning of Recurrent Neural Networks by Locally Aligning Distributed RepresentationsCode0
Continual and Multi-Task Architecture SearchCode0
Continual adaptation for efficient machine communicationCode0
Alibaba LingmaAgent: Improving Automated Issue Resolution via Comprehensive Repository ExplorationCode0
How to Unleash the Power of Large Language Models for Few-shot Relation Extraction?Code0
Beyond Concept Bottleneck Models: How to Make Black Boxes Intervenable?Code0
Explainable and Discourse Topic-aware Neural Language UnderstandingCode0
Beyond Accuracy Optimization: Computer Vision Losses for Large Language Model Fine-TuningCode0
How transformers learn structured data: insights from hierarchical filteringCode0
Beyond 512 Tokens: Siamese Multi-depth Transformer-based Hierarchical Encoder for Long-Form Document MatchingCode0
An Exploration of Softmax Alternatives Belonging to the Spherical Loss FamilyCode0
Contextual String Embeddings for Sequence LabelingCode0
Improving Language Generation with Sentence Coherence ObjectiveCode0
Better Long-Range Dependency By Bootstrapping A Mutual Information RegularizerCode0
An Evalutation of Programming Language Models' performance on Software Defect DetectionCode0
Generative Relevance Feedback and Convergence of Adaptive Re-Ranking: University of Glasgow Terrier Team at TREC DL 2023Code0
Explainable Verbal Reasoner Plus (EVR+): A Natural Language Reasoning Framework that Supports Diverse Compositional ReasoningCode0
A Co-Attentive Cross-Lingual Neural Model for Dialogue Breakdown DetectionCode0
Better Language Model with Hypernym Class PredictionCode0
Dial-MAE: ConTextual Masked Auto-Encoder for Retrieval-based Dialogue SystemsCode0
Investigating the translation capabilities of Large Language Models trained on parallel data onlyCode0
Explaining Context Length Scaling and Bounds for Language ModelsCode0
Contextualized Word Representations for Reading ComprehensionCode0
360^REA: Towards A Reusable Experience Accumulation with 360° Assessment for Multi-Agent SystemCode0
Generative Social ChoiceCode0
Explaining Natural Language Processing Classifiers with Occlusion and Language ModelingCode0
How would Stance Detection Techniques Evolve after the Launch of ChatGPT?Code0
Leveraging Entity Information for Cross-Modality Correlation Learning: The Entity-Guided Multimodal SummarizationCode0
CWTM: Leveraging Contextualized Word Embeddings from BERT for Neural Topic ModelingCode0
A Latent Variable Recurrent Neural Network for Discourse Relation Language ModelsCode0
Explanation Graph Generation via Generative Pre-training over Synthetic GraphsCode0
Generative Text Modeling through Short Run InferenceCode0
Show:102550
← PrevPage 345 of 353Next →

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1Decay RNNValidation perplexity76.67Unverified
2GRUValidation perplexity53.78Unverified
3LSTMValidation perplexity52.73Unverified
4LSTMTest perplexity48.7Unverified
5Temporal CNNTest perplexity45.2Unverified
6TCNTest perplexity45.19Unverified
7GCNN-8Test perplexity44.9Unverified
8Neural cache model (size = 100)Test perplexity44.8Unverified
9Neural cache model (size = 2,000)Test perplexity40.8Unverified
10GPT-2 SmallTest perplexity37.5Unverified
#ModelMetricClaimedVerifiedStatus
1TCNTest perplexity108.47Unverified
2Seq-U-NetTest perplexity107.95Unverified
3GRU (Bai et al., 2018)Test perplexity92.48Unverified
4R-TransformerTest perplexity84.38Unverified
5Zaremba et al. (2014) - LSTM (medium)Test perplexity82.7Unverified
6Gal & Ghahramani (2016) - Variational LSTM (medium)Test perplexity79.7Unverified
7LSTM (Bai et al., 2018)Test perplexity78.93Unverified
8Zaremba et al. (2014) - LSTM (large)Test perplexity78.4Unverified
9Gal & Ghahramani (2016) - Variational LSTM (large)Test perplexity75.2Unverified
10Inan et al. (2016) - Variational RHNTest perplexity66Unverified
#ModelMetricClaimedVerifiedStatus
1LSTM (7 layers)Bit per Character (BPC)1.67Unverified
2HypernetworksBit per Character (BPC)1.34Unverified
3SHA-LSTM (4 layers, h=1024, no attention head)Bit per Character (BPC)1.33Unverified
4LN HM-LSTMBit per Character (BPC)1.32Unverified
5ByteNetBit per Character (BPC)1.31Unverified
6Recurrent Highway NetworksBit per Character (BPC)1.27Unverified
7Large FS-LSTM-4Bit per Character (BPC)1.25Unverified
8Large mLSTMBit per Character (BPC)1.24Unverified
9AWD-LSTM (3 layers)Bit per Character (BPC)1.23Unverified
10Cluster-Former (#C=512)Bit per Character (BPC)1.22Unverified
#ModelMetricClaimedVerifiedStatus
1Smaller Transformer 126M (pre-trained)Test perplexity33Unverified
2OPT 125MTest perplexity32.26Unverified
3Larger Transformer 771M (pre-trained)Test perplexity28.1Unverified
4OPT 1.3BTest perplexity19.55Unverified
5GPT-Neo 125MTest perplexity17.83Unverified
6OPT 2.7BTest perplexity17.81Unverified
7Smaller Transformer 126M (fine-tuned)Test perplexity12Unverified
8GPT-Neo 1.3BTest perplexity11.46Unverified
9Transformer 125MTest perplexity10.7Unverified
10GPT-Neo 2.7BTest perplexity10.44Unverified