SOTAVerified

Language Modelling

A language model is a model of natural language. Language models are useful for a variety of tasks, including speech recognition, machine translation, natural language generation (generating more human-like text), optical character recognition, route optimization, handwriting recognition, grammar induction, and information retrieval.

Large language models (LLMs), currently their most advanced form, are predominantly based on transformers trained on larger datasets (frequently using words scraped from the public internet). They have superseded recurrent neural network-based models, which had previously superseded the purely statistical models, such as word n-gram language model.

Source: Wikipedia

Papers

Showing 52515300 of 17610 papers

TitleStatusHype
Dynamic Pyramid Network for Efficient Multimodal Large Language ModelCode0
Improving Transformer Models by Reordering their SublayersCode0
A Zero-Shot LLM Framework for Automatic Assignment Grading in Higher EducationCode0
Closing the Loop: Learning to Generate Writing Feedback via Language Model Simulated Student RevisionsCode0
Improving Variational Autoencoder for Text Modelling with Timestep-Wise RegularisationCode0
Improving Variational Autoencoders with Density Gap-based RegularizationCode0
AlphaZip: Neural Network-Enhanced Lossless Text CompressionCode0
Accessible Smart Contracts Verification: Synthesizing Formal Models with Tamed LLMsCode0
Dynamic Word EmbeddingsCode0
INA: An Integrative Approach for Enhancing Negotiation Strategies with Reward-Based Dialogue SystemCode0
E2S2: Encoding-Enhanced Sequence-to-Sequence Pretraining for Language Understanding and GenerationCode0
In BLOOM: Creativity and Affinity in Artificial Lyrics and ArtCode0
E2TP: Element to Tuple Prompting Improves Aspect Sentiment Tuple PredictionCode0
In-context Examples Selection for Machine TranslationCode0
Self-supervised Multi-modal Training from Uncurated Image and Reports Enables Zero-shot Oversight Artificial Intelligence in RadiologyCode0
Babysit A Language Model From Scratch: Interactive Language Learning by Trials and DemonstrationsCode0
A Quantum Many-body Wave Function Inspired Language Modeling ApproachCode0
In-Context Learning through the Bayesian PrismCode0
A Quick, trustworthy spectral knowledge Q&A system leveraging retrieval-augmented generation on LLMCode0
EAT: Enhanced ASR-TTS for Self-supervised Speech RecognitionCode0
Incorporating Graph Attention Mechanism into Geometric Problem Solving Based on Deep Reinforcement LearningCode0
Alternating Synthetic and Real Gradients for Neural Language ModelingCode0
On the Usefulness of Embeddings, Clusters and Strings for Text Generator EvaluationCode0
Alternative structures for character-level RNNsCode0
Increasing Learning Efficiency of Self-Attention Networks through Direct Position Interactions, Learnable Temperature, and Convoluted AttentionCode0
Alternative Weighting Schemes for ELMo EmbeddingsCode0
Increasing The Performance of Cognitively Inspired Data-Efficient Language Models via Implicit Structure BuildingCode0
EchoNarrator: Generating natural text explanations for ejection fraction predictionsCode0
Clustering of Deep Contextualized Representations for Summarization of Biomedical TextsCode0
BADGE: BADminton report Generation and Evaluation with LLMCode0
Incremental Neural Lexical Coherence ModelingCode0
ECOL: Early Detection of COVID Lies Using Content, Prior Knowledge and Source InformationCode0
Incremental Processing in the Age of Non-Incremental Encoders: An Empirical Assessment of Bidirectional Models for Incremental NLUCode0
Arabic Synonym BERT-based Adversarial Examples for Text ClassificationCode0
Independently Recurrent Neural Network (IndRNN): Building A Longer and Deeper RNNCode0
ED-Copilot: Reduce Emergency Department Wait Time with Language Model Diagnostic AssistanceCode0
ArabicTransformer: Efficient Large Arabic Language Model with Funnel Transformer and ELECTRA ObjectiveCode0
Indic-Transformers: An Analysis of Transformer Language Models for Indian LanguagesCode0
WikiCREM: A Large Unsupervised Corpus for Coreference ResolutionCode0
Edinburgh Clinical NLP at SemEval-2024 Task 2: Fine-tune your model unless you have access to GPT-4Code0
Toward a Deeper Understanding: RetNet Viewed through ConvolutionCode0
Induced Model Matching: How Restricted Models Can Help Larger OnesCode0
Induced Model Matching: Restricted Models Help Train Full-Featured ModelsCode0
Induced Natural Language Rationales and Interleaved Markup Tokens Enable Extrapolation in Large Language ModelsCode0
Inducing brain-relevant bias in natural language processing modelsCode0
Edisum: Summarizing and Explaining Wikipedia Edits at ScaleCode0
Coarse-to-Fine Memory Matching for Joint Retrieval and ClassificationCode0
Inductive-bias Learning: Generating Code Models with Large Language ModelCode0
Accommodating Audio Modality in CLIP for Multimodal ProcessingCode0
CoAuthor: Designing a Human-AI Collaborative Writing Dataset for Exploring Language Model CapabilitiesCode0
Show:102550
← PrevPage 106 of 353Next →

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1Decay RNNValidation perplexity76.67Unverified
2GRUValidation perplexity53.78Unverified
3LSTMValidation perplexity52.73Unverified
4LSTMTest perplexity48.7Unverified
5Temporal CNNTest perplexity45.2Unverified
6TCNTest perplexity45.19Unverified
7GCNN-8Test perplexity44.9Unverified
8Neural cache model (size = 100)Test perplexity44.8Unverified
9Neural cache model (size = 2,000)Test perplexity40.8Unverified
10GPT-2 SmallTest perplexity37.5Unverified
#ModelMetricClaimedVerifiedStatus
1TCNTest perplexity108.47Unverified
2Seq-U-NetTest perplexity107.95Unverified
3GRU (Bai et al., 2018)Test perplexity92.48Unverified
4R-TransformerTest perplexity84.38Unverified
5Zaremba et al. (2014) - LSTM (medium)Test perplexity82.7Unverified
6Gal & Ghahramani (2016) - Variational LSTM (medium)Test perplexity79.7Unverified
7LSTM (Bai et al., 2018)Test perplexity78.93Unverified
8Zaremba et al. (2014) - LSTM (large)Test perplexity78.4Unverified
9Gal & Ghahramani (2016) - Variational LSTM (large)Test perplexity75.2Unverified
10Inan et al. (2016) - Variational RHNTest perplexity66Unverified
#ModelMetricClaimedVerifiedStatus
1LSTM (7 layers)Bit per Character (BPC)1.67Unverified
2HypernetworksBit per Character (BPC)1.34Unverified
3SHA-LSTM (4 layers, h=1024, no attention head)Bit per Character (BPC)1.33Unverified
4LN HM-LSTMBit per Character (BPC)1.32Unverified
5ByteNetBit per Character (BPC)1.31Unverified
6Recurrent Highway NetworksBit per Character (BPC)1.27Unverified
7Large FS-LSTM-4Bit per Character (BPC)1.25Unverified
8Large mLSTMBit per Character (BPC)1.24Unverified
9AWD-LSTM (3 layers)Bit per Character (BPC)1.23Unverified
10Cluster-Former (#C=512)Bit per Character (BPC)1.22Unverified
#ModelMetricClaimedVerifiedStatus
1Smaller Transformer 126M (pre-trained)Test perplexity33Unverified
2OPT 125MTest perplexity32.26Unverified
3Larger Transformer 771M (pre-trained)Test perplexity28.1Unverified
4OPT 1.3BTest perplexity19.55Unverified
5GPT-Neo 125MTest perplexity17.83Unverified
6OPT 2.7BTest perplexity17.81Unverified
7Smaller Transformer 126M (fine-tuned)Test perplexity12Unverified
8GPT-Neo 1.3BTest perplexity11.46Unverified
9Transformer 125MTest perplexity10.7Unverified
10GPT-Neo 2.7BTest perplexity10.44Unverified