SOTAVerified

Language Modelling

A language model is a model of natural language. Language models are useful for a variety of tasks, including speech recognition, machine translation, natural language generation (generating more human-like text), optical character recognition, route optimization, handwriting recognition, grammar induction, and information retrieval.

Large language models (LLMs), currently their most advanced form, are predominantly based on transformers trained on larger datasets (frequently using words scraped from the public internet). They have superseded recurrent neural network-based models, which had previously superseded the purely statistical models, such as word n-gram language model.

Source: Wikipedia

Papers

Showing 1635116400 of 17610 papers

TitleStatusHype
Distinguishing Non-natural from Natural Adversarial Samples for More Robust Pre-trained Language ModelCode0
Finding Syntactic Representations in Neural StacksCode0
A Multi-Pass Large Language Model Framework for Precise and Efficient Radiology Report Error DetectionCode0
IFShip: Interpretable Fine-grained Ship Classification with Domain Knowledge-Enhanced Vision-Language ModelsCode0
IGA : An Intent-Guided Authoring AssistantCode0
Classifier-guided CLIP Distillation for Unsupervised Multi-label ClassificationCode0
Distinguishability Calibration to In-Context LearningCode0
Dynamic Pyramid Network for Efficient Multimodal Large Language ModelCode0
Distilling Word Meaning in Context from Pre-trained Language ModelsCode0
Evaluating Online Continual Learning with CALMCode0
GrowOVER: How Can LLMs Adapt to Growing Real-World Knowledge?Code0
CLAIR-A: Leveraging Large Language Models to Judge Audio CaptionsCode0
A Survey on Large Language Model based Human-Agent SystemsCode0
FineDeb: A Debiasing Framework for Language ModelsCode0
Dynamic Word EmbeddingsCode0
Claim Optimization in Computational ArgumentationCode0
Distilling Knowledge Learned in BERT for Text GenerationCode0
Distilling Monolingual and Crosslingual Word-in-Context RepresentationsCode0
Distilling Large Language Models using Skill-Occupation Graph Context for HR-Related TasksCode0
Fine-Grained Behavior Simulation with Role-Playing Large Language Model on Social MediaCode0
E2S2: Encoding-Enhanced Sequence-to-Sequence Pretraining for Language Understanding and GenerationCode0
E2TP: Element to Tuple Prompting Improves Aspect Sentiment Tuple PredictionCode0
G-Safeguard: A Topology-Guided Security Lens and Treatment on LLM-based Multi-agent SystemsCode0
Fine-grained Contrastive Learning for Relation ExtractionCode0
Distilling ChatGPT for Explainable Automated Student Answer AssessmentCode0
Fine-Grained Emotion Prediction by Modeling Emotion DefinitionsCode0
G-SciEdBERT: A Contextualized LLM for Science Assessment Tasks in GermanCode0
CItruS: Chunked Instruction-aware State Eviction for Long Sequence ModelingCode0
IPOD: An Industrial and Professional Occupations Dataset and its Applications to Occupational Data Mining and AnalysisCode0
IgboBERT Models: Building and Training Transformer Models for the Igbo LanguageCode0
GTA: Gated Toxicity Avoidance for LM Performance PreservationCode0
Distantly-Supervised Joint Extraction with Noise-Robust LearningCode0
Dissecting vocabulary biases datasets through statistical testing and automated data augmentation for artifact mitigation in Natural Language InferenceCode0
A Glitch in the Matrix? Locating and Detecting Language Model Grounding with FakepediaCode0
Dissecting Deep Metric Learning Losses for Image-Text RetrievalCode0
Dispatcher: A Message-Passing Approach To Language ModellingCode0
EAT: Enhanced ASR-TTS for Self-supervised Speech RecognitionCode0
A character-based steganography using masked language modelingCode0
Improving LLM Unlearning Robustness via Random PerturbationsCode0
CitePrompt: Using Prompts to Identify Citation Intent in Scientific PapersCode0
Improving the Sample Efficiency of Prompt Tuning with Domain AdaptationCode0
Circuit Stability Characterizes Language Model GeneralizationCode0
Disentangling Logic: The Role of Context in Large Language Model Reasoning CapabilitiesCode0
A Multimodal Approach For Endoscopic VCE Image Classification Using BiomedCLIP-PubMedBERTCode0
Disentangling and Integrating Relational and Sensory Information in Transformer ArchitecturesCode0
Discriminative Policy Optimization for Token-Level Reward ModelsCode0
CILP-FGDI: Exploiting Vision-Language Model for Generalizable Person Re-IdentificationCode0
Activations and Gradients Compression for Model-Parallel TrainingCode0
A Survey for Biomedical Text Summarization: From Pre-trained to Large Language ModelsCode0
EchoNarrator: Generating natural text explanations for ejection fraction predictionsCode0
Show:102550
← PrevPage 328 of 353Next →

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1Decay RNNValidation perplexity76.67Unverified
2GRUValidation perplexity53.78Unverified
3LSTMValidation perplexity52.73Unverified
4LSTMTest perplexity48.7Unverified
5Temporal CNNTest perplexity45.2Unverified
6TCNTest perplexity45.19Unverified
7GCNN-8Test perplexity44.9Unverified
8Neural cache model (size = 100)Test perplexity44.8Unverified
9Neural cache model (size = 2,000)Test perplexity40.8Unverified
10GPT-2 SmallTest perplexity37.5Unverified
#ModelMetricClaimedVerifiedStatus
1TCNTest perplexity108.47Unverified
2Seq-U-NetTest perplexity107.95Unverified
3GRU (Bai et al., 2018)Test perplexity92.48Unverified
4R-TransformerTest perplexity84.38Unverified
5Zaremba et al. (2014) - LSTM (medium)Test perplexity82.7Unverified
6Gal & Ghahramani (2016) - Variational LSTM (medium)Test perplexity79.7Unverified
7LSTM (Bai et al., 2018)Test perplexity78.93Unverified
8Zaremba et al. (2014) - LSTM (large)Test perplexity78.4Unverified
9Gal & Ghahramani (2016) - Variational LSTM (large)Test perplexity75.2Unverified
10Inan et al. (2016) - Variational RHNTest perplexity66Unverified
#ModelMetricClaimedVerifiedStatus
1LSTM (7 layers)Bit per Character (BPC)1.67Unverified
2HypernetworksBit per Character (BPC)1.34Unverified
3SHA-LSTM (4 layers, h=1024, no attention head)Bit per Character (BPC)1.33Unverified
4LN HM-LSTMBit per Character (BPC)1.32Unverified
5ByteNetBit per Character (BPC)1.31Unverified
6Recurrent Highway NetworksBit per Character (BPC)1.27Unverified
7Large FS-LSTM-4Bit per Character (BPC)1.25Unverified
8Large mLSTMBit per Character (BPC)1.24Unverified
9AWD-LSTM (3 layers)Bit per Character (BPC)1.23Unverified
10Cluster-Former (#C=512)Bit per Character (BPC)1.22Unverified
#ModelMetricClaimedVerifiedStatus
1Smaller Transformer 126M (pre-trained)Test perplexity33Unverified
2OPT 125MTest perplexity32.26Unverified
3Larger Transformer 771M (pre-trained)Test perplexity28.1Unverified
4OPT 1.3BTest perplexity19.55Unverified
5GPT-Neo 125MTest perplexity17.83Unverified
6OPT 2.7BTest perplexity17.81Unverified
7Smaller Transformer 126M (fine-tuned)Test perplexity12Unverified
8GPT-Neo 1.3BTest perplexity11.46Unverified
9Transformer 125MTest perplexity10.7Unverified
10GPT-Neo 2.7BTest perplexity10.44Unverified