SOTAVerified

Language Modelling

A language model is a model of natural language. Language models are useful for a variety of tasks, including speech recognition, machine translation, natural language generation (generating more human-like text), optical character recognition, route optimization, handwriting recognition, grammar induction, and information retrieval.

Large language models (LLMs), currently their most advanced form, are predominantly based on transformers trained on larger datasets (frequently using words scraped from the public internet). They have superseded recurrent neural network-based models, which had previously superseded the purely statistical models, such as word n-gram language model.

Source: Wikipedia

Papers

Showing 29012950 of 17610 papers

TitleStatusHype
DiffSDS: A language diffusion model for protein backbone inpainting under geometric conditions and constraintsCode1
An Empirical Study of Metrics to Measure Representational Harms in Pre-Trained Language ModelsCode1
Debiasing the Cloze Task in Sequential Recommendation with Bidirectional TransformersCode1
Batch Prompting: Efficient Inference with Large Language Model APIsCode1
tasksource: A Dataset Harmonization Framework for Streamlined NLP Multi-Task Learning and EvaluationCode1
CLIP the Gap: A Single Domain Generalization Approach for Object DetectionCode1
See, Think, Confirm: Interactive Prompting Between Vision and Language Models for Knowledge-based Visual ReasoningCode1
MGeo: Multi-Modal Geographic Pre-Training MethodCode1
Dynamic Grained Encoder for Vision TransformersCode1
You Truly Understand What I Need: Intellectual and Friendly Dialogue Agents grounding Knowledge and PersonaCode1
t-SMILES: A Scalable Fragment-based Molecular Representation Framework for De Novo Molecule GenerationCode1
Large Language Models as Corporate LobbyistsCode1
Analysing Discrete Self Supervised Speech Representation for Spoken Language ModelingCode1
Fusing Pre-Trained Language Models With Multimodal Prompts Through Reinforcement LearningCode1
PODA: Prompt-driven Zero-shot Domain AdaptationCode1
Distilling DETR with Visual-Linguistic Knowledge for Open-Vocabulary Object DetectionCode1
Borrowing Knowledge From Pre-trained Language Model: A New Data-efficient Visual Learning ParadigmCode1
Dual Learning with Dynamic Knowledge Distillation for Partially Relevant Video RetrievalCode1
Rethinking with Retrieval: Faithful Large Language Model InferenceCode1
TeViS:Translating Text Synopses to Video StoryboardsCode1
MicroBERT: Effective Training of Low-resource Monolingual BERTs through Parameter Reduction and Multitask LearningCode1
OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of GeneralizationCode1
Critic-Guided Decoding for Controlled Text GenerationCode1
Zero-shot Triplet Extraction by Template InfillingCode1
Training language models to summarize narratives improves brain alignmentCode1
Does CLIP Bind Concepts? Probing Compositionality in Large Image ModelsCode1
Plug & Play Directed Evolution of Proteins with Gradient-based Discrete MCMCCode1
PLUE: Language Understanding Evaluation Benchmark for Privacy Policies in EnglishCode1
Are Deep Neural Networks SMARTer than Second Graders?Code1
ByGPT5: End-to-End Style-conditioned Poetry Generation with Token-free Language ModelsCode1
DISCO: Distilling Counterfactuals with Large Language ModelsCode1
Toward Human-Like Evaluation for Natural Language Generation with Error AnalysisCode1
PromptBoosting: Black-Box Text Classification with Ten Forward PassesCode1
Large Language Models are Better Reasoners with Self-VerificationCode1
Visconde: Multi-document QA with GPT-3 and Neural RerankingCode1
Evaluating Human-Language Model InteractionCode1
Python Code Generation by Asking Clarification QuestionsCode1
Unnatural Instructions: Tuning Language Models with (Almost) No Human LaborCode1
BLOOM+1: Adding Language Support to BLOOM for Zero-Shot PromptingCode1
Emergent Analogical Reasoning in Large Language ModelsCode1
TextGrad: Advancing Robustness Evaluation in NLP by Gradient-Driven OptimizationCode1
Rethinking the Role of Scale for In-Context Learning: An Interpretability-based Case Study at 66 Billion ScaleCode1
Can Retriever-Augmented Language Models Reason? The Blame Game Between the Retriever and the Language ModelCode1
HyPe: Better Pre-trained Language Model Fine-tuning with Hidden Representation PerturbationCode1
Enhancing Multi-modal and Multi-hop Question Answering via Structured Knowledge and Unified Retrieval-GenerationCode1
Efficient Pre-training of Masked Language Model via Concept-based Curriculum MaskingCode1
Efficient Long Sequence Modeling via State Space Augmented TransformerCode1
On Second Thought, Let's Not Think Step by Step! Bias and Toxicity in Zero-Shot ReasoningCode1
Dataflow Analysis-Inspired Deep Learning for Efficient Vulnerability DetectionCode1
Enhancing Indic Handwritten Text Recognition Using Global Semantic InformationCode1
Show:102550
← PrevPage 59 of 353Next →

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1Decay RNNValidation perplexity76.67Unverified
2GRUValidation perplexity53.78Unverified
3LSTMValidation perplexity52.73Unverified
4LSTMTest perplexity48.7Unverified
5Temporal CNNTest perplexity45.2Unverified
6TCNTest perplexity45.19Unverified
7GCNN-8Test perplexity44.9Unverified
8Neural cache model (size = 100)Test perplexity44.8Unverified
9Neural cache model (size = 2,000)Test perplexity40.8Unverified
10GPT-2 SmallTest perplexity37.5Unverified
#ModelMetricClaimedVerifiedStatus
1TCNTest perplexity108.47Unverified
2Seq-U-NetTest perplexity107.95Unverified
3GRU (Bai et al., 2018)Test perplexity92.48Unverified
4R-TransformerTest perplexity84.38Unverified
5Zaremba et al. (2014) - LSTM (medium)Test perplexity82.7Unverified
6Gal & Ghahramani (2016) - Variational LSTM (medium)Test perplexity79.7Unverified
7LSTM (Bai et al., 2018)Test perplexity78.93Unverified
8Zaremba et al. (2014) - LSTM (large)Test perplexity78.4Unverified
9Gal & Ghahramani (2016) - Variational LSTM (large)Test perplexity75.2Unverified
10Inan et al. (2016) - Variational RHNTest perplexity66Unverified
#ModelMetricClaimedVerifiedStatus
1LSTM (7 layers)Bit per Character (BPC)1.67Unverified
2HypernetworksBit per Character (BPC)1.34Unverified
3SHA-LSTM (4 layers, h=1024, no attention head)Bit per Character (BPC)1.33Unverified
4LN HM-LSTMBit per Character (BPC)1.32Unverified
5ByteNetBit per Character (BPC)1.31Unverified
6Recurrent Highway NetworksBit per Character (BPC)1.27Unverified
7Large FS-LSTM-4Bit per Character (BPC)1.25Unverified
8Large mLSTMBit per Character (BPC)1.24Unverified
9AWD-LSTM (3 layers)Bit per Character (BPC)1.23Unverified
10Cluster-Former (#C=512)Bit per Character (BPC)1.22Unverified
#ModelMetricClaimedVerifiedStatus
1Smaller Transformer 126M (pre-trained)Test perplexity33Unverified
2OPT 125MTest perplexity32.26Unverified
3Larger Transformer 771M (pre-trained)Test perplexity28.1Unverified
4OPT 1.3BTest perplexity19.55Unverified
5GPT-Neo 125MTest perplexity17.83Unverified
6OPT 2.7BTest perplexity17.81Unverified
7Smaller Transformer 126M (fine-tuned)Test perplexity12Unverified
8GPT-Neo 1.3BTest perplexity11.46Unverified
9Transformer 125MTest perplexity10.7Unverified
10GPT-Neo 2.7BTest perplexity10.44Unverified