SOTAVerified

Language Modelling

A language model is a model of natural language. Language models are useful for a variety of tasks, including speech recognition, machine translation, natural language generation (generating more human-like text), optical character recognition, route optimization, handwriting recognition, grammar induction, and information retrieval.

Large language models (LLMs), currently their most advanced form, are predominantly based on transformers trained on larger datasets (frequently using words scraped from the public internet). They have superseded recurrent neural network-based models, which had previously superseded the purely statistical models, such as word n-gram language model.

Source: Wikipedia

Papers

Showing 17511800 of 17610 papers

TitleStatusHype
GRENADE: Graph-Centric Language Model for Self-Supervised Representation Learning on Text-Attributed GraphsCode1
GREEK-BERT: The Greeks visiting Sesame StreetCode1
AutoBencher: Creating Salient, Novel, Difficult Datasets for Language ModelsCode1
ALYMPICS: LLM Agents Meet Game Theory -- Exploring Strategic Decision-Making with AI AgentsCode1
Cross-domain Retrieval in the Legal and Patent Domains: a Reproducibility StudyCode1
G-Refer: Graph Retrieval-Augmented Large Language Model for Explainable RecommendationCode1
Cross-Align: Modeling Deep Cross-lingual Interactions for Word AlignmentCode1
Development and bilingual evaluation of Japanese medical large language model within reasonably low computational resourcesCode1
Dialogue Action Tokens: Steering Language Models in Goal-Directed Dialogue with a Multi-Turn PlannerCode1
Grass: Compute Efficient Low-Memory LLM Training with Structured Sparse GradientsCode1
AutoDiff: combining Auto-encoder and Diffusion model for tabular data synthesizingCode1
AutoDIR: Automatic All-in-One Image Restoration with Latent DiffusionCode1
Dialogue Summaries as Dialogue States (DS2), Template-Guided Summarization for Few-shot Dialogue State TrackingCode1
DialoKG: Knowledge-Structure Aware Task-Oriented Dialogue GenerationCode1
DialogVED: A Pre-trained Latent Variable Encoder-Decoder Model for Dialog Response GenerationCode1
Improving Neural Machine Translation Models with Monolingual DataCode1
Cross-Care: Assessing the Healthcare Implications of Pre-training Data on Language Model BiasCode1
Differentiable Model Compression via Pseudo Quantization NoiseCode1
GraPPa: Grammar-Augmented Pre-Training for Table Semantic ParsingCode1
Improving Temporal Generalization of Pre-trained Language Models with Lexical Semantic ChangeCode1
Great Memory, Shallow Reasoning: Limits of kNN-LMsCode1
CDLM: Cross-Document Language ModelingCode1
GraphXForm: Graph transformer for computer-aided molecular designCode1
Differentiable Data Augmentation for Contrastive Sentence Representation LearningCode1
Great Models Think Alike and this Undermines AI OversightCode1
Grounded Compositional Outputs for Adaptive Language ModelingCode1
Differentiable Prompt Makes Pre-trained Language Models Better Few-shot LearnersCode1
Diffusion Sequence Models for Enhanced Protein Representation and GenerationCode1
CriticEval: Evaluating Large Language Model as CriticCode1
CreoPep: A Universal Deep Learning Framework for Target-Specific Peptide Design and OptimizationCode1
Graph Neural Prompting with Large Language ModelsCode1
Differential Privacy for Text Analytics via Natural Text SanitizationCode1
CRE-LLM: A Domain-Specific Chinese Relation Extraction Framework with Fine-tuned Large Language ModelCode1
DiffSDS: A language diffusion model for protein backbone inpainting under geometric conditions and constraintsCode1
GraphLLM: Boosting Graph Reasoning Ability of Large Language ModelCode1
Diffusion Guided Language ModelingCode1
GraphTeam: Facilitating Large Language Model-based Graph Analysis via Multi-Agent CollaborationCode1
AlephBERT:A Hebrew Large Pre-Trained Language Model to Start-off your Hebrew NLP Application WithCode1
CREAM: Consistency Regularized Self-Rewarding Language ModelsCode1
GraphFormers: GNN-nested Transformers for Representation Learning on Textual GraphCode1
Creative Agents: Empowering Agents with Imagination for Creative TasksCode1
ABNIRML: Analyzing the Behavior of Neural IR ModelsCode1
CrAM: A Compression-Aware MinimizerCode1
DISCO: Distilling Counterfactuals with Large Language ModelsCode1
Mask-Predict: Parallel Decoding of Conditional Masked Language ModelsCode1
Direction is what you need: Improving Word Embedding Compression in Large Language ModelsCode1
Critic-Guided Decoding for Controlled Text GenerationCode1
Incorporating Large Language Models into Production Systems for Enhanced Task Automation and FlexibilityCode1
Grounded Multi-Hop VideoQA in Long-Form Egocentric VideosCode1
CPT: A Pre-Trained Unbalanced Transformer for Both Chinese Language Understanding and GenerationCode1
Show:102550
← PrevPage 36 of 353Next →

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1Decay RNNValidation perplexity76.67Unverified
2GRUValidation perplexity53.78Unverified
3LSTMValidation perplexity52.73Unverified
4LSTMTest perplexity48.7Unverified
5Temporal CNNTest perplexity45.2Unverified
6TCNTest perplexity45.19Unverified
7GCNN-8Test perplexity44.9Unverified
8Neural cache model (size = 100)Test perplexity44.8Unverified
9Neural cache model (size = 2,000)Test perplexity40.8Unverified
10GPT-2 SmallTest perplexity37.5Unverified
#ModelMetricClaimedVerifiedStatus
1TCNTest perplexity108.47Unverified
2Seq-U-NetTest perplexity107.95Unverified
3GRU (Bai et al., 2018)Test perplexity92.48Unverified
4R-TransformerTest perplexity84.38Unverified
5Zaremba et al. (2014) - LSTM (medium)Test perplexity82.7Unverified
6Gal & Ghahramani (2016) - Variational LSTM (medium)Test perplexity79.7Unverified
7LSTM (Bai et al., 2018)Test perplexity78.93Unverified
8Zaremba et al. (2014) - LSTM (large)Test perplexity78.4Unverified
9Gal & Ghahramani (2016) - Variational LSTM (large)Test perplexity75.2Unverified
10Inan et al. (2016) - Variational RHNTest perplexity66Unverified
#ModelMetricClaimedVerifiedStatus
1LSTM (7 layers)Bit per Character (BPC)1.67Unverified
2HypernetworksBit per Character (BPC)1.34Unverified
3SHA-LSTM (4 layers, h=1024, no attention head)Bit per Character (BPC)1.33Unverified
4LN HM-LSTMBit per Character (BPC)1.32Unverified
5ByteNetBit per Character (BPC)1.31Unverified
6Recurrent Highway NetworksBit per Character (BPC)1.27Unverified
7Large FS-LSTM-4Bit per Character (BPC)1.25Unverified
8Large mLSTMBit per Character (BPC)1.24Unverified
9AWD-LSTM (3 layers)Bit per Character (BPC)1.23Unverified
10Cluster-Former (#C=512)Bit per Character (BPC)1.22Unverified
#ModelMetricClaimedVerifiedStatus
1Smaller Transformer 126M (pre-trained)Test perplexity33Unverified
2OPT 125MTest perplexity32.26Unverified
3Larger Transformer 771M (pre-trained)Test perplexity28.1Unverified
4OPT 1.3BTest perplexity19.55Unverified
5GPT-Neo 125MTest perplexity17.83Unverified
6OPT 2.7BTest perplexity17.81Unverified
7Smaller Transformer 126M (fine-tuned)Test perplexity12Unverified
8GPT-Neo 1.3BTest perplexity11.46Unverified
9Transformer 125MTest perplexity10.7Unverified
10GPT-Neo 2.7BTest perplexity10.44Unverified