SOTAVerified

Language Modelling

A language model is a model of natural language. Language models are useful for a variety of tasks, including speech recognition, machine translation, natural language generation (generating more human-like text), optical character recognition, route optimization, handwriting recognition, grammar induction, and information retrieval.

Large language models (LLMs), currently their most advanced form, are predominantly based on transformers trained on larger datasets (frequently using words scraped from the public internet). They have superseded recurrent neural network-based models, which had previously superseded the purely statistical models, such as word n-gram language model.

Source: Wikipedia

Papers

Showing 53515400 of 17610 papers

TitleStatusHype
Automated Validation of Textual Constraints Against AutomationML via LLMs and SHACLCode0
Baseline: A Library for Rapid Modeling, Experimentation and Development of Deep Learning Algorithms targeting NLPCode0
A Framework for Adapting Human-Robot Interaction to Diverse User GroupsCode0
Intent Factored Generation: Unleashing the Diversity in Your Language ModelCode0
Intention Recognition in Real-Time Interactive Navigation MapsCode0
Efficient Inference for Large Language Model-based Generative RecommendationCode0
BatchPrompt: Accomplish more with lessCode0
Coding Textual Inputs Boosts the Accuracy of Neural NetworksCode0
Efficient Language Model Training through Cross-Lingual and Progressive Transfer LearningCode0
Efficient Large-Scale Language Model Training on GPU Clusters Using Megatron-LMCode0
CoF: Coarse to Fine-Grained Image Understanding for Multi-modal Large Language ModelsCode0
A Reality Check on Context Utilisation for Retrieval-Augmented GenerationCode0
CogALex-VI Shared Task: Transrelation - A Robust Multilingual Language Model for Multilingual Relation IdentificationCode0
Internal Causal Mechanisms Robustly Predict Language Model Out-of-Distribution BehaviorsCode0
Cognate Transformer for Automated Phonological Reconstruction and Cognate Reflex PredictionCode0
InternLM-XComposer2-4KHD: A Pioneering Large Vision-Language Model Handling Resolutions from 336 Pixels to 4K HDCode0
Efficient Machine Translation Domain AdaptationCode0
Efficient Medical Question Answering with Knowledge-Augmented Question GenerationCode0
Interpretable and Low-Resource Entity Matching via Decoupling Feature Learning from Decision MakingCode0
Interpretable-by-Design Text Understanding with Iteratively Generated Concept BottleneckCode0
Interpretable Word Sense Representations via Definition Generation: The Case of Semantic Change AnalysisCode0
Interpreting Biomedical VLMs on High-Imbalance Out-of-Distributions: An Insight into BiomedCLIP on RadiologyCode0
Interpreting Large Text-to-Image Diffusion Models with Dictionary LearningCode0
A Recurrent BERT-based Model for Question GenerationCode0
Efficient Prompt Tuning of Large Vision-Language Model for Fine-Grained Ship ClassificationCode0
A Melody-Conditioned Lyrics Language ModelCode0
Interweaving Memories of a Siamese Large Language ModelCode0
Bayesian Neural Network Language Modeling for Speech RecognitionCode0
Into the crossfire: evaluating the use of a language model to crowdsource gun violence reportsCode0
Into the Unknown: Generating Geospatial Descriptions for New EnvironmentsCode0
Intra-Layer Recurrence in Transformers for Language ModelingCode0
Intrinsic evaluation of language models for code-switchingCode0
ColBERT Retrieval and Ensemble Response Scoring for Language Model Question AnsweringCode0
Introducing Aspects of Creativity in Automatic Poetry GenerationCode0
Bayesian Parameter-Efficient Fine-Tuning for Overcoming Catastrophic ForgettingCode0
Intruding with Words: Towards Understanding Graph Injection Attacks at the Text LevelCode0
Invalidator: Automated Patch Correctness Assessment via Semantic and Syntactic ReasoningCode0
Spurious Feature Eraser: Stabilizing Test-Time Adaptation for Vision-Language Foundation ModelCode0
Collaborative Development of NLP modelsCode0
EgyBERT: A Large Language Model Pretrained on Egyptian Dialect CorporaCode0
Investigating and Extending Homans' Social Exchange Theory with Large Language Model based AgentsCode0
Investigating and Scaling up Code-Switching for Multilingual Language Model Pre-TrainingCode0
Collaborative Stance Detection via Small-Large Language Model Consistency VerificationCode0
AdaPlus: Integrating Nesterov Momentum and Precise Stepsize Adjustment on AdamW BasisCode0
EIT: Enhanced Interactive TransformerCode0
Elaborative Subtopic Query Reformulation for Broad and Indirect Queries in Travel Destination RecommendationCode0
Electoral Agitation Data Set: The Use Case of the Polish ElectionCode0
ELECTRA is a Zero-Shot Learner, TooCode0
Investigating the Impact of Data Selection Strategies on Language Model PerformanceCode0
Elevating Code-mixed Text Handling through Auditory Information of WordsCode0
Show:102550
← PrevPage 108 of 353Next →

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1Decay RNNValidation perplexity76.67Unverified
2GRUValidation perplexity53.78Unverified
3LSTMValidation perplexity52.73Unverified
4LSTMTest perplexity48.7Unverified
5Temporal CNNTest perplexity45.2Unverified
6TCNTest perplexity45.19Unverified
7GCNN-8Test perplexity44.9Unverified
8Neural cache model (size = 100)Test perplexity44.8Unverified
9Neural cache model (size = 2,000)Test perplexity40.8Unverified
10GPT-2 SmallTest perplexity37.5Unverified
#ModelMetricClaimedVerifiedStatus
1TCNTest perplexity108.47Unverified
2Seq-U-NetTest perplexity107.95Unverified
3GRU (Bai et al., 2018)Test perplexity92.48Unverified
4R-TransformerTest perplexity84.38Unverified
5Zaremba et al. (2014) - LSTM (medium)Test perplexity82.7Unverified
6Gal & Ghahramani (2016) - Variational LSTM (medium)Test perplexity79.7Unverified
7LSTM (Bai et al., 2018)Test perplexity78.93Unverified
8Zaremba et al. (2014) - LSTM (large)Test perplexity78.4Unverified
9Gal & Ghahramani (2016) - Variational LSTM (large)Test perplexity75.2Unverified
10Inan et al. (2016) - Variational RHNTest perplexity66Unverified
#ModelMetricClaimedVerifiedStatus
1LSTM (7 layers)Bit per Character (BPC)1.67Unverified
2HypernetworksBit per Character (BPC)1.34Unverified
3SHA-LSTM (4 layers, h=1024, no attention head)Bit per Character (BPC)1.33Unverified
4LN HM-LSTMBit per Character (BPC)1.32Unverified
5ByteNetBit per Character (BPC)1.31Unverified
6Recurrent Highway NetworksBit per Character (BPC)1.27Unverified
7Large FS-LSTM-4Bit per Character (BPC)1.25Unverified
8Large mLSTMBit per Character (BPC)1.24Unverified
9AWD-LSTM (3 layers)Bit per Character (BPC)1.23Unverified
10Cluster-Former (#C=512)Bit per Character (BPC)1.22Unverified
#ModelMetricClaimedVerifiedStatus
1Smaller Transformer 126M (pre-trained)Test perplexity33Unverified
2OPT 125MTest perplexity32.26Unverified
3Larger Transformer 771M (pre-trained)Test perplexity28.1Unverified
4OPT 1.3BTest perplexity19.55Unverified
5GPT-Neo 125MTest perplexity17.83Unverified
6OPT 2.7BTest perplexity17.81Unverified
7Smaller Transformer 126M (fine-tuned)Test perplexity12Unverified
8GPT-Neo 1.3BTest perplexity11.46Unverified
9Transformer 125MTest perplexity10.7Unverified
10GPT-Neo 2.7BTest perplexity10.44Unverified