SOTAVerified

Language Modelling

A language model is a model of natural language. Language models are useful for a variety of tasks, including speech recognition, machine translation, natural language generation (generating more human-like text), optical character recognition, route optimization, handwriting recognition, grammar induction, and information retrieval.

Large language models (LLMs), currently their most advanced form, are predominantly based on transformers trained on larger datasets (frequently using words scraped from the public internet). They have superseded recurrent neural network-based models, which had previously superseded the purely statistical models, such as word n-gram language model.

Source: Wikipedia

Papers

Showing 1735117400 of 17610 papers

TitleStatusHype
An Empirical Study of Language CNN for Image CaptioningCode0
Improving LSTM-based Video Description with Linguistic Knowledge Mined from TextCode0
Scaling Trends in Language Model RobustnessCode0
Exploring Iterative Enhancement for Improving Learnersourced Multiple-Choice Question Explanations with Large Language ModelsCode0
Barack's Wife Hillary: Using Knowledge Graphs for Fact-Aware Language ModelingCode0
Barack's Wife Hillary: Using Knowledge-Graphs for Fact-Aware Language ModelingCode0
Bangla Grammatical Error Detection Using T5 Transformer ModelCode0
Balancing Speciality and Versatility: a Coarse to Fine Framework for Supervised Fine-tuning Large Language ModelCode0
Balancing Label Quantity and Quality for Scalable ElicitationCode0
BADGE: BADminton report Generation and Evaluation with LLMCode0
ConQueR: Contextualized Query Reduction using Search LogsCode0
Improving Machine Reading Comprehension with General Reading StrategiesCode0
AdaptVision: Dynamic Input Scaling in MLLMs for Versatile Scene UnderstandingCode0
Babysit A Language Model From Scratch: Interactive Language Learning by Trials and DemonstrationsCode0
Global Autoregressive Models for Data-Efficient Sequence LearningCode0
Exploring the Design Space of Visual Context Representation in Video MLLMsCode0
Exploring the Effectiveness of Multi-stage Fine-tuning for Cross-encoder Re-rankersCode0
Global Constraints with Prompting for Zero-Shot Event Argument ClassificationCode0
Problematic Tokens: Tokenizer Bias in Large Language ModelsCode0
A Zero-Shot LLM Framework for Automatic Assignment Grading in Higher EducationCode0
ConnPrompt: Connective-cloze Prompt Learning for Implicit Discourse Relation RecognitionCode0
A Language Model of Java Methods with Train/Test DeduplicationCode0
Exploring the Landscape for Generative Sequence Models for Specialized Data SynthesisCode0
Connections between Schedule-Free Optimizers, AdEMAMix, and Accelerated SGD VariantsCode0
AxomiyaBERTa: A Phonologically-aware Transformer Model for AssameseCode0
A Language Model for Spell Checking of Educational Texts in Kurdish (Sorani)Code0
Improving Medical Multi-modal Contrastive Learning with Expert AnnotationsCode0
On the Relationship between Sentence Analogy Identification and Sentence Structure Encoding in Large Language ModelsCode0
Is attention required for ICL? Exploring the Relationship Between Model Architecture and In-Context Learning AbilityCode0
Exploring the Reliability of Self-explanation and its Relationship with Classification in Language Model-driven Financial AnalysisCode0
Adapt or Get Left Behind: Domain Adaptation through BERT Language Model Finetuning for Aspect-Target Sentiment ClassificationCode0
Transformers on Multilingual Clause-Level MorphologyCode0
Exploring the Syntactic Abilities of RNNs with Multi-task LearningCode0
Intruding with Words: Towards Understanding Graph Injection Attacks at the Text LevelCode0
Infinity Parser: Layout Aware Reinforcement Learning for Scanned Document ParsingCode0
A Language-Guided Benchmark for Weakly Supervised Open Vocabulary Semantic SegmentationCode0
AX-MABSA: A Framework for Extremely Weakly Supervised Multi-label Aspect Based Sentiment AnalysisCode0
Adaptive User Modeling with Long and Short-Term Preferences for Personalized RecommendationCode0
A Weakly Supervised Dataset of Fine-Grained Emotions in PortugueseCode0
Exploring the zero-shot limit of FewRelCode0
Exploring Transformer ExtrapolationCode0
Exploring Unsupervised Pretraining Objectives for Machine TranslationCode0
Exploring User Retrieval Integration towards Large Language Models for Cross-Domain Sequential RecommendationCode0
ConMe: Rethinking Evaluation of Compositional Reasoning for Modern VLMsCode0
Avoiding exp(R_max) scaling in RLHF through Preference-based ExplorationCode0
Exploring Weight Symmetry in Deep Neural NetworksCode0
An Empirical Study and Analysis of Text-to-Image Generation Using Large Language Model-Powered Textual RepresentationCode0
Avoiding Copyright Infringement via Large Language Model UnlearningCode0
Confidential Prompting: Protecting User Prompts from Cloud LLM ProvidersCode0
Glyce: Glyph-vectors for Chinese Character RepresentationsCode0
Show:102550
← PrevPage 348 of 353Next →

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1Decay RNNValidation perplexity76.67Unverified
2GRUValidation perplexity53.78Unverified
3LSTMValidation perplexity52.73Unverified
4LSTMTest perplexity48.7Unverified
5Temporal CNNTest perplexity45.2Unverified
6TCNTest perplexity45.19Unverified
7GCNN-8Test perplexity44.9Unverified
8Neural cache model (size = 100)Test perplexity44.8Unverified
9Neural cache model (size = 2,000)Test perplexity40.8Unverified
10GPT-2 SmallTest perplexity37.5Unverified
#ModelMetricClaimedVerifiedStatus
1TCNTest perplexity108.47Unverified
2Seq-U-NetTest perplexity107.95Unverified
3GRU (Bai et al., 2018)Test perplexity92.48Unverified
4R-TransformerTest perplexity84.38Unverified
5Zaremba et al. (2014) - LSTM (medium)Test perplexity82.7Unverified
6Gal & Ghahramani (2016) - Variational LSTM (medium)Test perplexity79.7Unverified
7LSTM (Bai et al., 2018)Test perplexity78.93Unverified
8Zaremba et al. (2014) - LSTM (large)Test perplexity78.4Unverified
9Gal & Ghahramani (2016) - Variational LSTM (large)Test perplexity75.2Unverified
10Inan et al. (2016) - Variational RHNTest perplexity66Unverified
#ModelMetricClaimedVerifiedStatus
1LSTM (7 layers)Bit per Character (BPC)1.67Unverified
2HypernetworksBit per Character (BPC)1.34Unverified
3SHA-LSTM (4 layers, h=1024, no attention head)Bit per Character (BPC)1.33Unverified
4LN HM-LSTMBit per Character (BPC)1.32Unverified
5ByteNetBit per Character (BPC)1.31Unverified
6Recurrent Highway NetworksBit per Character (BPC)1.27Unverified
7Large FS-LSTM-4Bit per Character (BPC)1.25Unverified
8Large mLSTMBit per Character (BPC)1.24Unverified
9AWD-LSTM (3 layers)Bit per Character (BPC)1.23Unverified
10Cluster-Former (#C=512)Bit per Character (BPC)1.22Unverified
#ModelMetricClaimedVerifiedStatus
1Smaller Transformer 126M (pre-trained)Test perplexity33Unverified
2OPT 125MTest perplexity32.26Unverified
3Larger Transformer 771M (pre-trained)Test perplexity28.1Unverified
4OPT 1.3BTest perplexity19.55Unverified
5GPT-Neo 125MTest perplexity17.83Unverified
6OPT 2.7BTest perplexity17.81Unverified
7Smaller Transformer 126M (fine-tuned)Test perplexity12Unverified
8GPT-Neo 1.3BTest perplexity11.46Unverified
9Transformer 125MTest perplexity10.7Unverified
10GPT-Neo 2.7BTest perplexity10.44Unverified