SOTAVerified

Language Modelling

A language model is a model of natural language. Language models are useful for a variety of tasks, including speech recognition, machine translation, natural language generation (generating more human-like text), optical character recognition, route optimization, handwriting recognition, grammar induction, and information retrieval.

Large language models (LLMs), currently their most advanced form, are predominantly based on transformers trained on larger datasets (frequently using words scraped from the public internet). They have superseded recurrent neural network-based models, which had previously superseded the purely statistical models, such as word n-gram language model.

Source: Wikipedia

Papers

Showing 1665116700 of 17610 papers

TitleStatusHype
A Simple Baseline for Predicting Events with Auto-Regressive Tabular TransformersCode0
Formulating Few-shot Fine-tuning Towards Language Model Pre-training: A Pilot Study on Named Entity RecognitionCode0
FOSI: Hybrid First and Second Order OptimizationCode0
StructEval: Deepen and Broaden Large Language Model Assessment via Structured EvaluationCode0
Deep Independently Recurrent Neural Network (IndRNN)Code0
EMMA-500: Enhancing Massively Multilingual Adaptation of Large Language ModelsCode0
Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed TrainingCode0
Can We Trust the Performance Evaluation of Uncertainty Estimation Methods in Text Summarization?Code0
Deep Gradient Compression Reduce the Communication Bandwidth For distributed TraningCode0
Deep-FSMN for Large Vocabulary Continuous Speech RecognitionCode0
In-context Examples Selection for Machine TranslationCode0
Deeper Text Understanding for IR with Contextual Neural Language ModelingCode0
Emoji Prediction in Tweets using BERTCode0
EmoNews: A Spoken Dialogue System for Expressive News ConversationsCode0
Foundations of Large Language Model Compression -- Part 1: Weight QuantizationCode0
Emotional Neural Language Generation Grounded in Situational ContextsCode0
Can We Learn Communication-Efficient Optimizers?Code0
Health Text Simplification: An Annotated Corpus for Digestive Cancer Education and Novel Strategies for Reinforcement LearningCode0
DeepArt: A Benchmark to Advance Fidelity Research in AI-Generated ContentCode0
Heaps' Law in GPT-Neo Large Language Model Emulated CorporaCode0
Can We Guide a Multi-Hop Reasoning Language Model to Incrementally Learn at Each Single-Hop?Code0
Interpretable-by-Design Text Understanding with Iteratively Generated Concept BottleneckCode0
EmotionGIF-Yankee: A Sentiment Classifier with Robust Model Based Ensemble MethodsCode0
Alternating Synthetic and Real Gradients for Neural Language ModelingCode0
Emotion-Infused Models for Explainable Psychological Stress DetectionCode0
EmotionX-IDEA: Emotion BERT -- an Affectional Model for ConversationCode0
EmotionX-KU: BERT-Max based Contextual Emotion ClassifierCode0
Can Unsupervised Knowledge Transfer from Social Discussions Help Argument Mining?Code0
FPT: Feature Prompt Tuning for Few-shot Readability AssessmentCode0
Affective-NLI: Towards Accurate and Interpretable Personality Recognition in ConversationCode0
Empathic Grounding: Explorations using Multimodal Interaction and Large Language Models with Conversational AgentsCode0
“It doesn’t look good for a date”: Transforming Critiques into Preferences for Conversational Recommendation SystemsCode0
Implicit Deep Latent Variable Models for Text GenerationCode0
FRAGE: Frequency-Agnostic Word RepresentationCode0
Can Textual Semantics Mitigate Sounding Object Segmentation Preference?Code0
Empirical Error Modeling Improves Robustness of Noisy Neural Sequence LabelingCode0
TAGPRIME: A Unified Framework for Relational Structure ExtractionCode0
Empirical Sufficiency Lower Bounds for Language Modeling with Locally-Bootstrapped Semantic StructuresCode0
Implicit Language Model in LSTM for OCRCode0
Fraternal DropoutCode0
DEEPAGÉ: Answering Questions in Portuguese about the Brazilian EnvironmentCode0
Self-supervised Multi-modal Training from Uncurated Image and Reports Enables Zero-shot Oversight Artificial Intelligence in RadiologyCode0
Canonical and Surface Morphological Segmentation for Nguni LanguagesCode0
A Sentiment-annotated Dataset of English Causal ConnectivesCode0
Can LLM-Augmented autonomous agents cooperate?, An evaluation of their cooperative capabilities through Melting PotCode0
A Comparison of Large Language Model and Human Performance on Random Number Generation TasksCode0
DE-COP: Detecting Copyrighted Content in Language Models Training DataCode0
Decoding the Silent Majority: Inducing Belief Augmented Social Graph with Large Language Model for Response ForecastingCode0
I-AI: A Controllable & Interpretable AI System for Decoding Radiologists' Intense Focus for Accurate CXR DiagnosesCode0
Decoding fMRI Data into Captions using Prefix Language ModelingCode0
Show:102550
← PrevPage 334 of 353Next →

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1Decay RNNValidation perplexity76.67Unverified
2GRUValidation perplexity53.78Unverified
3LSTMValidation perplexity52.73Unverified
4LSTMTest perplexity48.7Unverified
5Temporal CNNTest perplexity45.2Unverified
6TCNTest perplexity45.19Unverified
7GCNN-8Test perplexity44.9Unverified
8Neural cache model (size = 100)Test perplexity44.8Unverified
9Neural cache model (size = 2,000)Test perplexity40.8Unverified
10GPT-2 SmallTest perplexity37.5Unverified
#ModelMetricClaimedVerifiedStatus
1TCNTest perplexity108.47Unverified
2Seq-U-NetTest perplexity107.95Unverified
3GRU (Bai et al., 2018)Test perplexity92.48Unverified
4R-TransformerTest perplexity84.38Unverified
5Zaremba et al. (2014) - LSTM (medium)Test perplexity82.7Unverified
6Gal & Ghahramani (2016) - Variational LSTM (medium)Test perplexity79.7Unverified
7LSTM (Bai et al., 2018)Test perplexity78.93Unverified
8Zaremba et al. (2014) - LSTM (large)Test perplexity78.4Unverified
9Gal & Ghahramani (2016) - Variational LSTM (large)Test perplexity75.2Unverified
10Inan et al. (2016) - Variational RHNTest perplexity66Unverified
#ModelMetricClaimedVerifiedStatus
1LSTM (7 layers)Bit per Character (BPC)1.67Unverified
2HypernetworksBit per Character (BPC)1.34Unverified
3SHA-LSTM (4 layers, h=1024, no attention head)Bit per Character (BPC)1.33Unverified
4LN HM-LSTMBit per Character (BPC)1.32Unverified
5ByteNetBit per Character (BPC)1.31Unverified
6Recurrent Highway NetworksBit per Character (BPC)1.27Unverified
7Large FS-LSTM-4Bit per Character (BPC)1.25Unverified
8Large mLSTMBit per Character (BPC)1.24Unverified
9AWD-LSTM (3 layers)Bit per Character (BPC)1.23Unverified
10Cluster-Former (#C=512)Bit per Character (BPC)1.22Unverified
#ModelMetricClaimedVerifiedStatus
1Smaller Transformer 126M (pre-trained)Test perplexity33Unverified
2OPT 125MTest perplexity32.26Unverified
3Larger Transformer 771M (pre-trained)Test perplexity28.1Unverified
4OPT 1.3BTest perplexity19.55Unverified
5GPT-Neo 125MTest perplexity17.83Unverified
6OPT 2.7BTest perplexity17.81Unverified
7Smaller Transformer 126M (fine-tuned)Test perplexity12Unverified
8GPT-Neo 1.3BTest perplexity11.46Unverified
9Transformer 125MTest perplexity10.7Unverified
10GPT-Neo 2.7BTest perplexity10.44Unverified