SOTAVerified

Language Modelling

A language model is a model of natural language. Language models are useful for a variety of tasks, including speech recognition, machine translation, natural language generation (generating more human-like text), optical character recognition, route optimization, handwriting recognition, grammar induction, and information retrieval.

Large language models (LLMs), currently their most advanced form, are predominantly based on transformers trained on larger datasets (frequently using words scraped from the public internet). They have superseded recurrent neural network-based models, which had previously superseded the purely statistical models, such as word n-gram language model.

Source: Wikipedia

Papers

Showing 65516600 of 17610 papers

TitleStatusHype
The Knowledge Alignment Problem: Bridging Human and External Knowledge for Large Language ModelsCode0
UFIN: Universal Feature Interaction Network for Multi-Domain Click-Through Rate PredictionCode0
Lipsum-FT: Robust Fine-Tuning of Zero-Shot Models Using Random Text GuidanceCode0
Structural Self-Supervised Objectives for TransformersCode0
Leveraging Online Data to Enhance Medical Knowledge in a Small Persian Language ModelCode0
Self-supervised Preference Optimization: Enhance Your Language Model with Preference Degree AwarenessCode0
On the Complementary Nature of Knowledge Graph Embedding, Fine Grain Entity Types, and Language ModelingCode0
Self-Supervised Knowledge Assimilation for Expert-Layman Text Style TransferCode0
KRLS: Improving End-to-End Response Generation in Task Oriented Dialog with Reinforced Keywords LearningCode0
LecEval: An Automated Metric for Multimodal Knowledge Acquisition in Multimedia LearningCode0
Phone-ing it in: Towards Flexible Multi-Modal Language Model Training by Phonetic Representations of DataCode0
Self-Supervised Contrastive Learning with Adversarial Perturbations for Defending Word Substitution-based AttacksCode0
Structured Content Preservation for Unsupervised Text Style TransferCode0
Leveraging Multimodal LLM for Inspirational User Interface SearchCode0
Structured Dialogue System for Mental Health: An LLM Chatbot Leveraging the PM+ GuidelinesCode0
Regulation of Language Models With Interpretability Will Likely Result In A Performance Trade-OffCode0
Mitigating Reversal Curse in Large Language Models via Semantic-aware Permutation TrainingCode0
Mitigating Test-Time Bias for Fair Image RetrievalCode0
Structured Like a Language Model: Analysing AI as an Automated SubjectCode0
Mitigating the Bias of Large Language Model EvaluationCode0
Phonemic Transcription of Low-Resource Tonal LanguagesCode0
Recurrent Neural Networks Hardware Implementation on FPGACode0
Mitigating the Impact of Outlier Channels for Language Model Quantization with Activation RegularizationCode0
Structured Sequence Modeling with Graph Convolutional Recurrent NetworksCode0
Self-Refined Large Language Model as Automated Reward Function Designer for Deep Reinforcement Learning in RoboticsCode0
Native Design Bias: Studying the Impact of English Nativeness on Language Model PerformanceCode0
MultiHal: Multilingual Dataset for Knowledge-Graph Grounded Evaluation of LLM HallucinationsCode0
Understanding the Vulnerability of CLIP to Image CompressionCode0
Phonotactic Complexity across DialectsCode0
Learning to Learn Words from Visual ScenesCode0
Protein language model rescue mutations highlight variant effects and structure in clinically relevant genesCode0
Toward Open-Set Human Object Interaction DetectionCode0
On the adequacy of untuned warmup for adaptive optimizationCode0
Leveraging LLMs in Scholarly Knowledge Graph Question AnsweringCode0
Negated and Misprimed Probes for Pretrained Language Models: Birds Can Talk, But Cannot FlyCode0
MADLAD-400: A Multilingual And Document-Level Large Audited DatasetCode0
Leveraging LLMs for Unsupervised Dense Retriever RankingCode0
Ranking Manipulation for Conversational Search EnginesCode0
Language Models Meet Anomaly Detection for Better Interpretability and GeneralizabilityCode0
ProtiGeno: a prokaryotic short gene finder using protein language modelsCode0
Studying word order through iterative shufflingCode0
Language Generation via Combinatorial Constraint Satisfaction: A Tree Search Enhanced Monte-Carlo ApproachCode0
Improving Deep Learning Optimization through Constrained Parameter RegularizationCode0
Style2Code: A Style-Controllable Code Generation Framework with Dual-Modal Contrastive Representation LearningCode0
Measuring Copyright Risks of Large Language Model via Partial Information ProbingCode0
Phrase break prediction with bidirectional encoder representations in Japanese text-to-speech synthesisCode0
UIO at SemEval-2023 Task 12: Multilingual fine-tuning for sentiment classification in low-resource languagesCode0
Measuring Contextual Informativeness in Child-Directed TextCode0
Mixout: Effective Regularization to Finetune Large-scale Pretrained Language ModelsCode0
Regularizing RNNs by Stabilizing ActivationsCode0
Show:102550
← PrevPage 132 of 353Next →

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1Decay RNNValidation perplexity76.67Unverified
2GRUValidation perplexity53.78Unverified
3LSTMValidation perplexity52.73Unverified
4LSTMTest perplexity48.7Unverified
5Temporal CNNTest perplexity45.2Unverified
6TCNTest perplexity45.19Unverified
7GCNN-8Test perplexity44.9Unverified
8Neural cache model (size = 100)Test perplexity44.8Unverified
9Neural cache model (size = 2,000)Test perplexity40.8Unverified
10GPT-2 SmallTest perplexity37.5Unverified
#ModelMetricClaimedVerifiedStatus
1TCNTest perplexity108.47Unverified
2Seq-U-NetTest perplexity107.95Unverified
3GRU (Bai et al., 2018)Test perplexity92.48Unverified
4R-TransformerTest perplexity84.38Unverified
5Zaremba et al. (2014) - LSTM (medium)Test perplexity82.7Unverified
6Gal & Ghahramani (2016) - Variational LSTM (medium)Test perplexity79.7Unverified
7LSTM (Bai et al., 2018)Test perplexity78.93Unverified
8Zaremba et al. (2014) - LSTM (large)Test perplexity78.4Unverified
9Gal & Ghahramani (2016) - Variational LSTM (large)Test perplexity75.2Unverified
10Inan et al. (2016) - Variational RHNTest perplexity66Unverified
#ModelMetricClaimedVerifiedStatus
1LSTM (7 layers)Bit per Character (BPC)1.67Unverified
2HypernetworksBit per Character (BPC)1.34Unverified
3SHA-LSTM (4 layers, h=1024, no attention head)Bit per Character (BPC)1.33Unverified
4LN HM-LSTMBit per Character (BPC)1.32Unverified
5ByteNetBit per Character (BPC)1.31Unverified
6Recurrent Highway NetworksBit per Character (BPC)1.27Unverified
7Large FS-LSTM-4Bit per Character (BPC)1.25Unverified
8Large mLSTMBit per Character (BPC)1.24Unverified
9AWD-LSTM (3 layers)Bit per Character (BPC)1.23Unverified
10Cluster-Former (#C=512)Bit per Character (BPC)1.22Unverified
#ModelMetricClaimedVerifiedStatus
1Smaller Transformer 126M (pre-trained)Test perplexity33Unverified
2OPT 125MTest perplexity32.26Unverified
3Larger Transformer 771M (pre-trained)Test perplexity28.1Unverified
4OPT 1.3BTest perplexity19.55Unverified
5GPT-Neo 125MTest perplexity17.83Unverified
6OPT 2.7BTest perplexity17.81Unverified
7Smaller Transformer 126M (fine-tuned)Test perplexity12Unverified
8GPT-Neo 1.3BTest perplexity11.46Unverified
9Transformer 125MTest perplexity10.7Unverified
10GPT-Neo 2.7BTest perplexity10.44Unverified