SOTAVerified

Language Modelling

A language model is a model of natural language. Language models are useful for a variety of tasks, including speech recognition, machine translation, natural language generation (generating more human-like text), optical character recognition, route optimization, handwriting recognition, grammar induction, and information retrieval.

Large language models (LLMs), currently their most advanced form, are predominantly based on transformers trained on larger datasets (frequently using words scraped from the public internet). They have superseded recurrent neural network-based models, which had previously superseded the purely statistical models, such as word n-gram language model.

Source: Wikipedia

Papers

Showing 56515700 of 17610 papers

TitleStatusHype
Mask-Free Neuron Concept Annotation for Interpreting Neural Networks in Medical DomainCode0
PreAlign: Boosting Cross-Lingual Transfer by Early Establishment of Multilingual AlignmentCode0
Leveraging Content and Acoustic Representations for Speech Emotion RecognitionCode0
Precise Task Formalization Matters in Winograd Schema EvaluationsCode0
Mask-Enhanced Autoregressive Prediction: Pay Less Attention to Learn MoreCode0
Neural Shuffle-Exchange Networks - Sequence Processing in O(n log n) TimeCode0
Techniques to Improve Neural Math Word Problem SolversCode0
The Role of Language Imbalance in Cross-lingual Generalisation: Insights from Cloned Language ExperimentsCode0
TeDA: Boosting Vision-Lanuage Models for Zero-Shot 3D Object Retrieval via Testing-time Distribution AlignmentCode0
PreCogIIITH at HinglishEval : Leveraging Code-Mixing Metrics & Language Model Embeddings To Estimate Code-Mix QualityCode0
Low-Rank Constraints for Fast Inference in Structured ModelsCode0
Predefined Sparseness in Recurrent Sequence ModelsCode0
Towards Ontology-Enhanced Representation Learning for Large Language ModelsCode0
TEII: Think, Explain, Interact and Iterate with Large Language Models to Solve Cross-lingual Emotion DetectionCode0
Learning Intrinsic Sparse Structures within Long Short-Term MemoryCode0
TRINS: Towards Multimodal Language Models that Can ReadCode0
REDAffectiveLM: Leveraging Affect Enriched Embedding and Transformer-based Neural Language Model for Readers' Emotion DetectionCode0
LLM as Dataset Analyst: Subpopulation Structure Discovery with Large Language ModelCode0
Telling Stories for Common Sense Zero-Shot Action RecognitionCode0
Knowledge-aware Collaborative Filtering with Pre-trained Language Model for Personalized Review-based Rating PredictionCode0
Tell me what I need to know: Exploring LLM-based (Personalized) Abstractive Multi-Source Meeting SummarizationCode0
Predicting 3D Human Dynamics from VideoCode0
Reformulating Sequential Recommendation: Learning Dynamic User Interest with Content-enriched Language ModelingCode0
Models In a Spelling Bee: Language Models Implicitly Learn the Character Composition of TokensCode0
LLM as OS, Agents as Apps: Envisioning AIOS, Agents and the AIOS-Agent EcosystemCode0
Long Short-Term Memory-Networks for Machine ReadingCode0
K-12BERT: BERT for K-12 educationCode0
Masked Latent Semantic Modeling: an Efficient Pre-training Alternative to Masked Language ModelingCode0
Local and Global Decoding in Text GenerationCode0
TempoGPT: Enhancing Temporal Reasoning via Quantizing EmbeddingCode0
Temporal Action Detection Using a Statistical Language ModelCode0
Know Your Needs Better: Towards Structured Understanding of Marketer Demands with Analogical Reasoning Augmented LLMsCode0
Temporal Analysis of Language through Neural Language ModelsCode0
Model-tuning Via Prompts Makes NLP Models Adversarially RobustCode0
On the Choice of Modeling Unit for Sequence-to-Sequence Speech RecognitionCode0
Sample Efficient Text Summarization Using a Single Pre-Trained TransformerCode0
Predicting Class Distribution Shift for Reliable Domain Adaptive Object DetectionCode0
Temporal-Oriented Recipe for Transferring Large Vision-Language Model to Video UnderstandingCode0
LLM-Assisted Multi-Teacher Continual Learning for Visual Question Answering in Robotic SurgeryCode0
Towards Personalized Evaluation of Large Language Models with An Anonymous Crowd-Sourcing PlatformCode0
Sameness Entices, but Novelty Enchants in Fanfiction OnlineCode0
Learning Longer Memory in Recurrent Neural NetworksCode0
Tensorized Embedding Layers for Efficient Model CompressionCode0
Objectively Evaluating the Reliability of Cell Type Annotation Using LLM-Based StrategiesCode0
Learn from Downstream and Be Yourself in Multimodal Large Language Model Fine-TuningCode0
Network Traffic Anomaly Detection Using Recurrent Neural NetworksCode0
Tensor Product Attention Is All You NeedCode0
Neural Shuffle-Exchange Networks -- Sequence Processing in O(n log n) TimeCode0
Masked Language Models are Good Heterogeneous Graph GeneralizersCode0
Learn from Failure: Fine-Tuning LLMs with Trial-and-Error Data for Intuitionistic Propositional Logic ProvingCode0
Show:102550
← PrevPage 114 of 353Next →

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1Decay RNNValidation perplexity76.67Unverified
2GRUValidation perplexity53.78Unverified
3LSTMValidation perplexity52.73Unverified
4LSTMTest perplexity48.7Unverified
5Temporal CNNTest perplexity45.2Unverified
6TCNTest perplexity45.19Unverified
7GCNN-8Test perplexity44.9Unverified
8Neural cache model (size = 100)Test perplexity44.8Unverified
9Neural cache model (size = 2,000)Test perplexity40.8Unverified
10GPT-2 SmallTest perplexity37.5Unverified
#ModelMetricClaimedVerifiedStatus
1TCNTest perplexity108.47Unverified
2Seq-U-NetTest perplexity107.95Unverified
3GRU (Bai et al., 2018)Test perplexity92.48Unverified
4R-TransformerTest perplexity84.38Unverified
5Zaremba et al. (2014) - LSTM (medium)Test perplexity82.7Unverified
6Gal & Ghahramani (2016) - Variational LSTM (medium)Test perplexity79.7Unverified
7LSTM (Bai et al., 2018)Test perplexity78.93Unverified
8Zaremba et al. (2014) - LSTM (large)Test perplexity78.4Unverified
9Gal & Ghahramani (2016) - Variational LSTM (large)Test perplexity75.2Unverified
10Inan et al. (2016) - Variational RHNTest perplexity66Unverified
#ModelMetricClaimedVerifiedStatus
1LSTM (7 layers)Bit per Character (BPC)1.67Unverified
2HypernetworksBit per Character (BPC)1.34Unverified
3SHA-LSTM (4 layers, h=1024, no attention head)Bit per Character (BPC)1.33Unverified
4LN HM-LSTMBit per Character (BPC)1.32Unverified
5ByteNetBit per Character (BPC)1.31Unverified
6Recurrent Highway NetworksBit per Character (BPC)1.27Unverified
7Large FS-LSTM-4Bit per Character (BPC)1.25Unverified
8Large mLSTMBit per Character (BPC)1.24Unverified
9AWD-LSTM (3 layers)Bit per Character (BPC)1.23Unverified
10Cluster-Former (#C=512)Bit per Character (BPC)1.22Unverified
#ModelMetricClaimedVerifiedStatus
1Smaller Transformer 126M (pre-trained)Test perplexity33Unverified
2OPT 125MTest perplexity32.26Unverified
3Larger Transformer 771M (pre-trained)Test perplexity28.1Unverified
4OPT 1.3BTest perplexity19.55Unverified
5GPT-Neo 125MTest perplexity17.83Unverified
6OPT 2.7BTest perplexity17.81Unverified
7Smaller Transformer 126M (fine-tuned)Test perplexity12Unverified
8GPT-Neo 1.3BTest perplexity11.46Unverified
9Transformer 125MTest perplexity10.7Unverified
10GPT-Neo 2.7BTest perplexity10.44Unverified