SOTAVerified

Language Modelling

A language model is a model of natural language. Language models are useful for a variety of tasks, including speech recognition, machine translation, natural language generation (generating more human-like text), optical character recognition, route optimization, handwriting recognition, grammar induction, and information retrieval.

Large language models (LLMs), currently their most advanced form, are predominantly based on transformers trained on larger datasets (frequently using words scraped from the public internet). They have superseded recurrent neural network-based models, which had previously superseded the purely statistical models, such as word n-gram language model.

Source: Wikipedia

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 7601–7650 of 17610 papers

TitleStatusHype
FoundationPose: Unified 6D Pose Estimation and Tracking of Novel ObjectsCode4
CoIE: Chain-of-Instruct Editing for Multi-Attribute Face Manipulation—0
Large Language Model Enhanced Multi-Agent Systems for 6G Communications—0
Fine-Grained Image-Text Alignment in Medical Imaging Enables Explainable Cyclic Image-Report Generation—0
SwitchHead: Accelerating Transformers with Mixture-of-Experts AttentionCode1
ViLA: Efficient Video-Language Alignment for Video Question AnsweringCode1
Breaking the Silence: the Threats of Using LLMs in Software EngineeringCode0
A Foundational Multimodal Vision Language AI Assistant for Human Pathology—0
LD-SDM: Language-Driven Hierarchical Species Distribution Modeling—0
Prompt Engineering-assisted Malware Dynamic Analysis Using GPT-4Code1
Building Open-Ended Embodied Agent via Language-Policy Bidirectional Adaptation—0
Translating Natural Language Queries to SQL Using the T5 Model—0
ComplexityNet: Increasing LLM Inference Efficiency by Learning Task Complexity—0
Rethinking the Instruction Quality: LIFT is What You Need—0
LLM in a flash: Efficient Large Language Model Inference with Limited Memory—0
A dynamical clipping approach with task feedback for Proximal Policy OptimizationCode0
The GUA-Speech System Description for CNVSRC Challenge 2023—0
Astrocyte-Enabled Advancements in Spiking Neural Networks for Large Language Modeling—0
Classifying complex documents: comparing bespoke solutions to large language models—0
READ: Recurrent Adapter with Partial Video-Language Alignment for Parameter-Efficient Transfer Learning in Low-Resource Video-Language ModelingCode1
VILA: On Pre-training for Visual Language ModelsCode4
Neural Machine Translation of Clinical Text: An Empirical Investigation into Multilingual Pre-Trained Language Models and Transfer-LearningCode0
Remote Sensing Vision-Language Foundation Models without Annotations via Ground Remote Alignment—0
Hallucination Augmented Contrastive Learning for Multimodal Large Language ModelCode1
SM70: A Large Language Model for Medical Devices—0
SGLang: Efficient Execution of Structured Language Model ProgramsCode6
On Diversified Preferences of Large Language Model AlignmentCode1
SCCA: Shifted Cross Chunk Attention for long contextual semantic expansion—0
Vision-language Assisted Attribute Learning—0
Audio-Visual LLM for Video Understanding—0
Evaluating ChatGPT as a Question Answering System: A Comprehensive Analysis and Comparison with Existing Models—0
Decoupling SQL Query Hardness Parsing for Text-to-SQL—0
GTA: Gated Toxicity Avoidance for LM Performance PreservationCode0
Generative Large Language Models Are All-purpose Text Analytics Engines: Text-to-text Learning Is All Your Need—0
Gated Linear Attention Transformers with Hardware-Efficient TrainingCode1
Order Matters in the Presence of Dataset Imbalance for Multilingual Learning—0
Linguistic and Structural Basis of Engineering Design Knowledge—0
METAL: Metamorphic Testing Framework for Analyzing Large-Language Model Qualities—0
PromptMTopic: Unsupervised Multimodal Topic Modeling of Memes using Large Language ModelsCode0
Progressive Multi-Modality Learning for Inverse Protein FoldingCode1
Where exactly does contextualization in a PLM happen?—0
Leveraging Generative Language Models for Weakly Supervised Sentence Component Analysis in Video-Language Joint Learning—0
Negative Pre-aware for Noisy Cross-modal MatchingCode1
Preserving Privacy Through Dememorization: An Unlearning Technique For Mitigating Memorization Risks In Language ModelsCode0
Labrador: Exploring the Limits of Masked Language Modeling for Laboratory DataCode1
History Matters: Temporal Knowledge Editing in Large Language ModelCode1
Stateful Large Language Model Serving with Pensieve—0
Teamwork Dimensions Classification Using BERT—0
TCNCA: Temporal Convolution Network with Chunked Attention for Scalable Sequence Processing—0
Enhancing Medical Specialty Assignment to Patients using NLP Techniques—0
Show:102550
← PrevPage 153 of 353Next →
All datasetsWikiText-103Penn Treebank (Word Level)enwik8The PileWikiText-2LAMBADAOne Billion WordText8Penn Treebank (Character Level)Hutter PrizeOpenWebTextSALMon

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1Decay RNNValidation perplexity76.67—Unverified
2GRUValidation perplexity53.78—Unverified
3LSTMValidation perplexity52.73—Unverified
4LSTMTest perplexity48.7—Unverified
5Temporal CNNTest perplexity45.2—Unverified
6TCNTest perplexity45.19—Unverified
7GCNN-8Test perplexity44.9—Unverified
8Neural cache model (size = 100)Test perplexity44.8—Unverified
9Neural cache model (size = 2,000)Test perplexity40.8—Unverified
10GPT-2 SmallTest perplexity37.5—Unverified
#ModelMetricClaimedVerifiedStatus
1TCNTest perplexity108.47—Unverified
2Seq-U-NetTest perplexity107.95—Unverified
3GRU (Bai et al., 2018)Test perplexity92.48—Unverified
4R-TransformerTest perplexity84.38—Unverified
5Zaremba et al. (2014) - LSTM (medium)Test perplexity82.7—Unverified
6Gal & Ghahramani (2016) - Variational LSTM (medium)Test perplexity79.7—Unverified
7LSTM (Bai et al., 2018)Test perplexity78.93—Unverified
8Zaremba et al. (2014) - LSTM (large)Test perplexity78.4—Unverified
9Gal & Ghahramani (2016) - Variational LSTM (large)Test perplexity75.2—Unverified
10Inan et al. (2016) - Variational RHNTest perplexity66—Unverified
#ModelMetricClaimedVerifiedStatus
1LSTM (7 layers)Bit per Character (BPC)1.67—Unverified
2HypernetworksBit per Character (BPC)1.34—Unverified
3SHA-LSTM (4 layers, h=1024, no attention head)Bit per Character (BPC)1.33—Unverified
4LN HM-LSTMBit per Character (BPC)1.32—Unverified
5ByteNetBit per Character (BPC)1.31—Unverified
6Recurrent Highway NetworksBit per Character (BPC)1.27—Unverified
7Large FS-LSTM-4Bit per Character (BPC)1.25—Unverified
8Large mLSTMBit per Character (BPC)1.24—Unverified
9AWD-LSTM (3 layers)Bit per Character (BPC)1.23—Unverified
10Cluster-Former (#C=512)Bit per Character (BPC)1.22—Unverified
#ModelMetricClaimedVerifiedStatus
1Smaller Transformer 126M (pre-trained)Test perplexity33—Unverified
2OPT 125MTest perplexity32.26—Unverified
3Larger Transformer 771M (pre-trained)Test perplexity28.1—Unverified
4OPT 1.3BTest perplexity19.55—Unverified
5GPT-Neo 125MTest perplexity17.83—Unverified
6OPT 2.7BTest perplexity17.81—Unverified
7Smaller Transformer 126M (fine-tuned)Test perplexity12—Unverified
8GPT-Neo 1.3BTest perplexity11.46—Unverified
9Transformer 125MTest perplexity10.7—Unverified
10GPT-Neo 2.7BTest perplexity10.44—Unverified