SOTAVerified

Language Modelling

A language model is a model of natural language. Language models are useful for a variety of tasks, including speech recognition, machine translation, natural language generation (generating more human-like text), optical character recognition, route optimization, handwriting recognition, grammar induction, and information retrieval.

Large language models (LLMs), currently their most advanced form, are predominantly based on transformers trained on larger datasets (frequently using words scraped from the public internet). They have superseded recurrent neural network-based models, which had previously superseded the purely statistical models, such as word n-gram language model.

Source: Wikipedia

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 11451–11500 of 17610 papers

TitleStatusHype
Exploring In-Context Learning Capabilities of Foundation Models for Generating Knowledge Graphs from Text—0
Estimating the Causal Effects of Natural Logic Features in Neural NLI Models—0
Unsupervised Sentence Representation Learning with Frequency-induced Adversarial Tuning and Incomplete Sentence FilteringCode0
Reconstruct Before Summarize: An Efficient Two-Step Framework for Condensing and Summarizing Meeting Transcripts—0
Scalable Educational Question Generation with Pre-trained Language ModelsCode0
Text2Cohort: Facilitating Intuitive Access to Biomedical Data with Natural Language Cohort DiscoveryCode0
MEGABYTE: Predicting Million-byte Sequences with Multiscale Transformers—0
Learning to Reason over Scene Graphs: A Case Study of Finetuning GPT-2 into a Robot Language Model for Grounded Task Planning—0
Prompt Learning to Mitigate Catastrophic Forgetting in Cross-lingual Transfer for Open-domain Dialogue GenerationCode0
Two-in-One: A Model Hijacking Attack Against Text Generation Models—0
Using Language Models to Detect Alarming Student Responses—0
Musketeer: Joint Training for Multi-task Vision Language Model with Task Explanation PromptsCode0
Simple Token-Level Confidence Improves Caption Correctness—0
Long-Tailed Question Answering in an Open World—0
Recommendation as Instruction Following: A Large Language Model Empowered Recommendation Approach—0
Masked Audio Text Encoders are Effective Multi-Modal Rescorers—0
Detecting Idiomatic Multiword Expressions in Clinical Terminology using Definition-Based Representation Learning—0
Unicode Normalization and Grapheme Parsing of Indic Languages—0
How Good are Commercial Large Language Models on African Languages?—0
Domain Incremental Lifelong Learning in an Open World—0
Adapter-TST: A Parameter Efficient Method for Multiple-Attribute Text Style Transfer—0
Enriching language models with graph-based context information to better understand textual dataCode0
Davinci the Dualist: the mind-body divide in large language models and in human learners—0
Say What You Mean! Large Language Models Speak Too Positively about Negative Commonsense KnowledgeCode0
Privacy-Preserving Prompt Tuning for Large Language Model Services—0
LACoS-BLOOM: Low-rank Adaptation with Contrastive objective on 8 bits Siamese-BLOOM—0
Towards an Automatic Optimisation Model Generator Assisted with Generative Pre-trained Transformer—0
Large Language Model Programs—0
Robust Acoustic and Semantic Contextual Biasing in Neural Transducers for Speech Recognition—0
PLM-GNN: A Webpage Classification Method based on Joint Pre-trained Language Model and Graph Neural Network—0
Estimating related words computationally using language model from the Mahabharata - an Indian epic—0
Detection of depression on social networks using transformers and ensemblesCode0
Tomography of Quantum States from Structured Measurements via quantum-aware transformer—0
Effects of sub-word segmentation on performance of transformer language models—0
DeepTextMark: A Deep Learning-Driven Text Watermarking Approach for Identifying Large Language Model Generated TextCode0
A Taxonomy of Foundation Model based Systems through the Lens of Software Architecture—0
Accessible Instruction-Following Agent—0
Event Knowledge Incorporation with Posterior Regularization for Event-Centric Question AnsweringCode0
GersteinLab at MEDIQA-Chat 2023: Clinical Note Summarization from Doctor-Patient Conversations through Fine-tuning and In-context Learning—0
ChatGPT: Vision and Challenges—0
Do Large Language Models Show Decision Heuristics Similar to Humans? A Case Study Using GPT-3.5—0
Knowledge Graph Guided Semantic Evaluation of Language Models For User Trust—0
Scene Text Recognition with Image-Text Matching-guided Dictionary—0
Token-Level Fitting Issues of Seq2seq Models—0
Knowledge-enhanced Agents for Interactive Text Games—0
Learning Summary-Worthy Visual Representation for Abstractive Summarization in Video—0
FACTIFY-5WQA: 5W Aspect-based Fact Verification through Question Answering—0
Empowering Language Model with Guided Knowledge Fusion for Biomedical Document Re-ranking—0
Refining the Responses of LLMs by ThemselvesCode0
Pre-training Language Model as a Multi-perspective Course Learner—0
Show:102550
← PrevPage 230 of 353Next →
All datasetsWikiText-103Penn Treebank (Word Level)enwik8The PileWikiText-2LAMBADAOne Billion WordText8Penn Treebank (Character Level)Hutter PrizeOpenWebTextSALMon

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1Decay RNNValidation perplexity76.67—Unverified
2GRUValidation perplexity53.78—Unverified
3LSTMValidation perplexity52.73—Unverified
4LSTMTest perplexity48.7—Unverified
5Temporal CNNTest perplexity45.2—Unverified
6TCNTest perplexity45.19—Unverified
7GCNN-8Test perplexity44.9—Unverified
8Neural cache model (size = 100)Test perplexity44.8—Unverified
9Neural cache model (size = 2,000)Test perplexity40.8—Unverified
10GPT-2 SmallTest perplexity37.5—Unverified
#ModelMetricClaimedVerifiedStatus
1TCNTest perplexity108.47—Unverified
2Seq-U-NetTest perplexity107.95—Unverified
3GRU (Bai et al., 2018)Test perplexity92.48—Unverified
4R-TransformerTest perplexity84.38—Unverified
5Zaremba et al. (2014) - LSTM (medium)Test perplexity82.7—Unverified
6Gal & Ghahramani (2016) - Variational LSTM (medium)Test perplexity79.7—Unverified
7LSTM (Bai et al., 2018)Test perplexity78.93—Unverified
8Zaremba et al. (2014) - LSTM (large)Test perplexity78.4—Unverified
9Gal & Ghahramani (2016) - Variational LSTM (large)Test perplexity75.2—Unverified
10Inan et al. (2016) - Variational RHNTest perplexity66—Unverified
#ModelMetricClaimedVerifiedStatus
1LSTM (7 layers)Bit per Character (BPC)1.67—Unverified
2HypernetworksBit per Character (BPC)1.34—Unverified
3SHA-LSTM (4 layers, h=1024, no attention head)Bit per Character (BPC)1.33—Unverified
4LN HM-LSTMBit per Character (BPC)1.32—Unverified
5ByteNetBit per Character (BPC)1.31—Unverified
6Recurrent Highway NetworksBit per Character (BPC)1.27—Unverified
7Large FS-LSTM-4Bit per Character (BPC)1.25—Unverified
8Large mLSTMBit per Character (BPC)1.24—Unverified
9AWD-LSTM (3 layers)Bit per Character (BPC)1.23—Unverified
10Cluster-Former (#C=512)Bit per Character (BPC)1.22—Unverified
#ModelMetricClaimedVerifiedStatus
1Smaller Transformer 126M (pre-trained)Test perplexity33—Unverified
2OPT 125MTest perplexity32.26—Unverified
3Larger Transformer 771M (pre-trained)Test perplexity28.1—Unverified
4OPT 1.3BTest perplexity19.55—Unverified
5GPT-Neo 125MTest perplexity17.83—Unverified
6OPT 2.7BTest perplexity17.81—Unverified
7Smaller Transformer 126M (fine-tuned)Test perplexity12—Unverified
8GPT-Neo 1.3BTest perplexity11.46—Unverified
9Transformer 125MTest perplexity10.7—Unverified
10GPT-Neo 2.7BTest perplexity10.44—Unverified