SOTAVerified

Language Modelling

A language model is a model of natural language. Language models are useful for a variety of tasks, including speech recognition, machine translation, natural language generation (generating more human-like text), optical character recognition, route optimization, handwriting recognition, grammar induction, and information retrieval.

Large language models (LLMs), currently their most advanced form, are predominantly based on transformers trained on larger datasets (frequently using words scraped from the public internet). They have superseded recurrent neural network-based models, which had previously superseded the purely statistical models, such as word n-gram language model.

Source: Wikipedia

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 9951–10000 of 17610 papers

TitleStatusHype
Unicode Normalization and Grapheme Parsing of Indic Languages—0
Masked Audio Text Encoders are Effective Multi-Modal Rescorers—0
Musketeer: Joint Training for Multi-task Vision Language Model with Task Explanation PromptsCode0
Recommendation as Instruction Following: A Large Language Model Empowered Recommendation Approach—0
Long-Tailed Question Answering in an Open World—0
Simple Token-Level Confidence Improves Caption Correctness—0
How to Index Item IDs for Recommendation Foundation ModelsCode2
Detecting Idiomatic Multiword Expressions in Clinical Terminology using Definition-Based Representation Learning—0
How Good are Commercial Large Language Models on African Languages?—0
Domain Incremental Lifelong Learning in an Open World—0
Enriching language models with graph-based context information to better understand textual dataCode0
Davinci the Dualist: the mind-body divide in large language models and in human learners—0
Bot or Human? Detecting ChatGPT Imposters with A Single QuestionCode1
LACoS-BLOOM: Low-rank Adaptation with Contrastive objective on 8 bits Siamese-BLOOM—0
Privacy-Preserving Prompt Tuning for Large Language Model Services—0
Say What You Mean! Large Language Models Speak Too Positively about Negative Commonsense KnowledgeCode0
Automatic Evaluation of Attribution by Large Language ModelsCode1
Adapter-TST: A Parameter Efficient Method for Multiple-Attribute Text Style Transfer—0
DeepTextMark: A Deep Learning-Driven Text Watermarking Approach for Identifying Large Language Model Generated TextCode0
Estimating related words computationally using language model from the Mahabharata - an Indian epic—0
Detection of depression on social networks using transformers and ensemblesCode0
A Taxonomy of Foundation Model based Systems through the Lens of Software Architecture—0
InternGPT: Solving Vision-Centric Tasks by Interacting with ChatGPT Beyond LanguageCode4
Tomography of Quantum States from Structured Measurements via quantum-aware transformer—0
PLM-GNN: A Webpage Classification Method based on Joint Pre-trained Language Model and Graph Neural Network—0
Towards an Automatic Optimisation Model Generator Assisted with Generative Pre-trained Transformer—0
Robust Acoustic and Semantic Contextual Biasing in Neural Transducers for Speech Recognition—0
Large Language Model Programs—0
Effects of sub-word segmentation on performance of transformer language models—0
ChatGPT: Vision and Challenges—0
Accessible Instruction-Following Agent—0
GersteinLab at MEDIQA-Chat 2023: Clinical Note Summarization from Doctor-Patient Conversations through Fine-tuning and In-context Learning—0
Knowledge Graph Guided Semantic Evaluation of Language Models For User Trust—0
Knowledge-enhanced Agents for Interactive Text Games—0
Learning Summary-Worthy Visual Representation for Abstractive Summarization in Video—0
Scene Text Recognition with Image-Text Matching-guided Dictionary—0
Toeplitz Neural Network for Sequence ModelingCode1
PromptRank: Unsupervised Keyphrase Extraction Using PromptCode1
Token-Level Fitting Issues of Seq2seq Models—0
MultiModal-GPT: A Vision and Language Model for Dialogue with HumansCode3
Do Large Language Models Show Decision Heuristics Similar to Humans? A Case Study Using GPT-3.5—0
A Multi-Modal Context Reasoning Approach for Conditional Inference on Joint Textual and Visual CluesCode1
Event Knowledge Incorporation with Posterior Regularization for Event-Centric Question AnsweringCode0
Generative Pretrained Autoregressive Transformer Graph Neural Network applied to the Analysis and Discovery of Novel ProteinsCode1
Empowering Language Model with Guided Knowledge Fusion for Biomedical Document Re-ranking—0
FACTIFY-5WQA: 5W Aspect-based Fact Verification through Question Answering—0
X-LLM: Bootstrapping Advanced Large Language Models by Treating Multi-Modalities as Foreign LanguagesCode3
Unified Demonstration Retriever for In-Context LearningCode1
Pre-training Language Model as a Multi-perspective Course Learner—0
Refining the Responses of LLMs by ThemselvesCode0
Show:102550
← PrevPage 200 of 353Next →
All datasetsWikiText-103Penn Treebank (Word Level)enwik8The PileWikiText-2LAMBADAOne Billion WordText8Penn Treebank (Character Level)Hutter PrizeOpenWebTextSALMon

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1Decay RNNValidation perplexity76.67—Unverified
2GRUValidation perplexity53.78—Unverified
3LSTMValidation perplexity52.73—Unverified
4LSTMTest perplexity48.7—Unverified
5Temporal CNNTest perplexity45.2—Unverified
6TCNTest perplexity45.19—Unverified
7GCNN-8Test perplexity44.9—Unverified
8Neural cache model (size = 100)Test perplexity44.8—Unverified
9Neural cache model (size = 2,000)Test perplexity40.8—Unverified
10GPT-2 SmallTest perplexity37.5—Unverified
#ModelMetricClaimedVerifiedStatus
1TCNTest perplexity108.47—Unverified
2Seq-U-NetTest perplexity107.95—Unverified
3GRU (Bai et al., 2018)Test perplexity92.48—Unverified
4R-TransformerTest perplexity84.38—Unverified
5Zaremba et al. (2014) - LSTM (medium)Test perplexity82.7—Unverified
6Gal & Ghahramani (2016) - Variational LSTM (medium)Test perplexity79.7—Unverified
7LSTM (Bai et al., 2018)Test perplexity78.93—Unverified
8Zaremba et al. (2014) - LSTM (large)Test perplexity78.4—Unverified
9Gal & Ghahramani (2016) - Variational LSTM (large)Test perplexity75.2—Unverified
10Inan et al. (2016) - Variational RHNTest perplexity66—Unverified
#ModelMetricClaimedVerifiedStatus
1LSTM (7 layers)Bit per Character (BPC)1.67—Unverified
2HypernetworksBit per Character (BPC)1.34—Unverified
3SHA-LSTM (4 layers, h=1024, no attention head)Bit per Character (BPC)1.33—Unverified
4LN HM-LSTMBit per Character (BPC)1.32—Unverified
5ByteNetBit per Character (BPC)1.31—Unverified
6Recurrent Highway NetworksBit per Character (BPC)1.27—Unverified
7Large FS-LSTM-4Bit per Character (BPC)1.25—Unverified
8Large mLSTMBit per Character (BPC)1.24—Unverified
9AWD-LSTM (3 layers)Bit per Character (BPC)1.23—Unverified
10Cluster-Former (#C=512)Bit per Character (BPC)1.22—Unverified
#ModelMetricClaimedVerifiedStatus
1Smaller Transformer 126M (pre-trained)Test perplexity33—Unverified
2OPT 125MTest perplexity32.26—Unverified
3Larger Transformer 771M (pre-trained)Test perplexity28.1—Unverified
4OPT 1.3BTest perplexity19.55—Unverified
5GPT-Neo 125MTest perplexity17.83—Unverified
6OPT 2.7BTest perplexity17.81—Unverified
7Smaller Transformer 126M (fine-tuned)Test perplexity12—Unverified
8GPT-Neo 1.3BTest perplexity11.46—Unverified
9Transformer 125MTest perplexity10.7—Unverified
10GPT-Neo 2.7BTest perplexity10.44—Unverified