SOTAVerified

Language Modelling

A language model is a model of natural language. Language models are useful for a variety of tasks, including speech recognition, machine translation, natural language generation (generating more human-like text), optical character recognition, route optimization, handwriting recognition, grammar induction, and information retrieval.

Large language models (LLMs), currently their most advanced form, are predominantly based on transformers trained on larger datasets (frequently using words scraped from the public internet). They have superseded recurrent neural network-based models, which had previously superseded the purely statistical models, such as word n-gram language model.

Source: Wikipedia

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 9201–9250 of 17610 papers

TitleStatusHype
Topic-to-essay generation with knowledge-based content selection—0
Nemotron-4 15B Technical Report—0
On Languaging a Simulation Engine—0
OncoGPT: A Medical Conversational Model Tailored with Oncology Domain Expertise on a Large Language Model Meta-AI (LLaMA)—0
Think Big, Generate Quick: LLM-to-SLM for Fast Autoregressive Decoding—0
Retrieval Augmented Generation Systems: Automatic Dataset Creation, Evaluation and Boolean Agent SetupCode0
LLM-Assisted Multi-Teacher Continual Learning for Visual Question Answering in Robotic SurgeryCode0
PIDformer: Transformer Meets Control Theory—0
NeSy is alive and well: A LLM-driven symbolic approach for better code comment data generation and classificationCode0
Text Understanding and Generation Using Transformer Models for Intelligent E-commerce Recommendations—0
Training a Bilingual Language Model by Mapping Tokens onto a Shared Character Space—0
ASETF: A Novel Method for Jailbreak Attack on LLMs through Translate Suffix Embeddings—0
Building Flexible Machine Learning Models for Scientific Computing at Scale—0
Don't Forget Your Reward Values: Language Model Alignment via Value-based Calibration—0
HypoTermQA: Hypothetical Terms Dataset for Benchmarking Hallucination Tendency of LLMsCode0
Bootstrapping Cognitive Agents with a Large Language Model—0
CLIPose: Category-Level Object Pose Estimation with Pre-trained Vision-Language Knowledge—0
ByteComposer: a Human-like Melody Composition Method based on Language Model Agent—0
HD-Eval: Aligning Large Language Model Evaluators Through Hierarchical Criteria Decomposition—0
Foot In The Door: Understanding Large Language Model Jailbreaking via Cognitive Psychology—0
FGBERT: Function-Driven Pre-trained Gene Language Model for Metagenomics—0
Enhancing Cloud-Based Large Language Model Processing with Elasticsearch and Transformer Models—0
Exploring Failure Cases in Multimodal Reasoning About Physical Dynamics—0
Enhanced User Interaction in Operating Systems through Machine Learning Language Models—0
NaVid: Video-based VLM Plans the Next Step for Vision-and-Language Navigation—0
PRP: Propagating Universal Perturbations to Attack Large Language Model Guard-Rails—0
PREDILECT: Preferences Delineated with Zero-Shot Language-based Reasoning in Reinforcement Learning—0
Substrate Prediction for RiPP Biosynthetic Enzymes via Masked Language Modeling and Transfer LearningCode0
Hands-Free VR—0
ArabianGPT: Native Arabic GPT-based Large Language Model—0
Item-side Fairness of Large Language Model-based Recommendation SystemCode0
Fine-Grained Self-Endorsement Improves Factuality and Reasoning—0
From Keywords to Structured Summaries: Streamlining Scholarly Information Access—0
A Decision-Language Model (DLM) for Dynamic Restless Multi-Armed Bandit Tasks in Public Health—0
COMPASS: Computational Mapping of Patient-Therapist Alliance Strategies with Language Modeling—0
Automating psychological hypothesis generation with AI: when large language models meet causal graph—0
Dependency Annotation of Ottoman Turkish with Multilingual BERT—0
CEV-LM: Controlled Edit Vector Language Model for Shaping Natural Language GenerationsCode0
Do Machines and Humans Focus on Similar Code? Exploring Explainability of Large Language Models in Code Summarization—0
Zero-shot cross-lingual transfer in instruction tuning of large language models—0
Tokenization counts: the impact of tokenization on arithmetic in frontier LLMsCode0
Optimizing Language Models for Human Preferences is a Causal Inference Problem—0
Small Language Models as Effective Guides for Large Language Models in Chinese Relation Extraction—0
Noise-BERT: A Unified Perturbation-Robust Framework with Noise Alignment Pre-training for Noisy Slot Filling Task—0
Mitigating the Linguistic Gap with Phonemic Representations for Robust Cross-lingual Transfer—0
Learning to Reduce: Optimal Representations of Structured Data in Prompting Large Language Models—0
Understanding the Dataset Practitioners Behind Large Language Model Development—0
Knowledge Graph Enhanced Large Language Model Editing—0
Measuring Social Biases in Masked Language Models by Proxy of Prediction QualityCode0
Kuaiji: the First Chinese Accounting Large Language Model—0
Show:102550
← PrevPage 185 of 353Next →
All datasetsWikiText-103Penn Treebank (Word Level)enwik8The PileWikiText-2LAMBADAOne Billion WordText8Penn Treebank (Character Level)Hutter PrizeOpenWebTextSALMon

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1Decay RNNValidation perplexity76.67—Unverified
2GRUValidation perplexity53.78—Unverified
3LSTMValidation perplexity52.73—Unverified
4LSTMTest perplexity48.7—Unverified
5Temporal CNNTest perplexity45.2—Unverified
6TCNTest perplexity45.19—Unverified
7GCNN-8Test perplexity44.9—Unverified
8Neural cache model (size = 100)Test perplexity44.8—Unverified
9Neural cache model (size = 2,000)Test perplexity40.8—Unverified
10GPT-2 SmallTest perplexity37.5—Unverified
#ModelMetricClaimedVerifiedStatus
1TCNTest perplexity108.47—Unverified
2Seq-U-NetTest perplexity107.95—Unverified
3GRU (Bai et al., 2018)Test perplexity92.48—Unverified
4R-TransformerTest perplexity84.38—Unverified
5Zaremba et al. (2014) - LSTM (medium)Test perplexity82.7—Unverified
6Gal & Ghahramani (2016) - Variational LSTM (medium)Test perplexity79.7—Unverified
7LSTM (Bai et al., 2018)Test perplexity78.93—Unverified
8Zaremba et al. (2014) - LSTM (large)Test perplexity78.4—Unverified
9Gal & Ghahramani (2016) - Variational LSTM (large)Test perplexity75.2—Unverified
10Inan et al. (2016) - Variational RHNTest perplexity66—Unverified
#ModelMetricClaimedVerifiedStatus
1LSTM (7 layers)Bit per Character (BPC)1.67—Unverified
2HypernetworksBit per Character (BPC)1.34—Unverified
3SHA-LSTM (4 layers, h=1024, no attention head)Bit per Character (BPC)1.33—Unverified
4LN HM-LSTMBit per Character (BPC)1.32—Unverified
5ByteNetBit per Character (BPC)1.31—Unverified
6Recurrent Highway NetworksBit per Character (BPC)1.27—Unverified
7Large FS-LSTM-4Bit per Character (BPC)1.25—Unverified
8Large mLSTMBit per Character (BPC)1.24—Unverified
9AWD-LSTM (3 layers)Bit per Character (BPC)1.23—Unverified
10Cluster-Former (#C=512)Bit per Character (BPC)1.22—Unverified
#ModelMetricClaimedVerifiedStatus
1Smaller Transformer 126M (pre-trained)Test perplexity33—Unverified
2OPT 125MTest perplexity32.26—Unverified
3Larger Transformer 771M (pre-trained)Test perplexity28.1—Unverified
4OPT 1.3BTest perplexity19.55—Unverified
5GPT-Neo 125MTest perplexity17.83—Unverified
6OPT 2.7BTest perplexity17.81—Unverified
7Smaller Transformer 126M (fine-tuned)Test perplexity12—Unverified
8GPT-Neo 1.3BTest perplexity11.46—Unverified
9Transformer 125MTest perplexity10.7—Unverified
10GPT-Neo 2.7BTest perplexity10.44—Unverified