SOTAVerified

Language Modelling

A language model is a model of natural language. Language models are useful for a variety of tasks, including speech recognition, machine translation, natural language generation (generating more human-like text), optical character recognition, route optimization, handwriting recognition, grammar induction, and information retrieval.

Large language models (LLMs), currently their most advanced form, are predominantly based on transformers trained on larger datasets (frequently using words scraped from the public internet). They have superseded recurrent neural network-based models, which had previously superseded the purely statistical models, such as word n-gram language model.

Source: Wikipedia

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 7801–7850 of 17610 papers

TitleStatusHype
How good are Large Language Models on African Languages?—0
How Good is ChatGPT at Audiovisual Deepfake Detection: A Comparative Study of ChatGPT, AI Models and Human Perception—0
How Green are Neural Language Models? Analyzing Energy Consumption in Text Summarization Fine-tuning—0
How Length Prediction Influence the Performance of Non-Autoregressive Translation?—0
How LSTM Encodes Syntax: Exploring Context Vectors and Semi-Quantization on Natural Text—0
How Many Languages Can a Language Model Model?—0
How Many Parameters Does it Take to Change a Light Bulb? Evaluating Performance in Self-Play of Conversational Games as a Function of Model Characteristics—0
How much do language models memorize?—0
How predictable is language model benchmark performance?—0
How Self-Attention Improves Rare Class Performance in a Question-Answering Dialogue Agent—0
How Teachers Can Use Large Language Models and Bloom's Taxonomy to Create Educational Quizzes—0
How to Adapt Your Large-Scale Vision-and-Language Model—0
How to Avoid Sentences Spelling Boring? Towards a Neural Approach to Unsupervised Metaphor Generation—0
How to Avoid Unwanted Pregnancies: Domain Adaptation using Neural Network Models—0
How to Bridge the Gap between Modalities: Survey on Multimodal Large Language Model—0
How to Build an AI Tutor That Can Adapt to Any Course Using Knowledge Graph-Enhanced Retrieval-Augmented Generation (KG-RAG)—0
How to Construct Deep Recurrent Neural Networks—0
How to Motivate Your Dragon: Teaching Goal-Driven Agents to Speak and Act in Fantasy Worlds—0
How to Prune Your Language Model: Recovering Accuracy on the "Sparsity May Cry'' Benchmark—0
How to represent a word and predict it, too: Improving tied architectures for language modelling—0
How User Language Affects Conflict Fatality Estimates in ChatGPT—0
How Well Can Vision Language Models See Image Details?—0
How Well Do Deep Learning Models Capture Human Concepts? The Case of the Typicality Effect—0
How will Language Modelers like ChatGPT Affect Occupations and Industries?—0
HPC-Coder-V2: Studying Code LLMs Across Low-Resource Parallel Languages—0
HPC-GPT: Integrating Large Language Model for High-Performance Computing—0
HPE-CogVLM: Advancing Vision Language Models with a Head Pose Grounding Task—0
HPS: Hard Preference Sampling for Human Preference Alignment—0
HRLAIF: Improvements in Helpfulness and Harmlessness in Open-domain Reinforcement Learning From AI Feedback—0
HSC-GPT: A Large Language Model for Human Settlements Construction—0
HSI-GPT: A General-Purpose Large Scene-Motion-Language Model for Human Scene Interaction—0
HSSA tree structures for BTG-based preordering in machine translation—0
HTLM: Hyper-Text Pre-Training and Prompting of Language Models—0
HuaAMS at SemEval-2022 Task 8: Combining Translation and Domain Pre-training for Cross-lingual News Article Similarity—0
Human Adversarial QA: Did the Model Understand the Paragraph?—0
HumanAesExpert: Advancing a Multi-Modality Foundation Model for Human Image Aesthetic Assessment—0
Human-centric Dialog Training via Offline Reinforcement Learning—0
Human-Centric NLP or AI-Centric Illusion?: A Critical Investigation—0
Human Evaluation of Procedural Knowledge Graph Extraction from Text with Large Language Models—0
Human Implicit Preference-Based Policy Fine-tuning for Multi-Agent Reinforcement Learning in USV Swarm—0
Human Instruction-Following with Deep Reinforcement Learning via Transfer-Learning from Text—0
Humanity's Last Exam—0
Human Language Modeling—0
Human Latency Conversational Turns for Spoken Avatar Systems—0
Human-like Natural Language Generation Using Monte Carlo Tree Search—0
Human Mobility Modeling with Limited Information via Large Language Models—0
Human-Object Interaction from Human-Level Instructions—0
Human-Object Interaction with Vision-Language Model Guided Relative Movement Dynamics—0
HumanOmni: A Large Vision-Speech Language Model for Human-Centric Video Understanding—0
Human or Not? A Gamified Approach to the Turing Test—0
Show:102550
← PrevPage 157 of 353Next →
All datasetsWikiText-103Penn Treebank (Word Level)enwik8The PileWikiText-2LAMBADAOne Billion WordText8Penn Treebank (Character Level)Hutter PrizeOpenWebTextSALMon

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1Decay RNNValidation perplexity76.67—Unverified
2GRUValidation perplexity53.78—Unverified
3LSTMValidation perplexity52.73—Unverified
4LSTMTest perplexity48.7—Unverified
5Temporal CNNTest perplexity45.2—Unverified
6TCNTest perplexity45.19—Unverified
7GCNN-8Test perplexity44.9—Unverified
8Neural cache model (size = 100)Test perplexity44.8—Unverified
9Neural cache model (size = 2,000)Test perplexity40.8—Unverified
10GPT-2 SmallTest perplexity37.5—Unverified
#ModelMetricClaimedVerifiedStatus
1TCNTest perplexity108.47—Unverified
2Seq-U-NetTest perplexity107.95—Unverified
3GRU (Bai et al., 2018)Test perplexity92.48—Unverified
4R-TransformerTest perplexity84.38—Unverified
5Zaremba et al. (2014) - LSTM (medium)Test perplexity82.7—Unverified
6Gal & Ghahramani (2016) - Variational LSTM (medium)Test perplexity79.7—Unverified
7LSTM (Bai et al., 2018)Test perplexity78.93—Unverified
8Zaremba et al. (2014) - LSTM (large)Test perplexity78.4—Unverified
9Gal & Ghahramani (2016) - Variational LSTM (large)Test perplexity75.2—Unverified
10Inan et al. (2016) - Variational RHNTest perplexity66—Unverified
#ModelMetricClaimedVerifiedStatus
1LSTM (7 layers)Bit per Character (BPC)1.67—Unverified
2HypernetworksBit per Character (BPC)1.34—Unverified
3SHA-LSTM (4 layers, h=1024, no attention head)Bit per Character (BPC)1.33—Unverified
4LN HM-LSTMBit per Character (BPC)1.32—Unverified
5ByteNetBit per Character (BPC)1.31—Unverified
6Recurrent Highway NetworksBit per Character (BPC)1.27—Unverified
7Large FS-LSTM-4Bit per Character (BPC)1.25—Unverified
8Large mLSTMBit per Character (BPC)1.24—Unverified
9AWD-LSTM (3 layers)Bit per Character (BPC)1.23—Unverified
10Cluster-Former (#C=512)Bit per Character (BPC)1.22—Unverified
#ModelMetricClaimedVerifiedStatus
1Smaller Transformer 126M (pre-trained)Test perplexity33—Unverified
2OPT 125MTest perplexity32.26—Unverified
3Larger Transformer 771M (pre-trained)Test perplexity28.1—Unverified
4OPT 1.3BTest perplexity19.55—Unverified
5GPT-Neo 125MTest perplexity17.83—Unverified
6OPT 2.7BTest perplexity17.81—Unverified
7Smaller Transformer 126M (fine-tuned)Test perplexity12—Unverified
8GPT-Neo 1.3BTest perplexity11.46—Unverified
9Transformer 125MTest perplexity10.7—Unverified
10GPT-Neo 2.7BTest perplexity10.44—Unverified