SOTAVerified

Language Modelling

A language model is a model of natural language. Language models are useful for a variety of tasks, including speech recognition, machine translation, natural language generation (generating more human-like text), optical character recognition, route optimization, handwriting recognition, grammar induction, and information retrieval.

Large language models (LLMs), currently their most advanced form, are predominantly based on transformers trained on larger datasets (frequently using words scraped from the public internet). They have superseded recurrent neural network-based models, which had previously superseded the purely statistical models, such as word n-gram language model.

Source: Wikipedia

Papers

Showing 27012750 of 17610 papers

TitleStatusHype
Improving Generalization in Language Model-Based Text-to-SQL Semantic Parsing: Two Simple Semantic Boundary-Based TechniquesCode1
Backpack Language ModelsCode1
LLMs and the Abstraction and Reasoning Corpus: Successes, Failures, and the Importance of Object-based RepresentationsCode1
Schema-Guided User Satisfaction Modeling for Task-Oriented DialoguesCode1
Language Models Implement Simple Word2Vec-style Vector ArithmeticCode1
GenerateCT: Text-Conditional Generation of 3D Chest CT VolumesCode1
ChatBridge: Bridging Modalities with Large Language Model as a Language CatalystCode1
Leftover Lunch: Advantage-based Offline Reinforcement Learning for Language ModelsCode1
Think Before You Act: Decision Transformers with Working MemoryCode1
LLMDet: A Third Party Large Language Models Generated Text Detection ToolCode1
SPRING: Studying the Paper and Reasoning to Play GamesCode1
PIVOINE: Instruction Tuning for Open-world Information ExtractionCode1
PathAsst: A Generative Foundation AI Assistant Towards Artificial General Intelligence of PathologyCode1
Inference-Time Policy Adapters (IPA): Tailoring Extreme-Scale LMs without Fine-tuningCode1
ClusterLLM: Large Language Models as a Guide for Text ClusteringCode1
C-STS: Conditional Semantic Textual SimilarityCode1
Text-Augmented Open Knowledge Graph Completion via Pre-Trained Language ModelsCode1
ComSL: A Composite Speech-Language Model for End-to-End Speech-to-Text TranslationCode1
MQuAKE: Assessing Knowledge Editing in Language Models via Multi-Hop QuestionsCode1
The Art of SOCRATIC QUESTIONING: Recursive Thinking with Large Language ModelsCode1
Meta-Learning Online Adaptation of Language ModelsCode1
An Efficient Multilingual Language Model Compression through Vocabulary TrimmingCode1
Automatic Model Selection with Large Language Models for ReasoningCode1
CLIP4STR: A Simple Baseline for Scene Text Recognition with Pre-trained Vision-Language ModelCode1
Preserving Knowledge Invariance: Rethinking Robustness Evaluation of Open Information ExtractionCode1
VisorGPT: Learning Visual Prior via Generative Pre-TrainingCode1
Aligning Large Language Models through Synthetic FeedbackCode1
MathDial: A Dialogue Tutoring Dataset with Rich Pedagogical Properties Grounded in Math Reasoning ProblemsCode1
RetICL: Sequential Retrieval of In-Context Examples with Reinforcement LearningCode1
Prompt-Based Monte-Carlo Tree Search for Goal-Oriented Dialogue Policy PlanningCode1
ConGraT: Self-Supervised Contrastive Pretraining for Joint Graph and Text EmbeddingsCode1
Goal-Driven Explainable Clustering via Language DescriptionsCode1
FOCUS: Effective Embedding Initialization for Monolingual Specialization of Multilingual ModelsCode1
Textually Pretrained Speech Language ModelsCode1
GQA: Training Generalized Multi-Query Transformer Models from Multi-Head CheckpointsCode1
A Study of Generative Large Language Model for Medical Research and HealthcareCode1
Making Language Models Better Tool Learners with Execution FeedbackCode1
Chain-of-Knowledge: Grounding Large Language Models via Dynamic Knowledge Adapting over Heterogeneous SourcesCode1
MvP: Multi-view Prompting Improves Aspect Sentiment Tuple PredictionCode1
How Language Model Hallucinations Can SnowballCode1
A Frustratingly Simple Decoding Method for Neural Text GenerationCode1
Word Embeddings Are Steers for Language ModelsCode1
PiVe: Prompting with Iterative Verification Improving Graph-based Generative Capability of LLMsCode1
Abstract Meaning Representation-Based Logic-Driven Data Augmentation for Logical ReasoningCode1
DisCo: Distilled Student Models Co-training for Semi-supervised Text MiningCode1
ESCOXLM-R: Multilingual Taxonomy-driven Pre-training for the Job Market DomainCode1
Glot500: Scaling Multilingual Corpora and Language Models to 500 LanguagesCode1
Empower Large Language Model to Perform Better on Industrial Domain-Specific Question AnsweringCode1
Syllable Discovery and Cross-Lingual Generalization in a Visually Grounded, Self-Supervised Speech ModelCode1
Decouple knowledge from parameters for plug-and-play language modelingCode1
Show:102550
← PrevPage 55 of 353Next →

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1Decay RNNValidation perplexity76.67Unverified
2GRUValidation perplexity53.78Unverified
3LSTMValidation perplexity52.73Unverified
4LSTMTest perplexity48.7Unverified
5Temporal CNNTest perplexity45.2Unverified
6TCNTest perplexity45.19Unverified
7GCNN-8Test perplexity44.9Unverified
8Neural cache model (size = 100)Test perplexity44.8Unverified
9Neural cache model (size = 2,000)Test perplexity40.8Unverified
10GPT-2 SmallTest perplexity37.5Unverified
#ModelMetricClaimedVerifiedStatus
1TCNTest perplexity108.47Unverified
2Seq-U-NetTest perplexity107.95Unverified
3GRU (Bai et al., 2018)Test perplexity92.48Unverified
4R-TransformerTest perplexity84.38Unverified
5Zaremba et al. (2014) - LSTM (medium)Test perplexity82.7Unverified
6Gal & Ghahramani (2016) - Variational LSTM (medium)Test perplexity79.7Unverified
7LSTM (Bai et al., 2018)Test perplexity78.93Unverified
8Zaremba et al. (2014) - LSTM (large)Test perplexity78.4Unverified
9Gal & Ghahramani (2016) - Variational LSTM (large)Test perplexity75.2Unverified
10Inan et al. (2016) - Variational RHNTest perplexity66Unverified
#ModelMetricClaimedVerifiedStatus
1LSTM (7 layers)Bit per Character (BPC)1.67Unverified
2HypernetworksBit per Character (BPC)1.34Unverified
3SHA-LSTM (4 layers, h=1024, no attention head)Bit per Character (BPC)1.33Unverified
4LN HM-LSTMBit per Character (BPC)1.32Unverified
5ByteNetBit per Character (BPC)1.31Unverified
6Recurrent Highway NetworksBit per Character (BPC)1.27Unverified
7Large FS-LSTM-4Bit per Character (BPC)1.25Unverified
8Large mLSTMBit per Character (BPC)1.24Unverified
9AWD-LSTM (3 layers)Bit per Character (BPC)1.23Unverified
10Cluster-Former (#C=512)Bit per Character (BPC)1.22Unverified
#ModelMetricClaimedVerifiedStatus
1Smaller Transformer 126M (pre-trained)Test perplexity33Unverified
2OPT 125MTest perplexity32.26Unverified
3Larger Transformer 771M (pre-trained)Test perplexity28.1Unverified
4OPT 1.3BTest perplexity19.55Unverified
5GPT-Neo 125MTest perplexity17.83Unverified
6OPT 2.7BTest perplexity17.81Unverified
7Smaller Transformer 126M (fine-tuned)Test perplexity12Unverified
8GPT-Neo 1.3BTest perplexity11.46Unverified
9Transformer 125MTest perplexity10.7Unverified
10GPT-Neo 2.7BTest perplexity10.44Unverified