SOTAVerified

Language Modelling

A language model is a model of natural language. Language models are useful for a variety of tasks, including speech recognition, machine translation, natural language generation (generating more human-like text), optical character recognition, route optimization, handwriting recognition, grammar induction, and information retrieval.

Large language models (LLMs), currently their most advanced form, are predominantly based on transformers trained on larger datasets (frequently using words scraped from the public internet). They have superseded recurrent neural network-based models, which had previously superseded the purely statistical models, such as word n-gram language model.

Source: Wikipedia

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 9501–9550 of 17610 papers

TitleStatusHype
QUERT: Continual Pre-training of Language Model for Query Understanding in Travel Domain SearchCode1
RoBERTweet: A BERT Language Model for Romanian Tweets—0
Language-Guided Traffic Simulation via Scene-Level Diffusion—0
Universal Language Modelling agent—0
Improving Non-autoregressive Translation Quality with Pretrained Language Model, Embedding Distillation and Upsampling Strategy for CTC—0
Large Language Models Are Semi-Parametric Reinforcement Learning AgentsCode1
Judging LLM-as-a-Judge with MT-Bench and Chatbot ArenaCode7
PoET: A generative model of protein families as sequences-of-sequencesCode1
14 Examples of How LLMs Can Transform Materials Science and Chemistry: A Reflection on a Large Language Model HackathonCode1
AVScan2Vec: Feature Learning on Antivirus Scan Data for Production-Scale Malware Corpora—0
Aladdin: Zero-Shot Hallucination of Stylized 3D Assets from Abstract Scene DescriptionsCode1
Language Models Can Learn Exceptions to Syntactic RulesCode0
Towards a Robust Detection of Language Model Generated Text: Is ChatGPT that Easy to Detect?—0
Word sense extensionCode0
FinGPT: Open-Source Financial Large Language ModelsCode6
How Can Recommender Systems Benefit from Large Language Models: A SurveyCode3
S^3: Increasing GPU Utilization during Generative Inference for Higher Throughput—0
Simple and Controllable Music GenerationCode6
Speech-to-Text Adapter and Speech-to-Entity Retriever Augmented LLMs for Speech Understanding—0
Hexatagging: Projective Dependency Parsing as TaggingCode1
Improving Language Model Integration for Neural Machine Translation—0
InfoPrompt: Information-Theoretic Soft Prompt Tuning for Natural Language Understanding—0
In-Context Learning through the Bayesian PrismCode0
Advancing Italian Biomedical Information Extraction with Transformers-based Models: Methodological Insights and Multicenter Practical Application—0
Multi-Modal Classifiers for Open-Vocabulary Object DetectionCode1
Mapping Brains with Language Models: A Survey—0
PIXIU: A Large Language Model, Instruction Data and Evaluation Benchmark for FinanceCode2
PandaLM: An Automatic Evaluation Benchmark for LLM Instruction Tuning OptimizationCode2
K2: A Foundation Language Model for Geoscience Knowledge Understanding and UtilizationCode2
RETA-LLM: A Retrieval-Augmented Large Language Model ToolkitCode2
Mixture-of-Supernets: Improving Weight-Sharing Supernet Training with Architecture-Routed Mixture-of-ExpertsCode0
Robot Task Planning Based on Large Language Model Representing Knowledge with Directed Graph StructuresCode0
T3L: Translate-and-Test Transfer Learning for Cross-Lingual Text ClassificationCode0
Soft-prompt Tuning for Large Language Models to Evaluate Bias—0
Privately generating tabular data using language modelsCode1
Absformer: Transformer-based Model for Unsupervised Multi-Document Abstractive Summarization—0
Dial-MAE: ConTextual Masked Auto-Encoder for Retrieval-based Dialogue SystemsCode0
Can current NLI systems handle German word order? Investigating language model performance on a new German challenge set of minimal pairsCode0
Evaluation of ChatGPT on Biomedical Tasks: A Zero-Shot Comparison with Fine-Tuned Generative Transformers—0
Benchmarking Foundation Models with Language-Model-as-an-Examiner—0
Transfer Learning from Pre-trained Language Models Improves End-to-End Speech Summarization—0
Youku-mPLUG: A 10 Million Large-scale Chinese Video-Language Dataset for Pre-training and BenchmarksCode2
Knowledge-Augmented Language Model Prompting for Zero-Shot Knowledge Graph Question Answering—0
Multi-Task Training with In-Domain Language Models for Diagnostic Reasoning—0
Language Models Get a Gender Makeover: Mitigating Gender Bias with Few-Shot Data Interventions—0
ModuleFormer: Modularity Emerges from Mixture-of-ExpertsCode2
Long-form analogies generated by chatGPT lack human-like psycholinguistic properties—0
Text-only Domain Adaptation using Unified Speech-Text Representation in Transducer—0
Inference-Time Intervention: Eliciting Truthful Answers from a Language ModelCode2
LLMZip: Lossless Text Compression using Large Language ModelsCode1
Show:102550
← PrevPage 191 of 353Next →
All datasetsWikiText-103Penn Treebank (Word Level)enwik8The PileWikiText-2LAMBADAOne Billion WordText8Penn Treebank (Character Level)Hutter PrizeOpenWebTextSALMon

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1Decay RNNValidation perplexity76.67—Unverified
2GRUValidation perplexity53.78—Unverified
3LSTMValidation perplexity52.73—Unverified
4LSTMTest perplexity48.7—Unverified
5Temporal CNNTest perplexity45.2—Unverified
6TCNTest perplexity45.19—Unverified
7GCNN-8Test perplexity44.9—Unverified
8Neural cache model (size = 100)Test perplexity44.8—Unverified
9Neural cache model (size = 2,000)Test perplexity40.8—Unverified
10GPT-2 SmallTest perplexity37.5—Unverified
#ModelMetricClaimedVerifiedStatus
1TCNTest perplexity108.47—Unverified
2Seq-U-NetTest perplexity107.95—Unverified
3GRU (Bai et al., 2018)Test perplexity92.48—Unverified
4R-TransformerTest perplexity84.38—Unverified
5Zaremba et al. (2014) - LSTM (medium)Test perplexity82.7—Unverified
6Gal & Ghahramani (2016) - Variational LSTM (medium)Test perplexity79.7—Unverified
7LSTM (Bai et al., 2018)Test perplexity78.93—Unverified
8Zaremba et al. (2014) - LSTM (large)Test perplexity78.4—Unverified
9Gal & Ghahramani (2016) - Variational LSTM (large)Test perplexity75.2—Unverified
10Inan et al. (2016) - Variational RHNTest perplexity66—Unverified
#ModelMetricClaimedVerifiedStatus
1LSTM (7 layers)Bit per Character (BPC)1.67—Unverified
2HypernetworksBit per Character (BPC)1.34—Unverified
3SHA-LSTM (4 layers, h=1024, no attention head)Bit per Character (BPC)1.33—Unverified
4LN HM-LSTMBit per Character (BPC)1.32—Unverified
5ByteNetBit per Character (BPC)1.31—Unverified
6Recurrent Highway NetworksBit per Character (BPC)1.27—Unverified
7Large FS-LSTM-4Bit per Character (BPC)1.25—Unverified
8Large mLSTMBit per Character (BPC)1.24—Unverified
9AWD-LSTM (3 layers)Bit per Character (BPC)1.23—Unverified
10Cluster-Former (#C=512)Bit per Character (BPC)1.22—Unverified
#ModelMetricClaimedVerifiedStatus
1Smaller Transformer 126M (pre-trained)Test perplexity33—Unverified
2OPT 125MTest perplexity32.26—Unverified
3Larger Transformer 771M (pre-trained)Test perplexity28.1—Unverified
4OPT 1.3BTest perplexity19.55—Unverified
5GPT-Neo 125MTest perplexity17.83—Unverified
6OPT 2.7BTest perplexity17.81—Unverified
7Smaller Transformer 126M (fine-tuned)Test perplexity12—Unverified
8GPT-Neo 1.3BTest perplexity11.46—Unverified
9Transformer 125MTest perplexity10.7—Unverified
10GPT-Neo 2.7BTest perplexity10.44—Unverified