SOTAVerified

Language Modelling

A language model is a model of natural language. Language models are useful for a variety of tasks, including speech recognition, machine translation, natural language generation (generating more human-like text), optical character recognition, route optimization, handwriting recognition, grammar induction, and information retrieval.

Large language models (LLMs), currently their most advanced form, are predominantly based on transformers trained on larger datasets (frequently using words scraped from the public internet). They have superseded recurrent neural network-based models, which had previously superseded the purely statistical models, such as word n-gram language model.

Source: Wikipedia

Papers

Showing 24512500 of 17610 papers

TitleStatusHype
Reasoning on Graphs: Faithful and Interpretable Large Language Model ReasoningCode1
What's the Magic Word? A Control Theory of LLM PromptingCode1
CAT-LM: Training Language Models on Aligned Code And TestsCode1
Fool Your (Vision and) Language Model With Embarrassingly Simple PermutationsCode1
A Framework for Inference Inspired by Human Memory MechanismsCode1
Empowering Many, Biasing a Few: Generalist Credit Scoring through Large Language ModelsCode1
AutomaTikZ: Text-Guided Synthesis of Scientific Vector Graphics with TikZCode1
NAYER: Noisy Layer Data Generation for Efficient and Effective Data-free Knowledge DistillationCode1
From Language Modeling to Instruction Following: Understanding the Behavior Shift in LLMs after Instruction TuningCode1
AdaRefiner: Refining Decisions of Language Models with Adaptive FeedbackCode1
Motif: Intrinsic Motivation from Artificial Intelligence FeedbackCode1
MotionLM: Multi-Agent Motion Forecasting as Language ModelingCode1
ChatCounselor: A Large Language Models for Mental Health SupportCode1
Graph Neural Prompting with Large Language ModelsCode1
AnyMAL: An Efficient and Scalable Any-Modality Augmented Language ModelCode1
HyPoradise: An Open Baseline for Generative Speech Recognition with Large Language ModelsCode1
MindGPT: Interpreting What You See with Non-invasive Brain RecordingsCode1
Identifying the Risks of LM Agents with an LM-Emulated SandboxCode1
LogGPT: Log Anomaly Detection via GPTCode1
Speaker anonymization using neural audio codec language modelsCode1
A Text Classification-Based Approach for Evaluating and Enhancing the Machine Interpretability of Building CodesCode1
BAMBOO: A Comprehensive Benchmark for Evaluating Long Text Modeling Capacities of Large Language ModelsCode1
GlotScript: A Resource and Tool for Low Resource Writing System IdentificationCode1
DRG-LLaMA : Tuning LLaMA Model to Predict Diagnosis-related Group for Hospitalized PatientsCode1
AceGPT, Localizing Large Language Models in ArabicCode1
A Sentence Speaks a Thousand Images: Domain Generalization through Distilling CLIP with Language GuidanceCode1
Striking Gold in Advertising: Standardization and Exploration of Ad Text GenerationCode1
Towards Joint Modeling of Dialogue Response and Speech Synthesis based on Large Language ModelCode1
CPLLM: Clinical Prediction with Large Language ModelsCode1
Retrieve-Rewrite-Answer: A KG-to-Text Enhanced LLMs Framework for Knowledge Graph Question AnsweringCode1
Natural Language Embedded Programs for Hybrid Language Symbolic ReasoningCode1
CFGPT: Chinese Financial Assistant with Large Language ModelCode1
Large language models can accurately predict searcher preferencesCode1
Language Modeling Is CompressionCode1
Estimating Contamination via Perplexity: Quantifying Memorisation in Language Model EvaluationCode1
LLM Platform Security: Applying a Systematic Evaluation Framework to OpenAI's ChatGPT PluginsCode1
Progressive Text-to-Image Diffusion with Soft Latent DirectionCode1
Embrace Divergence for Richer Insights: A Multi-document Summarization Benchmark and a Case Study on Summarizing Diverse Information from News ArticlesCode1
Exploring the impact of low-rank adaptation on the performance, efficiency, and regularization of RLHFCode1
MusiLingo: Bridging Music and Text with Pre-trained Language Models for Music Captioning and Query ResponseCode1
FedJudge: Federated Legal Large Language ModelCode1
InvestLM: A Large Language Model for Investment using Financial Domain Instruction TuningCode1
Find What You Want: Learning Demand-conditioned Object Attribute Space for Demand-driven NavigationCode1
ExpertQA: Expert-Curated Questions and Attributed AnswersCode1
SIB-200: A Simple, Inclusive, and Big Evaluation Dataset for Topic Classification in 200+ Languages and DialectsCode1
TextBind: Multi-turn Interleaved Multimodal Instruction-following in the WildCode1
TrafficGPT: Viewing, Processing and Interacting with Traffic Foundation ModelsCode1
Adaptive KalmanNet: Data-Driven Kalman Filter with Fast AdaptationCode1
Evaluation of large language models for discovery of gene set functionCode1
Supervised Learning and Large Language Model Benchmarks on Mental Health Datasets: Cognitive Distortions and Suicidal Risks in Chinese Social MediaCode1
Show:102550
← PrevPage 50 of 353Next →

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1Decay RNNValidation perplexity76.67Unverified
2GRUValidation perplexity53.78Unverified
3LSTMValidation perplexity52.73Unverified
4LSTMTest perplexity48.7Unverified
5Temporal CNNTest perplexity45.2Unverified
6TCNTest perplexity45.19Unverified
7GCNN-8Test perplexity44.9Unverified
8Neural cache model (size = 100)Test perplexity44.8Unverified
9Neural cache model (size = 2,000)Test perplexity40.8Unverified
10GPT-2 SmallTest perplexity37.5Unverified
#ModelMetricClaimedVerifiedStatus
1TCNTest perplexity108.47Unverified
2Seq-U-NetTest perplexity107.95Unverified
3GRU (Bai et al., 2018)Test perplexity92.48Unverified
4R-TransformerTest perplexity84.38Unverified
5Zaremba et al. (2014) - LSTM (medium)Test perplexity82.7Unverified
6Gal & Ghahramani (2016) - Variational LSTM (medium)Test perplexity79.7Unverified
7LSTM (Bai et al., 2018)Test perplexity78.93Unverified
8Zaremba et al. (2014) - LSTM (large)Test perplexity78.4Unverified
9Gal & Ghahramani (2016) - Variational LSTM (large)Test perplexity75.2Unverified
10Inan et al. (2016) - Variational RHNTest perplexity66Unverified
#ModelMetricClaimedVerifiedStatus
1LSTM (7 layers)Bit per Character (BPC)1.67Unverified
2HypernetworksBit per Character (BPC)1.34Unverified
3SHA-LSTM (4 layers, h=1024, no attention head)Bit per Character (BPC)1.33Unverified
4LN HM-LSTMBit per Character (BPC)1.32Unverified
5ByteNetBit per Character (BPC)1.31Unverified
6Recurrent Highway NetworksBit per Character (BPC)1.27Unverified
7Large FS-LSTM-4Bit per Character (BPC)1.25Unverified
8Large mLSTMBit per Character (BPC)1.24Unverified
9AWD-LSTM (3 layers)Bit per Character (BPC)1.23Unverified
10Cluster-Former (#C=512)Bit per Character (BPC)1.22Unverified
#ModelMetricClaimedVerifiedStatus
1Smaller Transformer 126M (pre-trained)Test perplexity33Unverified
2OPT 125MTest perplexity32.26Unverified
3Larger Transformer 771M (pre-trained)Test perplexity28.1Unverified
4OPT 1.3BTest perplexity19.55Unverified
5GPT-Neo 125MTest perplexity17.83Unverified
6OPT 2.7BTest perplexity17.81Unverified
7Smaller Transformer 126M (fine-tuned)Test perplexity12Unverified
8GPT-Neo 1.3BTest perplexity11.46Unverified
9Transformer 125MTest perplexity10.7Unverified
10GPT-Neo 2.7BTest perplexity10.44Unverified