SOTAVerified

Language Modelling

A language model is a model of natural language. Language models are useful for a variety of tasks, including speech recognition, machine translation, natural language generation (generating more human-like text), optical character recognition, route optimization, handwriting recognition, grammar induction, and information retrieval.

Large language models (LLMs), currently their most advanced form, are predominantly based on transformers trained on larger datasets (frequently using words scraped from the public internet). They have superseded recurrent neural network-based models, which had previously superseded the purely statistical models, such as word n-gram language model.

Source: Wikipedia

Papers

Showing 751800 of 17610 papers

TitleStatusHype
biorecap: an R package for summarizing bioRxiv preprints with a local LLMCode2
Critique-out-Loud Reward ModelsCode2
BEYOND DIALOGUE: A Profile-Dialogue Alignment Framework Towards General Role-Playing Language ModelCode2
PA-LLaVA: A Large Language-Vision Assistant for Human Pathology Image UnderstandingCode2
RoarGraph: A Projected Bipartite Graph for Efficient Cross-Modal Approximate Nearest Neighbor SearchCode2
ECG-Chat: A Large ECG-Language Model for Cardiac Disease DiagnosisCode2
EasyRec: Simple yet Effective Language Models for RecommendationCode2
Text2BIM: Generating Building Models Using a Large Language Model-based Multi-Agent FrameworkCode2
MedTsLLM: Leveraging LLMs for Multimodal Medical Time Series AnalysisCode2
ChemVLM: Exploring the Power of Multimodal Large Language Models in Chemistry AreaCode2
Causal Agent based on Large Language ModelCode2
Trans-Tokenization and Cross-lingual Vocabulary Transfers: Language Adaptation of LLMs for Low-Resource NLPCode2
500xCompressor: Generalized Prompt Compression for Large Language ModelsCode2
XMainframe: A Large Language Model for Mainframe ModernizationCode2
Improving Text Embeddings for Smaller Language Models Using Contrastive Fine-tuningCode2
DeliLaw: A Chinese Legal Counselling System Based on a Large Language ModelCode2
DiffArtist: Towards Structure and Appearance Controllable Image StylizationCode2
RAG-QA Arena: Evaluating Domain Robustness for Long-form Retrieval Augmented Question AnsweringCode2
Longhorn: State Space Models are Amortized Online LearnersCode2
T2V-CompBench: A Comprehensive Benchmark for Compositional Text-to-video GenerationCode2
Spectra: Surprising Effectiveness of Pretraining Ternary Language Models at ScaleCode2
Beyond Next Token Prediction: Patch-Level Training for Large Language ModelsCode2
LaMI-DETR: Open-Vocabulary Detection with Language Model InstructionCode2
UrbanWorld: An Urban World Model for 3D City GenerationCode2
DOCBENCH: A Benchmark for Evaluating LLM-based Document Reading SystemsCode2
Think-on-Graph 2.0: Deep and Faithful Large Language Model Reasoning with Knowledge-guided Retrieval Augmented GenerationCode2
AutoGRAMS: Autonomous Graphical Agent Modeling SoftwareCode2
GOFA: A Generative One-For-All Model for Joint Graph Language ModelingCode2
FBI-LLM: Scaling Up Fully Binarized LLMs from Scratch via Autoregressive DistillationCode2
Multimodal Self-Instruct: Synthetic Abstract Image and Visual Reasoning Instruction Using Language ModelCode2
SOLO: A Single Transformer for Scalable Vision-Language ModelingCode2
PsycoLLM: Enhancing LLM for Psychological Understanding and EvaluationCode2
iLLM-TSC: Integration reinforcement learning and large language model for traffic signal control policy improvementCode2
Just read twice: closing the recall gap for recurrent language modelsCode2
Language Representations Can be What Recommenders Need: Findings and PotentialsCode2
Mixture of A Million ExpertsCode2
MiniGPT-Med: Large Language Model as a General Interface for Radiology DiagnosisCode2
Planetarium: A Rigorous Benchmark for Translating Text to Structured Planning LanguagesCode2
A Bounding Box is Worth One Token: Interleaving Layout and Text in a Large Language Model for Document UnderstandingCode2
AutoFlow: Automated Workflow Generation for Large Language Model AgentsCode2
IBSEN: Director-Actor Agent Collaboration for Controllable and Interactive Drama Script GenerationCode2
RegMix: Data Mixture as Regression for Language Model Pre-trainingCode2
Learning Formal Mathematics From Intrinsic MotivationCode2
Teola: Towards End-to-End Optimization of LLM-based ApplicationsCode2
RoboUniView: Visual-Language Model with Unified View Representation for Robotic ManipulationCode2
Finding Transformer Circuits with Edge PruningCode2
EDGE-LLM: Enabling Efficient Large Language Model Adaptation on Edge Devices via Layerwise Unified Compression and Adaptive Layer Tuning and VotingCode2
MoA: Mixture of Sparse Attention for Automatic Large Language Model CompressionCode2
FIRST: Faster Improved Listwise Reranking with Single Token DecodingCode2
LLM-A*: Large Language Model Enhanced Incremental Heuristic Search on Path PlanningCode2
Show:102550
← PrevPage 16 of 353Next →

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1Decay RNNValidation perplexity76.67Unverified
2GRUValidation perplexity53.78Unverified
3LSTMValidation perplexity52.73Unverified
4LSTMTest perplexity48.7Unverified
5Temporal CNNTest perplexity45.2Unverified
6TCNTest perplexity45.19Unverified
7GCNN-8Test perplexity44.9Unverified
8Neural cache model (size = 100)Test perplexity44.8Unverified
9Neural cache model (size = 2,000)Test perplexity40.8Unverified
10GPT-2 SmallTest perplexity37.5Unverified
#ModelMetricClaimedVerifiedStatus
1TCNTest perplexity108.47Unverified
2Seq-U-NetTest perplexity107.95Unverified
3GRU (Bai et al., 2018)Test perplexity92.48Unverified
4R-TransformerTest perplexity84.38Unverified
5Zaremba et al. (2014) - LSTM (medium)Test perplexity82.7Unverified
6Gal & Ghahramani (2016) - Variational LSTM (medium)Test perplexity79.7Unverified
7LSTM (Bai et al., 2018)Test perplexity78.93Unverified
8Zaremba et al. (2014) - LSTM (large)Test perplexity78.4Unverified
9Gal & Ghahramani (2016) - Variational LSTM (large)Test perplexity75.2Unverified
10Inan et al. (2016) - Variational RHNTest perplexity66Unverified
#ModelMetricClaimedVerifiedStatus
1LSTM (7 layers)Bit per Character (BPC)1.67Unverified
2HypernetworksBit per Character (BPC)1.34Unverified
3SHA-LSTM (4 layers, h=1024, no attention head)Bit per Character (BPC)1.33Unverified
4LN HM-LSTMBit per Character (BPC)1.32Unverified
5ByteNetBit per Character (BPC)1.31Unverified
6Recurrent Highway NetworksBit per Character (BPC)1.27Unverified
7Large FS-LSTM-4Bit per Character (BPC)1.25Unverified
8Large mLSTMBit per Character (BPC)1.24Unverified
9AWD-LSTM (3 layers)Bit per Character (BPC)1.23Unverified
10Cluster-Former (#C=512)Bit per Character (BPC)1.22Unverified
#ModelMetricClaimedVerifiedStatus
1Smaller Transformer 126M (pre-trained)Test perplexity33Unverified
2OPT 125MTest perplexity32.26Unverified
3Larger Transformer 771M (pre-trained)Test perplexity28.1Unverified
4OPT 1.3BTest perplexity19.55Unverified
5GPT-Neo 125MTest perplexity17.83Unverified
6OPT 2.7BTest perplexity17.81Unverified
7Smaller Transformer 126M (fine-tuned)Test perplexity12Unverified
8GPT-Neo 1.3BTest perplexity11.46Unverified
9Transformer 125MTest perplexity10.7Unverified
10GPT-Neo 2.7BTest perplexity10.44Unverified