SOTAVerified

Language Modelling

A language model is a model of natural language. Language models are useful for a variety of tasks, including speech recognition, machine translation, natural language generation (generating more human-like text), optical character recognition, route optimization, handwriting recognition, grammar induction, and information retrieval.

Large language models (LLMs), currently their most advanced form, are predominantly based on transformers trained on larger datasets (frequently using words scraped from the public internet). They have superseded recurrent neural network-based models, which had previously superseded the purely statistical models, such as word n-gram language model.

Source: Wikipedia

Papers

Showing 1020110250 of 17610 papers

TitleStatusHype
PEACH: Pre-Training Sequence-to-Sequence Multilingual Models for Translation with Semi-Supervised Pseudo-Parallel Document GenerationCode0
Pythia: A Suite for Analyzing Large Language Models Across Training and ScalingCode6
Multi-Modal Perceiver Language Model for Outcome Prediction in Emergency Department0
Exploring the Use of Large Language Models for Reference-Free Text Quality Evaluation: An Empirical Study0
Baize: An Open-Source Chat Model with Parameter-Efficient Tuning on Self-Chat DataCode4
Open-Vocabulary Semantic Segmentation with Decoupled One-Pass NetworkCode1
PK-Chat: Pointer Network Guided Knowledge Driven Generative Dialogue ModelCode0
Demonstration of InsightPilot: An LLM-Empowered Automated Data Exploration System0
A Measurement-Based Quantum-Like Language Model for Text Matching0
Network Visualization of ChatGPT Research: a study based on term and keyword co-occurrence network analysis0
CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model SocietyCode6
Extracting Thyroid Nodules Characteristics from Ultrasound Reports Using Transformer-based Natural Language Processing Methods0
Identifying Symptoms of Delirium from Clinical Narratives Using Natural Language Processing0
A Survey of Large Language ModelsCode6
Assessing Language Model Deployment with Risk CardsCode5
Pair Programming with Large Language Models for Sampling and Estimation of Copulas0
Quick Dense Retrievers Consume KALE: Post Training Kullback Leibler Alignment of Embeddings for Asymmetrical dual encoders0
Language Models can Solve Computer TasksCode2
Prefix tuning for automated audio captioningCode1
A BERT-based Unsupervised Grammatical Error Correction Framework0
Vision-Language Modelling For Radiological Imaging and Reports In The Low Data Regime0
WavCaps: A ChatGPT-Assisted Weakly-Labelled Audio Captioning Dataset for Audio-Language Multimodal ResearchCode2
The Nordic Pile: A 1.2TB Nordic Dataset for Language Modeling0
DERA: Enhancing Large Language Model Completions with Dialog-Enabled Resolving Agents0
Elastic Weight Removal for Faithful and Abstractive Dialogue GenerationCode1
BloombergGPT: A Large Language Model for Finance0
Advances in apparent conceptual physics reasoning in GPT-40
Improving Large Language Models for Clinical Named Entity Recognition via Prompt EngineeringCode1
ProtFIM: Fill-in-Middle Protein Sequence Design via Protein Language Models0
Mask-free OVIS: Open-Vocabulary Instance Segmentation without Manual Mask Annotations0
Reference-less Analysis of Context Specificity in Translation with Personalised Language ModelsCode0
Larger Probes Tell a Different Story: Extending Psycholinguistic Datasets Via In-Context LearningCode0
Joint unsupervised and supervised learning for context-aware language identification0
Model and Evaluation: Towards Fairness in Multilingual Text Classification0
Structured Video-Language Modeling with Temporal Grouping and Spatial Grounding0
Hallucinations in Large Multilingual Translation ModelsCode1
Training Language Models with Language Feedback at ScaleCode1
LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init AttentionCode5
Planning with Sequence Models through Iterative Energy Minimization0
Debiasing Scores and Prompts of 2D Diffusion for View-consistent Text-to-3D Generation0
Linguistically Informed ChatGPT Prompts to Enhance Japanese-Chinese Machine Translation: A Case Study on Attributive Clauses0
LMCanvas: Object-Oriented Interaction to Personalize Large Language Model-Powered Writing Environments0
Unified Text Structuralization with Instruction-tuned Language Models0
Typhoon: Towards an Effective Task-Specific Masking Strategy for Pre-trained Language Models0
Cross-utterance ASR Rescoring with Graph-based Label Propagation0
Fine-grained Audible Video DescriptionCode1
Gazeformer: Scalable, Effective and Fast Prediction of Goal-Directed Human AttentionCode1
WinCLIP: Zero-/Few-Shot Anomaly Classification and SegmentationCode2
SmartBook: AI-Assisted Situation Report Generation for Intelligence AnalystsCode1
Backdoor Attacks with Input-unique Triggers in NLP0
Show:102550
← PrevPage 205 of 353Next →

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1Decay RNNValidation perplexity76.67Unverified
2GRUValidation perplexity53.78Unverified
3LSTMValidation perplexity52.73Unverified
4LSTMTest perplexity48.7Unverified
5Temporal CNNTest perplexity45.2Unverified
6TCNTest perplexity45.19Unverified
7GCNN-8Test perplexity44.9Unverified
8Neural cache model (size = 100)Test perplexity44.8Unverified
9Neural cache model (size = 2,000)Test perplexity40.8Unverified
10GPT-2 SmallTest perplexity37.5Unverified
#ModelMetricClaimedVerifiedStatus
1TCNTest perplexity108.47Unverified
2Seq-U-NetTest perplexity107.95Unverified
3GRU (Bai et al., 2018)Test perplexity92.48Unverified
4R-TransformerTest perplexity84.38Unverified
5Zaremba et al. (2014) - LSTM (medium)Test perplexity82.7Unverified
6Gal & Ghahramani (2016) - Variational LSTM (medium)Test perplexity79.7Unverified
7LSTM (Bai et al., 2018)Test perplexity78.93Unverified
8Zaremba et al. (2014) - LSTM (large)Test perplexity78.4Unverified
9Gal & Ghahramani (2016) - Variational LSTM (large)Test perplexity75.2Unverified
10Inan et al. (2016) - Variational RHNTest perplexity66Unverified
#ModelMetricClaimedVerifiedStatus
1LSTM (7 layers)Bit per Character (BPC)1.67Unverified
2HypernetworksBit per Character (BPC)1.34Unverified
3SHA-LSTM (4 layers, h=1024, no attention head)Bit per Character (BPC)1.33Unverified
4LN HM-LSTMBit per Character (BPC)1.32Unverified
5ByteNetBit per Character (BPC)1.31Unverified
6Recurrent Highway NetworksBit per Character (BPC)1.27Unverified
7Large FS-LSTM-4Bit per Character (BPC)1.25Unverified
8Large mLSTMBit per Character (BPC)1.24Unverified
9AWD-LSTM (3 layers)Bit per Character (BPC)1.23Unverified
10Cluster-Former (#C=512)Bit per Character (BPC)1.22Unverified
#ModelMetricClaimedVerifiedStatus
1Smaller Transformer 126M (pre-trained)Test perplexity33Unverified
2OPT 125MTest perplexity32.26Unverified
3Larger Transformer 771M (pre-trained)Test perplexity28.1Unverified
4OPT 1.3BTest perplexity19.55Unverified
5GPT-Neo 125MTest perplexity17.83Unverified
6OPT 2.7BTest perplexity17.81Unverified
7Smaller Transformer 126M (fine-tuned)Test perplexity12Unverified
8GPT-Neo 1.3BTest perplexity11.46Unverified
9Transformer 125MTest perplexity10.7Unverified
10GPT-Neo 2.7BTest perplexity10.44Unverified