SOTAVerified

Language Modelling

A language model is a model of natural language. Language models are useful for a variety of tasks, including speech recognition, machine translation, natural language generation (generating more human-like text), optical character recognition, route optimization, handwriting recognition, grammar induction, and information retrieval.

Large language models (LLMs), currently their most advanced form, are predominantly based on transformers trained on larger datasets (frequently using words scraped from the public internet). They have superseded recurrent neural network-based models, which had previously superseded the purely statistical models, such as word n-gram language model.

Source: Wikipedia

Papers

Showing 83518400 of 17610 papers

TitleStatusHype
Investigating the effect of auxiliary objectives for the automated grading of learner English speech transcriptions0
Effects of sub-word segmentation on performance of transformer language models0
Investigating the Effects of Large-Scale Pseudo-Stereo Data and Different Speech Foundation Model on Dialogue Generative Spoken Language Model0
Investigating the Impact of Text Summarization on Topic Modeling0
Investigating the Impact of Word Informativeness on Speech Emotion Recognition0
Investigating the Potential of Large Language Model-Based Router Multi-Agent Architectures for Foundation Design Automation: A Task Classification and Expert Selection Study0
Investigating the Synergistic Effects of Dropout and Residual Connections on Language Model Training0
Investigating the Timescales of Language Processing with EEG and Language Models0
Investigating Training Strategies and Model Robustness of Low-Rank Adaptation for Language Modeling in Speech Recognition0
Investigating Vision-Language Model for Point Cloud-based Vehicle Classification0
Investigation of Japanese PnG BERT language model in text-to-speech synthesis for pitch accent language0
Investigation of Large-Margin Softmax in Neural Language Modeling0
Investigation on N-gram Approximated RNNLMs for Recognition of Morphologically Rich Speech0
Investigations in Exact Inference for Hierarchical Translation0
Investigations on Phrase-based Decoding with Recurrent Neural Network Language and Translation Models0
INVESTORBENCH: A Benchmark for Financial Decision-Making Tasks with LLM-based Agent0
IP-MOT: Instance Prompt Learning for Cross-Domain Multi-Object Tracking0
IPPON: Common Sense Guided Informative Path Planning for Object Goal Navigation0
iPrOp: Interactive Prompt Optimization for Large Language Models with a Human in the Loop0
IQLS: Framework for leveraging Metadata to enable Large Language Model based queries to complex, versatile Data0
IRCologne at GermEval 2021: Toxicity Classification0
iREPO: implicit Reward Pairwise Difference based Empirical Preference Optimization0
IRISA participation to BioNLP-ST13: lazy-learning and information retrieval for information extraction tasks0
IRIT at TRAC 20200
Irreducible Curriculum for Language Model Pretraining0
Irrelevant Alternatives Bias Large Language Model Hiring Decisions0
Is a 3D-Tokenized LLM the Key to Reliable Autonomous Driving?0
Is Bad Structure Better Than No Structure?: Unsupervised Parsing for Realisation Ranking0
Is ChatGPT a Financial Expert? Evaluating Language Models on Financial Natural Language Processing0
Is ChatGPT a Highly Fluent Grammatical Error Correction System? A Comprehensive Evaluation0
Is ChatGPT Equipped with Emotional Dialogue Capabilities?0
Is Context Helpful for Chat Translation Evaluation?0
Is Crowdsourcing Breaking Your Bank? Cost-Effective Fine-Tuning of Pre-trained Language Models with Proximal Policy Optimization0
I See Dead People: Gray-Box Adversarial Attack on Image-To-Text Models0
Is Einstein more agreeable and less neurotic than Hitler? A computational exploration of the emotional and personality profiles of historical persons0
Is Encoder-Decoder Redundant for Neural Machine Translation?0
Is English the New Programming Language? How About Pseudo-code Engineering?0
Is GPT-4 a reliable rater? Evaluating Consistency in GPT-4 Text Ratings0
Is it an i or an l: Test-time Adaptation of Text Line Recognition Models0
Is it Possible to Modify Text to a Target Readability Level? An Initial Investigation Using Zero-Shot Large Language Models0
Is Language Modeling Enough? Evaluating Effective Embedding Combinations0
Is Large Language Model Good at Triple Set Prediction? An Empirical Study0
Is My Model Using The Right Evidence? Systematic Probes for Examining Evidence-Based Tabular Reasoning0
Is My Text in Your AI Model? Gradient-based Membership Inference Test applied to LLMs0
ISO: Overlap of Computation and Communication within Seqenence For LLM Inference0
Toward Trustworthy Neural Program Synthesis0
Unveiling Code Pre-Trained Models: Investigating Syntax and Semantics Capacities0
Is Supervised Syntactic Parsing Beneficial for Language Understanding Tasks? An Empirical Investigation0
Is Surprisal in Issue Trackers Actionable?0
Is There Any Social Principle for LLM-Based Agents?0
Show:102550
← PrevPage 168 of 353Next →

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1Decay RNNValidation perplexity76.67Unverified
2GRUValidation perplexity53.78Unverified
3LSTMValidation perplexity52.73Unverified
4LSTMTest perplexity48.7Unverified
5Temporal CNNTest perplexity45.2Unverified
6TCNTest perplexity45.19Unverified
7GCNN-8Test perplexity44.9Unverified
8Neural cache model (size = 100)Test perplexity44.8Unverified
9Neural cache model (size = 2,000)Test perplexity40.8Unverified
10GPT-2 SmallTest perplexity37.5Unverified
#ModelMetricClaimedVerifiedStatus
1TCNTest perplexity108.47Unverified
2Seq-U-NetTest perplexity107.95Unverified
3GRU (Bai et al., 2018)Test perplexity92.48Unverified
4R-TransformerTest perplexity84.38Unverified
5Zaremba et al. (2014) - LSTM (medium)Test perplexity82.7Unverified
6Gal & Ghahramani (2016) - Variational LSTM (medium)Test perplexity79.7Unverified
7LSTM (Bai et al., 2018)Test perplexity78.93Unverified
8Zaremba et al. (2014) - LSTM (large)Test perplexity78.4Unverified
9Gal & Ghahramani (2016) - Variational LSTM (large)Test perplexity75.2Unverified
10Inan et al. (2016) - Variational RHNTest perplexity66Unverified
#ModelMetricClaimedVerifiedStatus
1LSTM (7 layers)Bit per Character (BPC)1.67Unverified
2HypernetworksBit per Character (BPC)1.34Unverified
3SHA-LSTM (4 layers, h=1024, no attention head)Bit per Character (BPC)1.33Unverified
4LN HM-LSTMBit per Character (BPC)1.32Unverified
5ByteNetBit per Character (BPC)1.31Unverified
6Recurrent Highway NetworksBit per Character (BPC)1.27Unverified
7Large FS-LSTM-4Bit per Character (BPC)1.25Unverified
8Large mLSTMBit per Character (BPC)1.24Unverified
9AWD-LSTM (3 layers)Bit per Character (BPC)1.23Unverified
10Cluster-Former (#C=512)Bit per Character (BPC)1.22Unverified
#ModelMetricClaimedVerifiedStatus
1Smaller Transformer 126M (pre-trained)Test perplexity33Unverified
2OPT 125MTest perplexity32.26Unverified
3Larger Transformer 771M (pre-trained)Test perplexity28.1Unverified
4OPT 1.3BTest perplexity19.55Unverified
5GPT-Neo 125MTest perplexity17.83Unverified
6OPT 2.7BTest perplexity17.81Unverified
7Smaller Transformer 126M (fine-tuned)Test perplexity12Unverified
8GPT-Neo 1.3BTest perplexity11.46Unverified
9Transformer 125MTest perplexity10.7Unverified
10GPT-Neo 2.7BTest perplexity10.44Unverified