SOTAVerified

Language Modelling

A language model is a model of natural language. Language models are useful for a variety of tasks, including speech recognition, machine translation, natural language generation (generating more human-like text), optical character recognition, route optimization, handwriting recognition, grammar induction, and information retrieval.

Large language models (LLMs), currently their most advanced form, are predominantly based on transformers trained on larger datasets (frequently using words scraped from the public internet). They have superseded recurrent neural network-based models, which had previously superseded the purely statistical models, such as word n-gram language model.

Source: Wikipedia

Papers

Showing 1625116300 of 17610 papers

TitleStatusHype
Toward a Deeper Understanding: RetNet Viewed through ConvolutionCode0
Do Large Language Model Understand Multi-Intent Spoken Language ?Code0
Few-shot learning through contextual data augmentationCode0
Clustering of Deep Contextualized Representations for Summarization of Biomedical TextsCode0
Do Large Language Models Solve ARC Visual Analogies Like People Do?Code0
Analysing Word Representation from the Input and Output Embeddings in Neural Network Language ModelsCode0
Do Large Language Models know what humans know?Code0
On the Usefulness of Embeddings, Clusters and Strings for Text Generator EvaluationCode0
InRanker: Distilled Rankers for Zero-shot Information RetrievalCode0
Few-Shot NLG with Pre-Trained Language ModelCode0
Identifying Nuances in Fake News vs. Satire: Using Semantic and Linguistic CuesCode0
Do language models plan ahead for future tokens?Code0
Attention-Seeker: Dynamic Self-Attention Scoring for Unsupervised Keyphrase ExtractionCode0
Attention as a Guide for Simultaneous Speech TranslationCode0
Do Language Models Know When They're Hallucinating References?Code0
In Search of the Long-Tail: Systematic Generation of Long-Tail Inferential Knowledge via Logical Rule Guided SearchCode0
Drop Dropout on Single-Epoch Language Model PretrainingCode0
DropMicroFluidAgents (DMFAs): Autonomous Droplet Microfluidic Research Framework Through Large Language Model AgentsCode0
InternLM-XComposer2-4KHD: A Pioneering Large Vision-Language Model Handling Resolutions from 336 Pixels to 4K HDCode0
DRPruning: Efficient Large Language Model Pruning through Distributionally Robust OptimizationCode0
Grid Long Short-Term MemoryCode0
Do Language Models Exhibit Human-like Structural Priming Effects?Code0
DrugChat: Towards Enabling ChatGPT-Like Capabilities on Drug Molecule GraphsCode0
DrugImproverGPT: A Large Language Model for Drug Optimization with Fine-Tuning via Structured Policy OptimizationCode0
Few-Shot Upsampling for Protest Size DetectionCode0
Identifying Reasons for Contraceptive Switching from Real-World Data Using Large Language ModelsCode0
DrugTar Improves Druggability Prediction by Integrating Large Language Models and Gene OntologiesCode0
DSC IIT-ISM at SemEval-2020 Task 6: Boosting BERT with Dependencies for Definition ExtractionCode0
Do Images really do the Talking? Analysing the significance of Images in Tamil Troll meme classificationCode0
DSGram: Dynamic Weighting Sub-Metrics for Grammatical Error Correction in the Era of Large Language ModelsCode0
Attacks on Third-Party APIs of Large Language ModelsCode0
Closing the Loop: Learning to Generate Writing Feedback via Language Model Simulated Student RevisionsCode0
A Hybrid Approach to Automatic Corpus Generation for Chinese Spelling CheckCode0
Active Inference for Self-Organizing Multi-LLM Systems: A Bayesian Thermodynamic Approach to AdaptationCode0
Does Unlearning Truly Unlearn? A Black Box Evaluation of LLM Unlearning MethodsCode0
DS-TOD: Efficient Domain Specialization for Task Oriented DialogCode0
FGeo-DRL: Deductive Reasoning for Geometric Problems through Deep Reinforcement LearningCode0
DS-TOD: Efficient Domain Specialization for Task-Oriented DialogCode0
Does Transliteration Help Multilingual Language Modeling?Code0
Does Commonsense help in detecting Sarcasm?Code0
Document Screenshot Retrievers are Vulnerable to Pixel Poisoning AttacksCode0
An agentic system with reinforcement-learned subsystem improvements for parsing form-like documentsCode0
Document Modeling with External Attention for Sentence ExtractionCode0
A Transformer with Stack AttentionCode0
FIDAVL: Fake Image Detection and Attribution using Vision-Language ModelCode0
Document Informed Neural Autoregressive Topic ModelsCode0
A Training Data Recipe to Accelerate A* Search with Language ModelsCode0
Fidelity-Enriched Contrastive Search: Reconciling the Faithfulness-Diversity Trade-Off in Text GenerationCode0
A Toolkit for Efficient Learning of Lexical Units for Speech RecognitionCode0
Dual Learning for Machine TranslationCode0
Show:102550
← PrevPage 326 of 353Next →

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1Decay RNNValidation perplexity76.67Unverified
2GRUValidation perplexity53.78Unverified
3LSTMValidation perplexity52.73Unverified
4LSTMTest perplexity48.7Unverified
5Temporal CNNTest perplexity45.2Unverified
6TCNTest perplexity45.19Unverified
7GCNN-8Test perplexity44.9Unverified
8Neural cache model (size = 100)Test perplexity44.8Unverified
9Neural cache model (size = 2,000)Test perplexity40.8Unverified
10GPT-2 SmallTest perplexity37.5Unverified
#ModelMetricClaimedVerifiedStatus
1TCNTest perplexity108.47Unverified
2Seq-U-NetTest perplexity107.95Unverified
3GRU (Bai et al., 2018)Test perplexity92.48Unverified
4R-TransformerTest perplexity84.38Unverified
5Zaremba et al. (2014) - LSTM (medium)Test perplexity82.7Unverified
6Gal & Ghahramani (2016) - Variational LSTM (medium)Test perplexity79.7Unverified
7LSTM (Bai et al., 2018)Test perplexity78.93Unverified
8Zaremba et al. (2014) - LSTM (large)Test perplexity78.4Unverified
9Gal & Ghahramani (2016) - Variational LSTM (large)Test perplexity75.2Unverified
10Inan et al. (2016) - Variational RHNTest perplexity66Unverified
#ModelMetricClaimedVerifiedStatus
1LSTM (7 layers)Bit per Character (BPC)1.67Unverified
2HypernetworksBit per Character (BPC)1.34Unverified
3SHA-LSTM (4 layers, h=1024, no attention head)Bit per Character (BPC)1.33Unverified
4LN HM-LSTMBit per Character (BPC)1.32Unverified
5ByteNetBit per Character (BPC)1.31Unverified
6Recurrent Highway NetworksBit per Character (BPC)1.27Unverified
7Large FS-LSTM-4Bit per Character (BPC)1.25Unverified
8Large mLSTMBit per Character (BPC)1.24Unverified
9AWD-LSTM (3 layers)Bit per Character (BPC)1.23Unverified
10Cluster-Former (#C=512)Bit per Character (BPC)1.22Unverified
#ModelMetricClaimedVerifiedStatus
1Smaller Transformer 126M (pre-trained)Test perplexity33Unverified
2OPT 125MTest perplexity32.26Unverified
3Larger Transformer 771M (pre-trained)Test perplexity28.1Unverified
4OPT 1.3BTest perplexity19.55Unverified
5GPT-Neo 125MTest perplexity17.83Unverified
6OPT 2.7BTest perplexity17.81Unverified
7Smaller Transformer 126M (fine-tuned)Test perplexity12Unverified
8GPT-Neo 1.3BTest perplexity11.46Unverified
9Transformer 125MTest perplexity10.7Unverified
10GPT-Neo 2.7BTest perplexity10.44Unverified