SOTAVerified

Language Modelling

A language model is a model of natural language. Language models are useful for a variety of tasks, including speech recognition, machine translation, natural language generation (generating more human-like text), optical character recognition, route optimization, handwriting recognition, grammar induction, and information retrieval.

Large language models (LLMs), currently their most advanced form, are predominantly based on transformers trained on larger datasets (frequently using words scraped from the public internet). They have superseded recurrent neural network-based models, which had previously superseded the purely statistical models, such as word n-gram language model.

Source: Wikipedia

Papers

Showing 75017550 of 17610 papers

TitleStatusHype
Train-Attention: Meta-Learning Where to Focus in Continual Knowledge LearningCode0
SimCT: A Simple Consistency Test Protocol in LLMs Development Lifecycle0
SDoH-GPT: Using Large Language Models to Extract Social Determinants of Health (SDoH)0
Towards Transfer Unlearning: Empirical Evidence of Cross-Domain Bias Mitigation0
ViPer: Visual Personalization of Generative Models via Individual Preference Learning0
AMONGAGENTS: Evaluating Large Language Models in the Interactive Text-Based Social Deduction GameCode0
Do LLMs Know When to NOT Answer? Investigating Abstention Abilities of Large Language Models0
FACTTRACK: Time-Aware World State Tracking in Story Outlines0
How to Leverage Personal Textual Knowledge for Personalized Conversational Information RetrievalCode0
A Comparative Study on Patient Language across Therapeutic Domains for Effective Patient Voice Classification in Online Health Discussions0
Graph-Structured Speculative Decoding0
Occlusion-Aware 3D Motion Interpretation for Abnormal Behavior Detection0
Networks of Networks: Complexity Class Principles Applied to Compound AI Systems Design0
Quantifying the Role of Textual Predictability in Automatic Speech Recognition0
PreAlign: Boosting Cross-Lingual Transfer by Early Establishment of Multilingual AlignmentCode0
Lawma: The Power of Specialization for Legal Tasks0
Promises and Pitfalls of Generative Masked Language Modeling: Theoretical Framework and Practical Guidelines0
Text-to-Battery Recipe: A language modeling-based protocol for automatic battery recipe extraction and retrieval0
Pre-Training and Prompting for Few-Shot Node Classification on Text-Attributed Graphs0
LLMExplainer: Large Language Model based Bayesian Inference for Graph Explanation Generation0
SAM2CLIP2SAM: Vision Language Model for Segmentation of 3D CT Scans for Covid-19 Detection0
SocialQuotes: Learning Contextual Roles of Social Media Quotes on the Web0
OMoS-QA: A Dataset for Cross-Lingual Extractive Question Answering in a German Migration Context0
J-CHAT: Japanese Large-scale Spoken Dialogue Corpus for Spoken Dialogue Language Modeling0
A Comparison of Language Modeling and Translation as Multilingual Pretraining ObjectivesCode0
ALLaM: Large Language Models for Arabic and English0
Decoding BACnet Packets: A Large Language Model Approach for Packet Interpretation0
Evidence-Based Temporal Fact Verification0
ReAttention: Training-Free Infinite Context with Finite Attention Scope0
Failures to Find Transferable Image Jailbreaks Between Vision-Language Models0
Large Language Model for Verilog Generation with Code-Structure-Guided Reinforcement LearningCode0
Learning to Compile Programs to Neural Networks0
Relational Database Augmented Large Language Model0
Seal: Advancing Speech Language Models to be Few-Shot Learners0
Generalization v.s. Memorization: Tracing Language Models' Capabilities Back to Pretraining Data0
Falcon2-11B Technical Report0
Can VLMs be used on videos for action recognition? LLMs are Visual Reasoning Coordinators0
Unipa-GPT: Large Language Models for university-oriented QA in ItalianCode0
CVE-LLM : Automatic vulnerability evaluation in medical device industry using large language models0
Contrastive Learning with Counterfactual Explanations for Radiology Report Generation0
Generative Language Model for Catalyst Discovery0
Automatic Classification of News Subjects in Broadcast News: Application to a Gender Bias Representation AnalysisCode0
EVLM: An Efficient Vision-Language Model for Visual Understanding0
LAPIS: Language Model-Augmented Police Investigation System0
Semantic-CC: Boosting Remote Sensing Image Change Captioning via Foundational Knowledge and Semantic Guidance0
Mixture of Experts with Mixture of Precisions for Tuning Quality of Service0
Large Language Model Enabled Semantic Communication Systems0
Performance Modeling and Workload Analysis of Distributed Large Language Model Training and Inference0
LLMs left, right, and center: Assessing GPT's capabilities to label political bias from web domains0
PD-APE: A Parallel Decoding Framework with Adaptive Position Encoding for 3D Visual Grounding0
Show:102550
← PrevPage 151 of 353Next →

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1Decay RNNValidation perplexity76.67Unverified
2GRUValidation perplexity53.78Unverified
3LSTMValidation perplexity52.73Unverified
4LSTMTest perplexity48.7Unverified
5Temporal CNNTest perplexity45.2Unverified
6TCNTest perplexity45.19Unverified
7GCNN-8Test perplexity44.9Unverified
8Neural cache model (size = 100)Test perplexity44.8Unverified
9Neural cache model (size = 2,000)Test perplexity40.8Unverified
10GPT-2 SmallTest perplexity37.5Unverified
#ModelMetricClaimedVerifiedStatus
1TCNTest perplexity108.47Unverified
2Seq-U-NetTest perplexity107.95Unverified
3GRU (Bai et al., 2018)Test perplexity92.48Unverified
4R-TransformerTest perplexity84.38Unverified
5Zaremba et al. (2014) - LSTM (medium)Test perplexity82.7Unverified
6Gal & Ghahramani (2016) - Variational LSTM (medium)Test perplexity79.7Unverified
7LSTM (Bai et al., 2018)Test perplexity78.93Unverified
8Zaremba et al. (2014) - LSTM (large)Test perplexity78.4Unverified
9Gal & Ghahramani (2016) - Variational LSTM (large)Test perplexity75.2Unverified
10Inan et al. (2016) - Variational RHNTest perplexity66Unverified
#ModelMetricClaimedVerifiedStatus
1LSTM (7 layers)Bit per Character (BPC)1.67Unverified
2HypernetworksBit per Character (BPC)1.34Unverified
3SHA-LSTM (4 layers, h=1024, no attention head)Bit per Character (BPC)1.33Unverified
4LN HM-LSTMBit per Character (BPC)1.32Unverified
5ByteNetBit per Character (BPC)1.31Unverified
6Recurrent Highway NetworksBit per Character (BPC)1.27Unverified
7Large FS-LSTM-4Bit per Character (BPC)1.25Unverified
8Large mLSTMBit per Character (BPC)1.24Unverified
9AWD-LSTM (3 layers)Bit per Character (BPC)1.23Unverified
10Cluster-Former (#C=512)Bit per Character (BPC)1.22Unverified
#ModelMetricClaimedVerifiedStatus
1Smaller Transformer 126M (pre-trained)Test perplexity33Unverified
2OPT 125MTest perplexity32.26Unverified
3Larger Transformer 771M (pre-trained)Test perplexity28.1Unverified
4OPT 1.3BTest perplexity19.55Unverified
5GPT-Neo 125MTest perplexity17.83Unverified
6OPT 2.7BTest perplexity17.81Unverified
7Smaller Transformer 126M (fine-tuned)Test perplexity12Unverified
8GPT-Neo 1.3BTest perplexity11.46Unverified
9Transformer 125MTest perplexity10.7Unverified
10GPT-Neo 2.7BTest perplexity10.44Unverified