SOTAVerified

Language Modelling

A language model is a model of natural language. Language models are useful for a variety of tasks, including speech recognition, machine translation, natural language generation (generating more human-like text), optical character recognition, route optimization, handwriting recognition, grammar induction, and information retrieval.

Large language models (LLMs), currently their most advanced form, are predominantly based on transformers trained on larger datasets (frequently using words scraped from the public internet). They have superseded recurrent neural network-based models, which had previously superseded the purely statistical models, such as word n-gram language model.

Source: Wikipedia

Papers

Showing 21512200 of 17610 papers

TitleStatusHype
Factorized Neural Transducer for Efficient Language Model AdaptationCode1
Extracting Training Data from Large Language ModelsCode1
Facilitating large language model Russian adaptation with Learned Embedding PropagationCode1
A Sentence Speaks a Thousand Images: Domain Generalization through Distilling CLIP with Language GuidanceCode1
Extracting Definienda in Mathematical Scholarly Articles with TransformersCode1
CORAL: Expert-Curated medical Oncology Reports to Advance Language Model InferenceCode1
Bootstrapping Interactive Image-Text Alignment for Remote Sensing Image CaptioningCode1
A Batch Normalized Inference Network Keeps the KL Vanishing AwayCode1
Extracting Cultural Commonsense Knowledge at ScaleCode1
Extracting Latent Steering Vectors from Pretrained Language ModelsCode1
Factual Probing Is [MASK]: Learning vs. Learning to RecallCode1
Exposing Numeracy Gaps: A Benchmark to Evaluate Fundamental Numerical Abilities in Large Language ModelsCode1
Extending Large Vision-Language Model for Diverse Interactive Tasks in Autonomous DrivingCode1
Exploring Versatile Generative Language Model Via Parameter-Efficient Transfer LearningCode1
A Cross-Modal Approach to Silent Speech with LLM-Enhanced RecognitionCode1
Extensive Self-Contrast Enables Feedback-Free Language Model AlignmentCode1
Exploring Structured Semantic Prior for Multi Label Recognition with Incomplete LabelsCode1
Exploring Quantization for Efficient Pre-Training of Transformer Language ModelsCode1
Exploring the impact of low-rank adaptation on the performance, efficiency, and regularization of RLHFCode1
Exploring Large Language Model for Graph Data Understanding in Online Job RecommendationsCode1
Exploring the Limits of Language ModelingCode1
Extracting and Inferring Personal Attributes from DialogueCode1
FADE: Few-shot/zero-shot Anomaly Detection Engine using Large Vision-Language ModelCode1
Feature Structure Distillation with Centered Kernel Alignment in BERT TransferringCode1
GeoGalactica: A Scientific Large Language Model in GeoscienceCode1
A Second Wave of UD Hebrew Treebanking and Cross-Domain ParsingCode1
Explaining Datasets in Words: Statistical Models with Natural Language ParametersCode1
Exploiting BERT For Multimodal Target Sentiment Classification Through Input Space TranslationCode1
ExpertQA: Expert-Curated Questions and Attributed AnswersCode1
Explaining Answers with Entailment TreesCode1
Exploiting Cloze Questions for Few Shot Text Classification and Natural Language InferenceCode1
Excuse me, sir? Your language model is leaking (information)Code1
Expert Knowledge-Aware Image Difference Graph Representation Learning for Difference-Aware Medical Visual Question AnsweringCode1
ExaRanker: Explanation-Augmented Neural RankerCode1
ARWKV: Pretrain is not what we need, an RNN-Attention-Based Language Model Born from TransformerCode1
Exchange-of-Thought: Enhancing Large Language Model Capabilities through Cross-Model CommunicationCode1
Exploiting Novel GPT-4 APIsCode1
e-ViL: A Dataset and Benchmark for Natural Language Explanations in Vision-Language TasksCode1
UnifiedMLLM: Enabling Unified Representation for Multi-modal Multi-tasks With Large Language ModelCode1
Emergent Representations of Program Semantics in Language Models Trained on ProgramsCode1
Evolutionary Large Language Model for Automated Feature TransformationCode1
Fact-checking information from large language models can decrease headline discernmentCode1
Evaluation of large language models for discovery of gene set functionCode1
Evaluation Benchmarks for Spanish Sentence RepresentationsCode1
A Critical Analysis of Biased Parsers in Unsupervised ParsingCode1
Can ChatGPT Replace Traditional KBQA Models? An In-depth Analysis of the Question Answering Performance of the GPT LLM FamilyCode1
Event Causality Identification via Derivative Prompt Joint LearningCode1
Evolving Deep Neural NetworksCode1
ArtGPT-4: Towards Artistic-understanding Large Vision-Language Models with Enhanced AdapterCode1
ART: Automatic Red-teaming for Text-to-Image Models to Protect Benign UsersCode1
Show:102550
← PrevPage 44 of 353Next →

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1Decay RNNValidation perplexity76.67Unverified
2GRUValidation perplexity53.78Unverified
3LSTMValidation perplexity52.73Unverified
4LSTMTest perplexity48.7Unverified
5Temporal CNNTest perplexity45.2Unverified
6TCNTest perplexity45.19Unverified
7GCNN-8Test perplexity44.9Unverified
8Neural cache model (size = 100)Test perplexity44.8Unverified
9Neural cache model (size = 2,000)Test perplexity40.8Unverified
10GPT-2 SmallTest perplexity37.5Unverified
#ModelMetricClaimedVerifiedStatus
1TCNTest perplexity108.47Unverified
2Seq-U-NetTest perplexity107.95Unverified
3GRU (Bai et al., 2018)Test perplexity92.48Unverified
4R-TransformerTest perplexity84.38Unverified
5Zaremba et al. (2014) - LSTM (medium)Test perplexity82.7Unverified
6Gal & Ghahramani (2016) - Variational LSTM (medium)Test perplexity79.7Unverified
7LSTM (Bai et al., 2018)Test perplexity78.93Unverified
8Zaremba et al. (2014) - LSTM (large)Test perplexity78.4Unverified
9Gal & Ghahramani (2016) - Variational LSTM (large)Test perplexity75.2Unverified
10Inan et al. (2016) - Variational RHNTest perplexity66Unverified
#ModelMetricClaimedVerifiedStatus
1LSTM (7 layers)Bit per Character (BPC)1.67Unverified
2HypernetworksBit per Character (BPC)1.34Unverified
3SHA-LSTM (4 layers, h=1024, no attention head)Bit per Character (BPC)1.33Unverified
4LN HM-LSTMBit per Character (BPC)1.32Unverified
5ByteNetBit per Character (BPC)1.31Unverified
6Recurrent Highway NetworksBit per Character (BPC)1.27Unverified
7Large FS-LSTM-4Bit per Character (BPC)1.25Unverified
8Large mLSTMBit per Character (BPC)1.24Unverified
9AWD-LSTM (3 layers)Bit per Character (BPC)1.23Unverified
10Cluster-Former (#C=512)Bit per Character (BPC)1.22Unverified
#ModelMetricClaimedVerifiedStatus
1Smaller Transformer 126M (pre-trained)Test perplexity33Unverified
2OPT 125MTest perplexity32.26Unverified
3Larger Transformer 771M (pre-trained)Test perplexity28.1Unverified
4OPT 1.3BTest perplexity19.55Unverified
5GPT-Neo 125MTest perplexity17.83Unverified
6OPT 2.7BTest perplexity17.81Unverified
7Smaller Transformer 126M (fine-tuned)Test perplexity12Unverified
8GPT-Neo 1.3BTest perplexity11.46Unverified
9Transformer 125MTest perplexity10.7Unverified
10GPT-Neo 2.7BTest perplexity10.44Unverified