SOTAVerified

Language Modelling

A language model is a model of natural language. Language models are useful for a variety of tasks, including speech recognition, machine translation, natural language generation (generating more human-like text), optical character recognition, route optimization, handwriting recognition, grammar induction, and information retrieval.

Large language models (LLMs), currently their most advanced form, are predominantly based on transformers trained on larger datasets (frequently using words scraped from the public internet). They have superseded recurrent neural network-based models, which had previously superseded the purely statistical models, such as word n-gram language model.

Source: Wikipedia

Papers

Showing 26012650 of 17610 papers

TitleStatusHype
HYTREL: Hypergraph-enhanced Tabular Data Representation LearningCode1
Gloss Attention for Gloss-free Sign Language TranslationCode1
Copy Is All You NeedCode1
In-context Autoencoder for Context Compression in a Large Language ModelCode1
Epidemic Modeling with Generative AgentsCode1
OntoChatGPT Information System: Ontology-Driven Structured Prompts for ChatGPT Meta-LearningCode1
Exploring Large Language Model for Graph Data Understanding in Online Job RecommendationsCode1
Linear Alignment of Vision-language Models for Image CaptioningCode1
ChatGPT in the Age of Generative AI and Large Language Models: A Concise SurveyCode1
ScriptWorld: Text Based Environment For Learning Procedural KnowledgeCode1
Opening up ChatGPT: Tracking openness, transparency, and accountability in instruction-tuned text generatorsCode1
LaunchpadGPT: Language Model as Music Visualization Designer on LaunchpadCode1
PRD: Peer Rank and Discussion Improve Large Language Model based EvaluationsCode1
Distilling Large Vision-Language Model with Out-of-Distribution GeneralizabilityCode1
Reasoning or Reciting? Exploring the Capabilities and Limitations of Language Models Through Counterfactual TasksCode1
Flacuna: Unleashing the Problem Solving Power of Vicuna using FLAN Fine-TuningCode1
Causal Discovery with Language Models as Imperfect ExpertsCode1
Trainable Transformer in TransformerCode1
GenRec: Large Language Model for Generative RecommendationCode1
Large Language Models Enable Few-Shot ClusteringCode1
How far is Language Model from 100% Few-shot Named Entity Recognition in Medical DomainCode1
LMBot: Distilling Graph Knowledge into Language Model for Graph-less Deployment in Twitter Bot DetectionCode1
LyricWhiz: Robust Multilingual Zero-shot Lyrics Transcription by Whispering to ChatGPTCode1
Could Small Language Models Serve as Recommenders? Towards Data-centric Cold-start RecommendationsCode1
Large Language Model as Attributed Training Data Generator: A Tale of Diversity and BiasCode1
Protein-DNA binding sites prediction based on pre-trained protein language model and contrastive learningCode1
Palm: Predicting Actions through Language Models @ Ego4D Long-Term Action Anticipation Challenge 2023Code1
Enhancing Dialogue Generation via Dynamic Graph Knowledge AggregationCode1
Composing Parameter-Efficient Modules with Arithmetic OperationsCode1
Kosmos-2: Grounding Multimodal Large Language Models to the WorldCode1
Fauno: The Italian Large Language Model that will leave you senza parole!Code1
DesCo: Learning Object Recognition with Rich Language DescriptionsCode1
Implementing contextual biasing in GPU decoder for online ASRCode1
Bring Your Own Data! Self-Supervised Evaluation for Large Language ModelsCode1
Generative Multimodal Entity LinkingCode1
NoRefER: a Referenceless Quality Metric for Automatic Speech Recognition via Semi-Supervised Language Model Fine-Tuning with Contrastive LearningCode1
Mass-Producing Failures of Multimodal Systems with Language ModelsCode1
OphGLM: Training an Ophthalmology Large Language-and-Vision Assistant based on Instructions and DialogueCode1
A Reference-less Quality Metric for Automatic Speech Recognition via Contrastive-Learning of a Multi-Language Model with Self-SupervisionCode1
Sparse Modular Activation for Efficient Sequence ModelingCode1
LLMVA-GEBC: Large Language Model with Video Adapter for Generic Event Boundary CaptioningCode1
Just One Byte (per gradient): A Note on Low-Bandwidth Decentralized Language Model Finetuning Using Shared RandomnessCode1
Conformal Language ModelingCode1
FALL-E: A Foley Sound Synthesis Model and StrategiesCode1
ChessGPT: Bridging Policy Learning and Language ModelingCode1
Semantic HELM: A Human-Readable Memory for Reinforcement LearningCode1
Pushing the Limits of Unsupervised Unit Discovery for SSL Speech RepresentationCode1
Propagating Knowledge Updates to LMs Through DistillationCode1
World-to-Words: Grounded Open Vocabulary Acquisition through Fast Mapping in Vision-Language ModelsCode1
Generate to Understand for RepresentationCode1
Show:102550
← PrevPage 53 of 353Next →

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1Decay RNNValidation perplexity76.67Unverified
2GRUValidation perplexity53.78Unverified
3LSTMValidation perplexity52.73Unverified
4LSTMTest perplexity48.7Unverified
5Temporal CNNTest perplexity45.2Unverified
6TCNTest perplexity45.19Unverified
7GCNN-8Test perplexity44.9Unverified
8Neural cache model (size = 100)Test perplexity44.8Unverified
9Neural cache model (size = 2,000)Test perplexity40.8Unverified
10GPT-2 SmallTest perplexity37.5Unverified
#ModelMetricClaimedVerifiedStatus
1TCNTest perplexity108.47Unverified
2Seq-U-NetTest perplexity107.95Unverified
3GRU (Bai et al., 2018)Test perplexity92.48Unverified
4R-TransformerTest perplexity84.38Unverified
5Zaremba et al. (2014) - LSTM (medium)Test perplexity82.7Unverified
6Gal & Ghahramani (2016) - Variational LSTM (medium)Test perplexity79.7Unverified
7LSTM (Bai et al., 2018)Test perplexity78.93Unverified
8Zaremba et al. (2014) - LSTM (large)Test perplexity78.4Unverified
9Gal & Ghahramani (2016) - Variational LSTM (large)Test perplexity75.2Unverified
10Inan et al. (2016) - Variational RHNTest perplexity66Unverified
#ModelMetricClaimedVerifiedStatus
1LSTM (7 layers)Bit per Character (BPC)1.67Unverified
2HypernetworksBit per Character (BPC)1.34Unverified
3SHA-LSTM (4 layers, h=1024, no attention head)Bit per Character (BPC)1.33Unverified
4LN HM-LSTMBit per Character (BPC)1.32Unverified
5ByteNetBit per Character (BPC)1.31Unverified
6Recurrent Highway NetworksBit per Character (BPC)1.27Unverified
7Large FS-LSTM-4Bit per Character (BPC)1.25Unverified
8Large mLSTMBit per Character (BPC)1.24Unverified
9AWD-LSTM (3 layers)Bit per Character (BPC)1.23Unverified
10Cluster-Former (#C=512)Bit per Character (BPC)1.22Unverified
#ModelMetricClaimedVerifiedStatus
1Smaller Transformer 126M (pre-trained)Test perplexity33Unverified
2OPT 125MTest perplexity32.26Unverified
3Larger Transformer 771M (pre-trained)Test perplexity28.1Unverified
4OPT 1.3BTest perplexity19.55Unverified
5GPT-Neo 125MTest perplexity17.83Unverified
6OPT 2.7BTest perplexity17.81Unverified
7Smaller Transformer 126M (fine-tuned)Test perplexity12Unverified
8GPT-Neo 1.3BTest perplexity11.46Unverified
9Transformer 125MTest perplexity10.7Unverified
10GPT-Neo 2.7BTest perplexity10.44Unverified