SOTAVerified

Language Modelling

A language model is a model of natural language. Language models are useful for a variety of tasks, including speech recognition, machine translation, natural language generation (generating more human-like text), optical character recognition, route optimization, handwriting recognition, grammar induction, and information retrieval.

Large language models (LLMs), currently their most advanced form, are predominantly based on transformers trained on larger datasets (frequently using words scraped from the public internet). They have superseded recurrent neural network-based models, which had previously superseded the purely statistical models, such as word n-gram language model.

Source: Wikipedia

Papers

Showing 62016250 of 17610 papers

TitleStatusHype
SEQ^3: Differentiable Sequence-to-Sequence-to-Sequence Autoencoder for Unsupervised Abstractive Sentence CompressionCode0
Language Models Can Learn Exceptions to Syntactic RulesCode0
Social Bias in Elicited Natural Language InferencesCode0
SocialGaze: Improving the Integration of Human Social Norms in Large Language ModelsCode0
Panoramic Interests: Stylistic-Content Aware Personalized Headline GenerationCode0
Open-domain Implicit Format Control for Large Language Model GenerationCode0
Social perception of faces in a vision-language modelCode0
SEP: Self-Enhanced Prompt Tuning for Visual-Language ModelCode0
Separating the Wheat from the Chaff with BREAD: An open-source benchmark and metrics to detect redundancy in textCode0
LSTM based Conversation ModelsCode0
MAMA: Meta-optimized Angular Margin Contrastive Framework for Video-Language Representation LearningCode0
SODAPOP: Open-Ended Discovery of Social Biases in Social Commonsense Reasoning ModelsCode0
Open-Domain Dialog Evaluation using Follow-Ups LikelihoodCode0
LoRec: Large Language Model for Robust Sequential Recommendation against Poisoning AttacksCode0
Soft Contextual Data Augmentation for Neural Machine TranslationCode0
Transfer learning from language models to image caption generators: Better models may not transfer betterCode0
Making Language Model a Hierarchical Classifier and GeneratorCode0
Language Models can Self-Improve at State-Value Estimation for Better SearchCode0
Probabilities of Chat LLMs Are Miscalibrated but Still Predict Correctness on Multiple-Choice Q&ACode0
MetaSC: Test-Time Safety Specification Optimization for Language ModelsCode0
Review Conversational Reading ComprehensionCode0
Leveraging Web-Crawled Data for High-Quality Fine-TuningCode0
OpenAi's GPT4 as coding assistantCode0
Sentiment-enhanced Graph-based Sarcasm Explanation in DialogueCode0
Planning with Multi-Constraints via Collaborative Language AgentsCode0
Knowing Where and What: Unified Word Block Pretraining for Document UnderstandingCode0
Oolong: Investigating What Makes Transfer Learning Hard with Controlled StudiesCode0
Sentiment analysis in tweets: an assessment study from classical to modern text representation modelsCode0
Solving Hard Analogy Questions with Relation Embedding ChainsCode0
Large Language Model Critics for Execution-Free Evaluation of Code ChangesCode0
Reverse-Engineering the ReaderCode0
Solving Math Word Problem with Problem Type ClassificationCode0
SentiLARE: Sentiment-Aware Language Representation Learning with Linguistic KnowledgeCode0
Keep Security! Benchmarking Security Policy Preservation in Large Language Model Contexts Against Indirect Attacks in Question AnsweringCode0
Understanding Domain Learning in Language Models Through Subpopulation AnalysisCode0
Lightweight Cross-Lingual Sentence Representation LearningCode0
Solving the Right Problem is Key for Translational NLP: A Case Study in UMLS Vocabulary InsertionCode0
Recurrent Neural Networks with Pre-trained Language Model Embedding for Slot Filling TaskCode0
On Training Bi-directional Neural Network Language Model with Noise Contrastive EstimationCode0
Ontology Enrichment from Texts: A Biomedical Dataset for Concept Discovery and PlacementCode0
Meeting Summarization with Pre-training and Clustering MethodsCode0
Sentence-level Media Bias Analysis with Event Relation GraphCode0
TiEBe: Tracking Language Model Recall of Notable Worldwide Events Through TimeCode0
On the Use of ArXiv as a DatasetCode0
Language Model Sentence Completion with a Parser-Driven Rhetorical Control MethodCode0
Prompting or Fine-tuning? A Comparative Study of Large Language Models for Taxonomy ConstructionCode0
Parameter Efficient Fine Tuning Llama 3.1 for Answering Arabic Legal Questions: A Case Study on Jordanian LawsCode0
SOS-1K: A Fine-grained Suicide Risk Classification Dataset for Chinese Social Media AnalysisCode0
Sentence Encoders on STILTs: Supplementary Training on Intermediate Labeled-data TasksCode0
Cross-model Back-translated Distillation for Unsupervised Machine TranslationCode0
Show:102550
← PrevPage 125 of 353Next →

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1Decay RNNValidation perplexity76.67Unverified
2GRUValidation perplexity53.78Unverified
3LSTMValidation perplexity52.73Unverified
4LSTMTest perplexity48.7Unverified
5Temporal CNNTest perplexity45.2Unverified
6TCNTest perplexity45.19Unverified
7GCNN-8Test perplexity44.9Unverified
8Neural cache model (size = 100)Test perplexity44.8Unverified
9Neural cache model (size = 2,000)Test perplexity40.8Unverified
10GPT-2 SmallTest perplexity37.5Unverified
#ModelMetricClaimedVerifiedStatus
1TCNTest perplexity108.47Unverified
2Seq-U-NetTest perplexity107.95Unverified
3GRU (Bai et al., 2018)Test perplexity92.48Unverified
4R-TransformerTest perplexity84.38Unverified
5Zaremba et al. (2014) - LSTM (medium)Test perplexity82.7Unverified
6Gal & Ghahramani (2016) - Variational LSTM (medium)Test perplexity79.7Unverified
7LSTM (Bai et al., 2018)Test perplexity78.93Unverified
8Zaremba et al. (2014) - LSTM (large)Test perplexity78.4Unverified
9Gal & Ghahramani (2016) - Variational LSTM (large)Test perplexity75.2Unverified
10Inan et al. (2016) - Variational RHNTest perplexity66Unverified
#ModelMetricClaimedVerifiedStatus
1LSTM (7 layers)Bit per Character (BPC)1.67Unverified
2HypernetworksBit per Character (BPC)1.34Unverified
3SHA-LSTM (4 layers, h=1024, no attention head)Bit per Character (BPC)1.33Unverified
4LN HM-LSTMBit per Character (BPC)1.32Unverified
5ByteNetBit per Character (BPC)1.31Unverified
6Recurrent Highway NetworksBit per Character (BPC)1.27Unverified
7Large FS-LSTM-4Bit per Character (BPC)1.25Unverified
8Large mLSTMBit per Character (BPC)1.24Unverified
9AWD-LSTM (3 layers)Bit per Character (BPC)1.23Unverified
10Cluster-Former (#C=512)Bit per Character (BPC)1.22Unverified
#ModelMetricClaimedVerifiedStatus
1Smaller Transformer 126M (pre-trained)Test perplexity33Unverified
2OPT 125MTest perplexity32.26Unverified
3Larger Transformer 771M (pre-trained)Test perplexity28.1Unverified
4OPT 1.3BTest perplexity19.55Unverified
5GPT-Neo 125MTest perplexity17.83Unverified
6OPT 2.7BTest perplexity17.81Unverified
7Smaller Transformer 126M (fine-tuned)Test perplexity12Unverified
8GPT-Neo 1.3BTest perplexity11.46Unverified
9Transformer 125MTest perplexity10.7Unverified
10GPT-Neo 2.7BTest perplexity10.44Unverified