SOTAVerified

Language Modelling

A language model is a model of natural language. Language models are useful for a variety of tasks, including speech recognition, machine translation, natural language generation (generating more human-like text), optical character recognition, route optimization, handwriting recognition, grammar induction, and information retrieval.

Large language models (LLMs), currently their most advanced form, are predominantly based on transformers trained on larger datasets (frequently using words scraped from the public internet). They have superseded recurrent neural network-based models, which had previously superseded the purely statistical models, such as word n-gram language model.

Source: Wikipedia

Papers

Showing 1150111550 of 17610 papers

TitleStatusHype
Now It Sounds Like You: Learning Personalized Vocabulary On Device0
Mask The Bias: Improving Domain-Adaptive Generalization of CTC-based ASR with Internal Language Model Estimation0
Simulating H.P. Lovecraft horror literature with the ChatGPT large language model0
Retrieval Augmented Chest X-Ray Report Generation using OpenAI GPT models0
Towards antigenic peptide discovery with better MHC-I binding prediction and improved benchmark methodology0
Exploring Softly Masked Language Modelling for Controllable Symbolic Music Generation0
A Low-Resource Approach to the Grammatical Error Correction of UkrainianCode0
Improving Code Example Recommendations on Informal Documentation Using BERT and Query-Aware LSH: A Comparative StudyCode0
Gpt-4: A Review on Advancements and Opportunities in Natural Language Processing0
Hybrid Transducer and Attention based Encoder-Decoder Modeling for Speech-to-Text Tasks0
FormNetV2: Multimodal Graph Contrastive Learning for Form Document Information Extraction0
Personalized Abstractive Summarization by Tri-agent Generation PipelineCode0
Interpretable Sentence Representation with Variational Autoencoders and Attention0
DN at SemEval-2023 Task 12: Low-Resource Language Text Classification via Multilingual Pretrained Language Model Fine-tuning0
Leveraging BERT Language Model for Arabic Long Document Classification0
Surveying Generative AI's Economic Expectations0
Towards Imperceptible Document Manipulations against Neural Ranking Models0
Using Language Models on Low-end Hardware0
Zero-Shot Listwise Document Reranking with a Large Language Model0
ChatGraph: Interpretable Text Classification by Converting ChatGPT Knowledge to GraphsCode0
WangLab at MEDIQA-Chat 2023: Clinical Note Generation from Doctor-Patient Conversations using Large Language Models0
Defending against Insertion-based Textual Backdoor Attacks via AttributionCode0
How to Unleash the Power of Large Language Models for Few-shot Relation Extraction?Code0
FreeLM: Fine-Tuning-Free Language Model0
KEPLET: Knowledge-Enhanced Pretrained Language Model with Topic Entity Awareness0
Self-Evaluation Guided Beam Search for Reasoning0
Retrieving Comparative Arguments using Ensemble Methods and Neural Information Retrieval0
CLIP-S^4: Language-Guided Self-Supervised Semantic Segmentation0
An Iterative Algorithm for Rescaled Hyperbolic Functions Regression0
A Review of ChatGPT Applications in Education, Marketing, Software Engineering, and Healthcare: Benefits, Drawbacks, and Research Directions0
Synthetic Cross-language Information Retrieval Training Data0
NLNDE at SemEval-2023 Task 12: Adaptive Pretraining and Source Language Selection for Low-Resource Multilingual Sentiment Analysis0
Explainable Verbal Reasoner Plus (EVR+): A Natural Language Reasoning Framework that Supports Diverse Compositional ReasoningCode0
ChatGPT Evaluation on Sentence Level Relations: A Focus on Temporal, Causal, and Discourse Relations0
ChatGPT in the Classroom: An Analysis of Its Strengths and Weaknesses for Solving Undergraduate Computer Science Questions0
Framing the News:From Human Perception to Large Language Model Inferences0
Controlled Text Generation with Natural Language Instructions0
Energy-based Models are Zero-Shot Planners for Compositional Scene Rearrangement0
A Modular Approach for Multilingual Timex Detection and Normalization using Deep Learning and Grammar-based methodsCode0
SweCTRL-Mini: a data-transparent Transformer-based large language model for controllable text generation in SwedishCode0
q2d: Turning Questions into Dialogs to Teach Models How to Search0
Large Language Models are Strong Zero-Shot Retriever0
UIO at SemEval-2023 Task 12: Multilingual fine-tuning for sentiment classification in low-resource languagesCode0
Learning Human-Human Interactions in Images from Weak Textual Supervision0
ZeroShotDataAug: Generating and Augmenting Training Data with ChatGPT0
Vision Conformer: Incorporating Convolutions into Vision Transformer LayersCode0
Extracting Structured Seed-Mediated Gold Nanorod Growth Procedures from Literature with GPT-30
MasonNLP+ at SemEval-2023 Task 8: Extracting Medical Questions, Experiences and Claims from Social Media using Knowledge-Augmented Pre-trained Language Models0
State Spaces Aren't Enough: Machine Translation Needs Attention0
KINLP at SemEval-2023 Task 12: Kinyarwanda Tweet Sentiment Analysis0
Show:102550
← PrevPage 231 of 353Next →

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1Decay RNNValidation perplexity76.67Unverified
2GRUValidation perplexity53.78Unverified
3LSTMValidation perplexity52.73Unverified
4LSTMTest perplexity48.7Unverified
5Temporal CNNTest perplexity45.2Unverified
6TCNTest perplexity45.19Unverified
7GCNN-8Test perplexity44.9Unverified
8Neural cache model (size = 100)Test perplexity44.8Unverified
9Neural cache model (size = 2,000)Test perplexity40.8Unverified
10GPT-2 SmallTest perplexity37.5Unverified
#ModelMetricClaimedVerifiedStatus
1TCNTest perplexity108.47Unverified
2Seq-U-NetTest perplexity107.95Unverified
3GRU (Bai et al., 2018)Test perplexity92.48Unverified
4R-TransformerTest perplexity84.38Unverified
5Zaremba et al. (2014) - LSTM (medium)Test perplexity82.7Unverified
6Gal & Ghahramani (2016) - Variational LSTM (medium)Test perplexity79.7Unverified
7LSTM (Bai et al., 2018)Test perplexity78.93Unverified
8Zaremba et al. (2014) - LSTM (large)Test perplexity78.4Unverified
9Gal & Ghahramani (2016) - Variational LSTM (large)Test perplexity75.2Unverified
10Inan et al. (2016) - Variational RHNTest perplexity66Unverified
#ModelMetricClaimedVerifiedStatus
1LSTM (7 layers)Bit per Character (BPC)1.67Unverified
2HypernetworksBit per Character (BPC)1.34Unverified
3SHA-LSTM (4 layers, h=1024, no attention head)Bit per Character (BPC)1.33Unverified
4LN HM-LSTMBit per Character (BPC)1.32Unverified
5ByteNetBit per Character (BPC)1.31Unverified
6Recurrent Highway NetworksBit per Character (BPC)1.27Unverified
7Large FS-LSTM-4Bit per Character (BPC)1.25Unverified
8Large mLSTMBit per Character (BPC)1.24Unverified
9AWD-LSTM (3 layers)Bit per Character (BPC)1.23Unverified
10Cluster-Former (#C=512)Bit per Character (BPC)1.22Unverified
#ModelMetricClaimedVerifiedStatus
1Smaller Transformer 126M (pre-trained)Test perplexity33Unverified
2OPT 125MTest perplexity32.26Unverified
3Larger Transformer 771M (pre-trained)Test perplexity28.1Unverified
4OPT 1.3BTest perplexity19.55Unverified
5GPT-Neo 125MTest perplexity17.83Unverified
6OPT 2.7BTest perplexity17.81Unverified
7Smaller Transformer 126M (fine-tuned)Test perplexity12Unverified
8GPT-Neo 1.3BTest perplexity11.46Unverified
9Transformer 125MTest perplexity10.7Unverified
10GPT-Neo 2.7BTest perplexity10.44Unverified