SOTAVerified

Language Modelling

A language model is a model of natural language. Language models are useful for a variety of tasks, including speech recognition, machine translation, natural language generation (generating more human-like text), optical character recognition, route optimization, handwriting recognition, grammar induction, and information retrieval.

Large language models (LLMs), currently their most advanced form, are predominantly based on transformers trained on larger datasets (frequently using words scraped from the public internet). They have superseded recurrent neural network-based models, which had previously superseded the purely statistical models, such as word n-gram language model.

Source: Wikipedia

Papers

Showing 1000110050 of 17610 papers

TitleStatusHype
A Low-Resource Approach to the Grammatical Error Correction of UkrainianCode0
Simulating H.P. Lovecraft horror literature with the ChatGPT large language model0
Mask The Bias: Improving Domain-Adaptive Generalization of CTC-based ASR with Internal Language Model Estimation0
T-SciQ: Teaching Multimodal Chain-of-Thought Reasoning via Mixed Large Language Model Signals for Science Question AnsweringCode1
Exploring Softly Masked Language Modelling for Controllable Symbolic Music Generation0
Towards antigenic peptide discovery with better MHC-I binding prediction and improved benchmark methodology0
LMEye: An Interactive Perception Network for Large Language ModelsCode1
Retrieval Augmented Chest X-Ray Report Generation using OpenAI GPT models0
Now It Sounds Like You: Learning Personalized Vocabulary On Device0
MindGames: Targeting Theory of Mind in Large Language Models with Dynamic Epistemic Modal LogicCode1
Principle-Driven Self-Alignment of Language Models from Scratch with Minimal Human SupervisionCode3
Leveraging BERT Language Model for Arabic Long Document Classification0
Hybrid Transducer and Attention based Encoder-Decoder Modeling for Speech-to-Text Tasks0
Cognitive Reframing of Negative Thoughts through Human-Language Model InteractionCode1
FormNetV2: Multimodal Graph Contrastive Learning for Form Document Information Extraction0
Improving Code Example Recommendations on Informal Documentation Using BERT and Query-Aware LSH: A Comparative StudyCode0
DN at SemEval-2023 Task 12: Low-Resource Language Text Classification via Multilingual Pretrained Language Model Fine-tuning0
Gpt-4: A Review on Advancements and Opportunities in Natural Language Processing0
Personalized Abstractive Summarization by Tri-agent Generation PipelineCode0
Masked Structural Growth for 2x Faster Language Model Pre-trainingCode1
Surveying Generative AI's Economic Expectations0
Interpretable Sentence Representation with Variational Autoencoders and Attention0
Sentence Embedding Leaks More Information than You Expect: Generative Embedding Inversion Attack to Recover the Whole SentenceCode1
On the Expressivity Role of LayerNorm in Transformers' AttentionCode1
Towards Imperceptible Document Manipulations against Neural Ranking Models0
Using Language Models on Low-end Hardware0
Defending against Insertion-based Textual Backdoor Attacks via AttributionCode0
Entity Tracking in Language ModelsCode1
ChatGraph: Interpretable Text Classification by Converting ChatGPT Knowledge to GraphsCode0
WangLab at MEDIQA-Chat 2023: Clinical Note Generation from Doctor-Patient Conversations using Large Language Models0
CodeGen2: Lessons for Training LLMs on Programming and Natural LanguagesCode5
Zero-Shot Listwise Document Reranking with a Large Language Model0
KEPLET: Knowledge-Enhanced Pretrained Language Model with Topic Entity Awareness0
The Benefits of Bad Advice: Autocontrastive Decoding across Model LayersCode1
Huatuo-26M, a Large-scale Chinese Medical QA DatasetCode2
How to Unleash the Power of Large Language Models for Few-shot Relation Extraction?Code0
FreeLM: Fine-Tuning-Free Language Model0
Self-Evaluation Guided Beam Search for Reasoning0
Retrieving Comparative Arguments using Ensemble Methods and Neural Information Retrieval0
CLIP-S^4: Language-Guided Self-Supervised Semantic Segmentation0
An Iterative Algorithm for Rescaled Hyperbolic Functions Regression0
How does GPT-2 compute greater-than?: Interpreting mathematical abilities in a pre-trained language modelCode1
Working Memory Capacity of ChatGPT: An Empirical StudyCode1
TALLRec: An Effective and Efficient Tuning Framework to Align Large Language Model with RecommendationCode2
Synthetic Cross-language Information Retrieval Training Data0
A Review of ChatGPT Applications in Education, Marketing, Software Engineering, and Healthcare: Benefits, Drawbacks, and Research Directions0
NLNDE at SemEval-2023 Task 12: Adaptive Pretraining and Source Language Selection for Low-Resource Multilingual Sentiment Analysis0
Towards autonomous system: flexible modular production system enhanced with large language model agentsCode1
Search-in-the-Chain: Interactively Enhancing Large Language Models with Search for Knowledge-intensive TasksCode1
Outline, Then Details: Syntactically Guided Coarse-To-Fine Code GenerationCode1
Show:102550
← PrevPage 201 of 353Next →

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1Decay RNNValidation perplexity76.67Unverified
2GRUValidation perplexity53.78Unverified
3LSTMValidation perplexity52.73Unverified
4LSTMTest perplexity48.7Unverified
5Temporal CNNTest perplexity45.2Unverified
6TCNTest perplexity45.19Unverified
7GCNN-8Test perplexity44.9Unverified
8Neural cache model (size = 100)Test perplexity44.8Unverified
9Neural cache model (size = 2,000)Test perplexity40.8Unverified
10GPT-2 SmallTest perplexity37.5Unverified
#ModelMetricClaimedVerifiedStatus
1TCNTest perplexity108.47Unverified
2Seq-U-NetTest perplexity107.95Unverified
3GRU (Bai et al., 2018)Test perplexity92.48Unverified
4R-TransformerTest perplexity84.38Unverified
5Zaremba et al. (2014) - LSTM (medium)Test perplexity82.7Unverified
6Gal & Ghahramani (2016) - Variational LSTM (medium)Test perplexity79.7Unverified
7LSTM (Bai et al., 2018)Test perplexity78.93Unverified
8Zaremba et al. (2014) - LSTM (large)Test perplexity78.4Unverified
9Gal & Ghahramani (2016) - Variational LSTM (large)Test perplexity75.2Unverified
10Inan et al. (2016) - Variational RHNTest perplexity66Unverified
#ModelMetricClaimedVerifiedStatus
1LSTM (7 layers)Bit per Character (BPC)1.67Unverified
2HypernetworksBit per Character (BPC)1.34Unverified
3SHA-LSTM (4 layers, h=1024, no attention head)Bit per Character (BPC)1.33Unverified
4LN HM-LSTMBit per Character (BPC)1.32Unverified
5ByteNetBit per Character (BPC)1.31Unverified
6Recurrent Highway NetworksBit per Character (BPC)1.27Unverified
7Large FS-LSTM-4Bit per Character (BPC)1.25Unverified
8Large mLSTMBit per Character (BPC)1.24Unverified
9AWD-LSTM (3 layers)Bit per Character (BPC)1.23Unverified
10Cluster-Former (#C=512)Bit per Character (BPC)1.22Unverified
#ModelMetricClaimedVerifiedStatus
1Smaller Transformer 126M (pre-trained)Test perplexity33Unverified
2OPT 125MTest perplexity32.26Unverified
3Larger Transformer 771M (pre-trained)Test perplexity28.1Unverified
4OPT 1.3BTest perplexity19.55Unverified
5GPT-Neo 125MTest perplexity17.83Unverified
6OPT 2.7BTest perplexity17.81Unverified
7Smaller Transformer 126M (fine-tuned)Test perplexity12Unverified
8GPT-Neo 1.3BTest perplexity11.46Unverified
9Transformer 125MTest perplexity10.7Unverified
10GPT-Neo 2.7BTest perplexity10.44Unverified