SOTAVerified

Language Modelling

A language model is a model of natural language. Language models are useful for a variety of tasks, including speech recognition, machine translation, natural language generation (generating more human-like text), optical character recognition, route optimization, handwriting recognition, grammar induction, and information retrieval.

Large language models (LLMs), currently their most advanced form, are predominantly based on transformers trained on larger datasets (frequently using words scraped from the public internet). They have superseded recurrent neural network-based models, which had previously superseded the purely statistical models, such as word n-gram language model.

Source: Wikipedia

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 10851–10900 of 17610 papers

TitleStatusHype
Bridging the Gap: Deciphering Tabular Data Using Large Language Model—0
Diversity Measures: Domain-Independent Proxies for Failure in Language Model QueriesCode0
Leveraging Language Model Capabilities for Sound Event Detection—0
Is There Any Social Principle for LLM-Based Agents?—0
Towards an On-device Agent for Text Rewriting—0
Tryage: Real-time, intelligent Routing of User Prompts to Large Language Models—0
Pre-training with Aspect-Content Text Mutual Prediction for Multi-Aspect Dense RetrievalCode0
Random Word Data Augmentation with CLIP for Zero-Shot Anomaly Detection—0
Can Language Models Learn to Listen?—0
Enhancing Recommender Systems with Large Language Model Reasoning Graphs—0
COCA: Classifier-Oriented Calibration via Textual Prototype for Source-Free Universal Domain AdaptationCode0
Indonesian Automatic Speech Recognition with XLSR-53—0
Imaginations of WALL-E : Reconstructing Experiences with an Imagination-Inspired Module for Advanced AI Systems—0
cantnlp@LT-EDI-2023: Homophobia/Transphobia Detection in Social Media Comments using Spatio-Temporally Retrained Language Models—0
FoodGPT: A Large Language Model in Food Testing Domain with Incremental Pre-training and Knowledge Graph Prompt—0
Inductive-bias Learning: Generating Code Models with Large Language ModelCode0
DUAW: Data-free Universal Adversarial Watermark against Stable Diffusion Customization—0
PACE: Improving Prompt with Actor-Critic Editing for Large Language Model—0
OCR Language Models with Custom Vocabularies—0
Latent State Models of Training DynamicsCode0
PUMGPT: A Large Vision-Language Model for Product Understanding—0
Exploring Sampling Techniques for Generating Melodies with a Transformer Language Model—0
BioMedGPT: Open Multimodal Generative Pre-trained Transformer for BioMedicine—0
Differentiable Retrieval Augmentation via Generative Language Modeling for E-commerce Query Intent Classification—0
Conversational Ontology Alignment with ChatGPT—0
Contrasting Linguistic Patterns in Human and LLM-Generated News Text—0
BERT4CTR: An Efficient Framework to Combine Pre-trained Language Model with Non-textual Features for CTR Prediction—0
Fine-grained Text and Image Guided Point Cloud Completion with CLIP Model—0
LLM-FuncMapper: Function Identification for Interpreting Complex Clauses in Building Codes via LLM—0
Towards Automatically Addressing Self-Admitted Technical Debt: How Far Are We?Code0
Reinforced Self-Training (ReST) for Language Modeling—0
Pre-training with Large Language Model-based Document Expansion for Dense Passage Retrieval—0
Visually-Aware Context Modeling for News Image CaptioningCode0
FootGPT : A Large Language Model Development Experiment on a Minimal Setting—0
BIOptimus: Pre-training an Optimal Biomedical Language Model with Curriculum Learning for Named Entity RecognitionCode0
ChatLogo: A Large Language Model-Driven Hybrid Natural-Programming Language Interface for Agent-based Modeling and Programming—0
Domain Adaptation for Code Model-based Unit Test Case Generation—0
Emerging Frontiers: Exploring the Impact of Generative AI Platforms on University Quantitative Finance Examinations—0
SPM: Structured Pretraining and Matching Architectures for Relevance Modeling in Meituan Search—0
Through the Lens of Core Competency: Survey on Evaluation of Large Language Models—0
LLM-Mini-CEX: Automatic Evaluation of Large Language Model for Diagnostic Conversation—0
Ternary Singular Value Decomposition as a Better Parameterized Form in Linear MappingCode0
Correct and Optimal: the Regular Expression Inference Challenge—0
RAVEN: In-Context Learning with Retrieval-Augmented Encoder-Decoder Language ModelsCode0
SpeechX: Neural Codec Language Model as a Versatile Speech Transformer—0
Neural Authorship Attribution: Stylometric Analysis on Large Language ModelsCode0
Position: Key Claims in LLM Research Have a Long Tail of Footnotes—0
Text Injection for Capitalization and Turn-Taking Prediction in Speech Models—0
AudioFormer: Audio Transformer learns audio feature representations from discrete acoustic codesCode0
ChatGPT in Drug Discovery: A Case Study on Anti-Cocaine Addiction Drug Development with Chatbots—0
Show:102550
← PrevPage 218 of 353Next →
All datasetsWikiText-103Penn Treebank (Word Level)enwik8The PileWikiText-2LAMBADAOne Billion WordText8Penn Treebank (Character Level)Hutter PrizeOpenWebTextSALMon

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1Decay RNNValidation perplexity76.67—Unverified
2GRUValidation perplexity53.78—Unverified
3LSTMValidation perplexity52.73—Unverified
4LSTMTest perplexity48.7—Unverified
5Temporal CNNTest perplexity45.2—Unverified
6TCNTest perplexity45.19—Unverified
7GCNN-8Test perplexity44.9—Unverified
8Neural cache model (size = 100)Test perplexity44.8—Unverified
9Neural cache model (size = 2,000)Test perplexity40.8—Unverified
10GPT-2 SmallTest perplexity37.5—Unverified
#ModelMetricClaimedVerifiedStatus
1TCNTest perplexity108.47—Unverified
2Seq-U-NetTest perplexity107.95—Unverified
3GRU (Bai et al., 2018)Test perplexity92.48—Unverified
4R-TransformerTest perplexity84.38—Unverified
5Zaremba et al. (2014) - LSTM (medium)Test perplexity82.7—Unverified
6Gal & Ghahramani (2016) - Variational LSTM (medium)Test perplexity79.7—Unverified
7LSTM (Bai et al., 2018)Test perplexity78.93—Unverified
8Zaremba et al. (2014) - LSTM (large)Test perplexity78.4—Unverified
9Gal & Ghahramani (2016) - Variational LSTM (large)Test perplexity75.2—Unverified
10Inan et al. (2016) - Variational RHNTest perplexity66—Unverified
#ModelMetricClaimedVerifiedStatus
1LSTM (7 layers)Bit per Character (BPC)1.67—Unverified
2HypernetworksBit per Character (BPC)1.34—Unverified
3SHA-LSTM (4 layers, h=1024, no attention head)Bit per Character (BPC)1.33—Unverified
4LN HM-LSTMBit per Character (BPC)1.32—Unverified
5ByteNetBit per Character (BPC)1.31—Unverified
6Recurrent Highway NetworksBit per Character (BPC)1.27—Unverified
7Large FS-LSTM-4Bit per Character (BPC)1.25—Unverified
8Large mLSTMBit per Character (BPC)1.24—Unverified
9AWD-LSTM (3 layers)Bit per Character (BPC)1.23—Unverified
10Cluster-Former (#C=512)Bit per Character (BPC)1.22—Unverified
#ModelMetricClaimedVerifiedStatus
1Smaller Transformer 126M (pre-trained)Test perplexity33—Unverified
2OPT 125MTest perplexity32.26—Unverified
3Larger Transformer 771M (pre-trained)Test perplexity28.1—Unverified
4OPT 1.3BTest perplexity19.55—Unverified
5GPT-Neo 125MTest perplexity17.83—Unverified
6OPT 2.7BTest perplexity17.81—Unverified
7Smaller Transformer 126M (fine-tuned)Test perplexity12—Unverified
8GPT-Neo 1.3BTest perplexity11.46—Unverified
9Transformer 125MTest perplexity10.7—Unverified
10GPT-Neo 2.7BTest perplexity10.44—Unverified