SOTAVerified

Language Modelling

A language model is a model of natural language. Language models are useful for a variety of tasks, including speech recognition, machine translation, natural language generation (generating more human-like text), optical character recognition, route optimization, handwriting recognition, grammar induction, and information retrieval.

Large language models (LLMs), currently their most advanced form, are predominantly based on transformers trained on larger datasets (frequently using words scraped from the public internet). They have superseded recurrent neural network-based models, which had previously superseded the purely statistical models, such as word n-gram language model.

Source: Wikipedia

Papers

Showing 92019250 of 17610 papers

TitleStatusHype
Large Language Model-based System to Provide Immediate Feedback to Students in Flipped Classroom Preparation Learning0
OUTFOX: LLM-Generated Essay Detection Through In-Context Learning with Adversarially Generated ExamplesCode1
Generator-Retriever-Generator Approach for Open-Domain Question AnsweringCode1
GIST: Generating Image-Specific Text for Fine-grained Object ClassificationCode1
A LLM Assisted Exploitation of AI-Guardian0
Transsion TSUP's speech recognition system for ASRU 2023 MADASR Challenge0
UP-DP: Unsupervised Prompt Learning for Data Pre-Selection with Vision-Language Models0
SC VALL-E: Style-Controllable Zero-Shot Text to Speech SynthesizerCode1
Topics, Authors, and Institutions in Large Language Model Research: Trends from 17K arXiv PapersCode1
Generative Language Models on Nucleotide Sequences of Human GenesCode0
Gender-tuning: Empowering Fine-tuning for Debiasing Pre-trained Language Models0
Identifying Interpretable Subspaces in Image RepresentationsCode1
Dynamic Large Language Models on Blockchains0
FLASK: Fine-grained Language Model Evaluation based on Alignment Skill SetsCode2
SciBench: Evaluating College-Level Scientific Problem-Solving Abilities of Large Language ModelsCode1
IvyGPT: InteractiVe Chinese pathwaY language model in medical domainCode1
Invalid Logic, Equivalent Gains: The Bizarreness of Reasoning in Language Model Prompting0
MediaGPT : A Large Language Model For Chinese Media0
PASTA: Pretrained Action-State Transformer Agents0
ProtiGeno: a prokaryotic short gene finder using protein language modelsCode0
Improving the Reusability of Pre-trained Language Models in Real-world Applications0
Can Instruction Fine-Tuned Language Models Identify Social Bias through Prompting?0
Efficient Guided Generation for Large Language ModelsCode6
Generative Prompt Model for Weakly Supervised Object LocalizationCode1
DialogStudio: Towards Richest and Most Diverse Unified Dataset Collection for Conversational AICode2
DAPrompt: Deterministic Assumption Prompt Learning for Event Causality Identification0
Exploring Transformer ExtrapolationCode0
RaTE: a Reproducible automatic Taxonomy Evaluation by Filling the GapCode0
Towards Federated Foundation Models: Scalable Dataset Pipelines for Group-Structured LearningCode1
MLP Fusion: Towards Efficient Fine-tuning of Dense and Mixture-of-Experts Language ModelsCode1
Promoting Exploration in Memory-Augmented Adam using Critical MomentaCode0
Multimodal LLMs for health grounded in individual-specific data0
Linearized Relative Positional EncodingCode0
Modular Multimodal Machine Learning for Extraction of Theorems and Proofs in Long Scientific Documents (Extended Version)Code1
SLMGAN: Exploiting Speech Language Model Representations for Unsupervised Zero-Shot Voice Conversion in GANs0
PromptCrafter: Crafting Text-to-Image Prompt through Mixed-Initiative Dialogue with LLM0
ChatSpot: Bootstrapping Multimodal LLMs via Precise Referring Instruction Tuning0
Integration of Large Language Models and Federated Learning0
How is ChatGPT's behavior changing over time?Code4
Domain Knowledge Distillation from Large Language Model: An Empirical Study in the Autonomous Driving Domain0
Abductive Reasoning with the GPT-4 Language Model: Case studies from criminal investigation, medical practice, scientific research0
Creating Image Datasets in Agricultural Environments using DALL.E: Generative AI-Powered Large Language Model0
Adapting Large Language Model with Speech for Fully Formatted End-to-End Speech RecognitionCode8
FlashAttention-2: Faster Attention with Better Parallelism and Work PartitioningCode6
Zero-Shot Image Harmonization with Generative Model PriorCode1
Retentive Network: A Successor to Transformer for Large Language ModelsCode3
M-FLAG: Medical Vision-Language Pre-training with Frozen Language Models and Latent Space Geometry OptimizationCode1
Gender mobility in the labor market with skills-based matching models0
Using an LLM to Help With Code Understanding0
Cross-Lingual NER for Financial Transaction Data in Low-Resource Languages0
Show:102550
← PrevPage 185 of 353Next →

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1Decay RNNValidation perplexity76.67Unverified
2GRUValidation perplexity53.78Unverified
3LSTMValidation perplexity52.73Unverified
4LSTMTest perplexity48.7Unverified
5Temporal CNNTest perplexity45.2Unverified
6TCNTest perplexity45.19Unverified
7GCNN-8Test perplexity44.9Unverified
8Neural cache model (size = 100)Test perplexity44.8Unverified
9Neural cache model (size = 2,000)Test perplexity40.8Unverified
10GPT-2 SmallTest perplexity37.5Unverified
#ModelMetricClaimedVerifiedStatus
1TCNTest perplexity108.47Unverified
2Seq-U-NetTest perplexity107.95Unverified
3GRU (Bai et al., 2018)Test perplexity92.48Unverified
4R-TransformerTest perplexity84.38Unverified
5Zaremba et al. (2014) - LSTM (medium)Test perplexity82.7Unverified
6Gal & Ghahramani (2016) - Variational LSTM (medium)Test perplexity79.7Unverified
7LSTM (Bai et al., 2018)Test perplexity78.93Unverified
8Zaremba et al. (2014) - LSTM (large)Test perplexity78.4Unverified
9Gal & Ghahramani (2016) - Variational LSTM (large)Test perplexity75.2Unverified
10Inan et al. (2016) - Variational RHNTest perplexity66Unverified
#ModelMetricClaimedVerifiedStatus
1LSTM (7 layers)Bit per Character (BPC)1.67Unverified
2HypernetworksBit per Character (BPC)1.34Unverified
3SHA-LSTM (4 layers, h=1024, no attention head)Bit per Character (BPC)1.33Unverified
4LN HM-LSTMBit per Character (BPC)1.32Unverified
5ByteNetBit per Character (BPC)1.31Unverified
6Recurrent Highway NetworksBit per Character (BPC)1.27Unverified
7Large FS-LSTM-4Bit per Character (BPC)1.25Unverified
8Large mLSTMBit per Character (BPC)1.24Unverified
9AWD-LSTM (3 layers)Bit per Character (BPC)1.23Unverified
10Cluster-Former (#C=512)Bit per Character (BPC)1.22Unverified
#ModelMetricClaimedVerifiedStatus
1Smaller Transformer 126M (pre-trained)Test perplexity33Unverified
2OPT 125MTest perplexity32.26Unverified
3Larger Transformer 771M (pre-trained)Test perplexity28.1Unverified
4OPT 1.3BTest perplexity19.55Unverified
5GPT-Neo 125MTest perplexity17.83Unverified
6OPT 2.7BTest perplexity17.81Unverified
7Smaller Transformer 126M (fine-tuned)Test perplexity12Unverified
8GPT-Neo 1.3BTest perplexity11.46Unverified
9Transformer 125MTest perplexity10.7Unverified
10GPT-Neo 2.7BTest perplexity10.44Unverified