SOTAVerified

Language Modelling

A language model is a model of natural language. Language models are useful for a variety of tasks, including speech recognition, machine translation, natural language generation (generating more human-like text), optical character recognition, route optimization, handwriting recognition, grammar induction, and information retrieval.

Large language models (LLMs), currently their most advanced form, are predominantly based on transformers trained on larger datasets (frequently using words scraped from the public internet). They have superseded recurrent neural network-based models, which had previously superseded the purely statistical models, such as word n-gram language model.

Source: Wikipedia

Papers

Showing 45514600 of 17610 papers

TitleStatusHype
DALLMi: Domain Adaption for LLM-based Multi-label ClassifierCode0
Fine-tuning the ESM2 protein language model to understand the functional impact of missense variantsCode0
Fingerprinting web servers through Transformer-encoded HTTP response headersCode0
Fingerspelling PoseNet: Enhancing Fingerspelling Translation with Pose-Based Transformer ModelsCode0
Finnish resources for evaluating language model semanticsCode0
FinTree: Financial Dataset Pretrain Transformer Encoder for Relation ExtractionCode0
FinTruthQA: A Benchmark Dataset for Evaluating the Quality of Financial Information DisclosureCode0
Fire Burns, Sword Cuts: Commonsense Inductive Bias for Exploration in Text-based GamesCode0
FIRE: Food Image to REcipe generationCode0
First Align, then Predict: Understanding the Cross-Lingual Ability of Multilingual BERTCode0
First Automatic Fongbe Continuous Speech Recognition System: Development of Acoustic Models and Language ModelsCode0
First Heuristic Then Rational: Dynamic Use of Heuristics in Language Model ReasoningCode0
A BERT-Based Transfer Learning Approach for Hate Speech Detection in Online Social MediaCode0
First-Pass Large Vocabulary Continuous Speech Recognition using Bi-Directional Recurrent DNNsCode0
FIRST: Teach A Reliable Large Language Model Through Efficient Trustworthy DistillationCode0
Fisher Mask Nodes for Language Model MergingCode0
FiSSA at SemEval-2020 Task 9: Fine-tuned For FeelingsCode0
DarijaBanking: A New Resource for Overcoming Language Barriers in Banking Intent Detection for Moroccan Arabic SpeakersCode0
Dispersed Exponential Family Mixture VAEs for Interpretable Text GenerationCode0
A Neural Model of Adaptation in ReadingCode0
A Comparison of Techniques for Language Model Integration in Encoder-Decoder Speech RecognitionCode0
Data Advisor: Dynamic Data Curation for Safety Alignment of Large Language ModelsCode0
FLAT-LLM: Fine-grained Low-rank Activation Space Transformation for Large Language Model CompressionCode0
Data Augmentation for Biomedical Factoid Question AnsweringCode0
FlauBERT : des mod\`eles de langue contextualis\'es pr\'e-entra\^ \'es pour le fran (FlauBERT : Unsupervised Language Model Pre-training for French)Code0
FlauBERT: Unsupervised Language Model Pre-training for FrenchCode0
CAiRE: An Empathetic Neural ChatbotCode0
ATM: Adversarial Tuning Multi-agent System Makes a Robust Retrieval-Augmented GeneratorCode0
Data augmentation using prosody and false starts to recognize non-native children's speechCode0
Calibrating LLM-Based EvaluatorCode0
A Tool for Facilitating OCR Postediting in Historical DocumentsCode0
DataChat: Prototyping a Conversational Agent for Dataset Search and VisualizationCode0
DATA: Differentiable ArchiTecture ApproximationCode0
A Tool for Generating Exceptional Behavior Tests With Large Language ModelsCode0
FNetAR: Mixing Tokens with Autoregressive Fourier TransformsCode0
A Toolkit for Efficient Learning of Lexical Units for Speech RecognitionCode0
ALOHA: Artificial Learning of Human Attributes for Dialogue AgentsCode0
A Training Data Recipe to Accelerate A* Search with Language ModelsCode0
FonMTL: Towards Multitask Learning for the Fon LanguageCode0
DataGpt-SQL-7B: An Open-Source Language Model for Text-to-SQLCode0
Data-Informed Global Sparseness in Attention Mechanisms for Deep Neural NetworksCode0
CamemBERT: a Tasty French Language ModelCode0
Data Noising as Smoothing in Neural Network Language ModelsCode0
An Evaluation of Explanation Methods for Black-Box Detectors of Machine-Generated TextCode0
Data Selection for Fine-tuning Large Language Models Using Transferred Shapley ValuesCode0
Dataset and Lessons Learned from the 2024 SaTML LLM Capture-the-Flag CompetitionCode0
Forget NLI, Use a Dictionary: Zero-Shot Topic Classification for Low-Resource Languages with Application to LuxembourgishCode0
Forging Multiple Training Objectives for Pre-trained Language Models via Meta-LearningCode0
Faithful and Efficient Explanations for Neural Networks via Neural Tangent Kernel Surrogate ModelsCode0
Data Similarity is Not Enough to Explain Language Model PerformanceCode0
Show:102550
← PrevPage 92 of 353Next →

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1Decay RNNValidation perplexity76.67Unverified
2GRUValidation perplexity53.78Unverified
3LSTMValidation perplexity52.73Unverified
4LSTMTest perplexity48.7Unverified
5Temporal CNNTest perplexity45.2Unverified
6TCNTest perplexity45.19Unverified
7GCNN-8Test perplexity44.9Unverified
8Neural cache model (size = 100)Test perplexity44.8Unverified
9Neural cache model (size = 2,000)Test perplexity40.8Unverified
10GPT-2 SmallTest perplexity37.5Unverified
#ModelMetricClaimedVerifiedStatus
1TCNTest perplexity108.47Unverified
2Seq-U-NetTest perplexity107.95Unverified
3GRU (Bai et al., 2018)Test perplexity92.48Unverified
4R-TransformerTest perplexity84.38Unverified
5Zaremba et al. (2014) - LSTM (medium)Test perplexity82.7Unverified
6Gal & Ghahramani (2016) - Variational LSTM (medium)Test perplexity79.7Unverified
7LSTM (Bai et al., 2018)Test perplexity78.93Unverified
8Zaremba et al. (2014) - LSTM (large)Test perplexity78.4Unverified
9Gal & Ghahramani (2016) - Variational LSTM (large)Test perplexity75.2Unverified
10Inan et al. (2016) - Variational RHNTest perplexity66Unverified
#ModelMetricClaimedVerifiedStatus
1LSTM (7 layers)Bit per Character (BPC)1.67Unverified
2HypernetworksBit per Character (BPC)1.34Unverified
3SHA-LSTM (4 layers, h=1024, no attention head)Bit per Character (BPC)1.33Unverified
4LN HM-LSTMBit per Character (BPC)1.32Unverified
5ByteNetBit per Character (BPC)1.31Unverified
6Recurrent Highway NetworksBit per Character (BPC)1.27Unverified
7Large FS-LSTM-4Bit per Character (BPC)1.25Unverified
8Large mLSTMBit per Character (BPC)1.24Unverified
9AWD-LSTM (3 layers)Bit per Character (BPC)1.23Unverified
10Cluster-Former (#C=512)Bit per Character (BPC)1.22Unverified
#ModelMetricClaimedVerifiedStatus
1Smaller Transformer 126M (pre-trained)Test perplexity33Unverified
2OPT 125MTest perplexity32.26Unverified
3Larger Transformer 771M (pre-trained)Test perplexity28.1Unverified
4OPT 1.3BTest perplexity19.55Unverified
5GPT-Neo 125MTest perplexity17.83Unverified
6OPT 2.7BTest perplexity17.81Unverified
7Smaller Transformer 126M (fine-tuned)Test perplexity12Unverified
8GPT-Neo 1.3BTest perplexity11.46Unverified
9Transformer 125MTest perplexity10.7Unverified
10GPT-Neo 2.7BTest perplexity10.44Unverified