SOTAVerified

Language Modelling

A language model is a model of natural language. Language models are useful for a variety of tasks, including speech recognition, machine translation, natural language generation (generating more human-like text), optical character recognition, route optimization, handwriting recognition, grammar induction, and information retrieval.

Large language models (LLMs), currently their most advanced form, are predominantly based on transformers trained on larger datasets (frequently using words scraped from the public internet). They have superseded recurrent neural network-based models, which had previously superseded the purely statistical models, such as word n-gram language model.

Source: Wikipedia

Papers

Showing 42514300 of 17610 papers

TitleStatusHype
Analyzing Phonetic and Graphemic Representations in End-to-End Automatic Speech RecognitionCode0
Continual adaptation for efficient machine communicationCode0
Bilingual Lexicon Induction through Unsupervised Machine TranslationCode0
Continual and Multi-Task Architecture SearchCode0
Continual Learning of Recurrent Neural Networks by Locally Aligning Distributed RepresentationsCode0
Adapting Learned Sparse Retrieval for Long DocumentsCode0
Evaluating the Validity of Word-level Adversarial Attacks with Large Language ModelsCode0
Evaluating Transformer Language Models on Arithmetic Operations Using Number DecompositionCode0
AgentStealth: Reinforcing Large Language Model for Anonymizing User-generated TextCode0
Continuous Language Model Interpolation for Dynamic and Controllable Text GenerationCode0
Analyzing Speaker Information in Self-Supervised Models to Improve Zero-Resource Speech ProcessingCode0
Adapting Multilingual LLMs to Low-Resource Languages with Knowledge Graphs via AdaptersCode0
Assay2Mol: large language model-based drug design using BioAssay contextCode0
Assertion Detection Large Language Model In-context Learning LoRA Fine-tuningCode0
Evaluation of Language Models in the Medical Context Under Resource-Constrained SettingsCode0
Continuous Speech Tokenizer in Text To SpeechCode0
Evaluation of sentence embeddings in downstream and linguistic probing tasksCode0
Evaluation Phonemic Transcription of Low-Resource Tonal Languages for Language DocumentationCode0
Event-based clinical findings extraction from radiology reports with pre-trained language modelCode0
Contrastive and Consistency Learning for Neural Noisy-Channel Model in Spoken Language UnderstandingCode0
Event Detection as Question Answering with Entity InformationCode0
EventGround: Narrative Reasoning by Grounding to Eventuality-centric Knowledge GraphsCode0
Event Knowledge Incorporation with Posterior Regularization for Event-Centric Question AnsweringCode0
A Geometric Notion of Causal ProbingCode0
Agglomerative AttentionCode0
Every Answer Matters: Evaluating Commonsense with Probabilistic MeasuresCode0
Evidence-backed Fact Checking using RAG and Few-Shot In-Context Learning with LLMsCode0
Contrastive Language Prompting to Ease False Positives in Medical Anomaly DetectionCode0
Evidence Is All You Need: Ordering Imaging Studies via Language Model Alignment with the ACR Appropriateness CriteriaCode0
Assessing Generative Language Models in Classification Tasks: Performance and Self-Evaluation Capabilities in the Environmental and Climate Change DomainCode0
Contrastive learning of T cell receptor representationsCode0
Evolutionary Stochastic Gradient Descent for Optimization of Deep Neural NetworksCode0
Evolutionary Verbalizer Search for Prompt-based Few Shot Text ClassificationCode0
Evolution of ESG-focused DLT Research: An NLP Analysis of the LiteratureCode0
Evolving Assembly Code in an Adversarial EnvironmentCode0
Adapting Multi-modal Large Language Model to Concept Drift From Pre-training OnwardsCode0
Evolving Subnetwork Training for Large Language ModelsCode0
Controllable Citation Sentence Generation with Language ModelsCode0
ACL Ready: RAG Based Assistant for the ACL ChecklistCode0
Examining Language Modeling Assumptions Using an Annotated Literary Dialect CorpusCode0
Controllable Neural Story Plot Generation via Reward ShapingCode0
Assessing the Ability of LSTMs to Learn Syntax-Sensitive DependenciesCode0
exBERT: A Visual Analysis Tool to Explore Learned Representations in Transformers ModelsCode0
Biological structure and function emerge from scaling unsupervised learning to 250 million protein sequencesCode0
EXMODD: An EXplanatory Multimodal Open-Domain Dialogue datasetCode0
Exp4Fuse: A Rank Fusion Framework for Enhanced Sparse Retrieval using Large Language Model-based Query ExpansionCode0
Agile-Quant: Activation-Guided Quantization for Faster Inference of LLMs on the EdgeCode0
An Analysis of Neural Language Modeling at Multiple ScalesCode0
A Glitch in the Matrix? Locating and Detecting Language Model Grounding with FakepediaCode0
Expanding the Vocabulary of BERT for Knowledge Base ConstructionCode0
Show:102550
← PrevPage 86 of 353Next →

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1Decay RNNValidation perplexity76.67Unverified
2GRUValidation perplexity53.78Unverified
3LSTMValidation perplexity52.73Unverified
4LSTMTest perplexity48.7Unverified
5Temporal CNNTest perplexity45.2Unverified
6TCNTest perplexity45.19Unverified
7GCNN-8Test perplexity44.9Unverified
8Neural cache model (size = 100)Test perplexity44.8Unverified
9Neural cache model (size = 2,000)Test perplexity40.8Unverified
10GPT-2 SmallTest perplexity37.5Unverified
#ModelMetricClaimedVerifiedStatus
1TCNTest perplexity108.47Unverified
2Seq-U-NetTest perplexity107.95Unverified
3GRU (Bai et al., 2018)Test perplexity92.48Unverified
4R-TransformerTest perplexity84.38Unverified
5Zaremba et al. (2014) - LSTM (medium)Test perplexity82.7Unverified
6Gal & Ghahramani (2016) - Variational LSTM (medium)Test perplexity79.7Unverified
7LSTM (Bai et al., 2018)Test perplexity78.93Unverified
8Zaremba et al. (2014) - LSTM (large)Test perplexity78.4Unverified
9Gal & Ghahramani (2016) - Variational LSTM (large)Test perplexity75.2Unverified
10Inan et al. (2016) - Variational RHNTest perplexity66Unverified
#ModelMetricClaimedVerifiedStatus
1LSTM (7 layers)Bit per Character (BPC)1.67Unverified
2HypernetworksBit per Character (BPC)1.34Unverified
3SHA-LSTM (4 layers, h=1024, no attention head)Bit per Character (BPC)1.33Unverified
4LN HM-LSTMBit per Character (BPC)1.32Unverified
5ByteNetBit per Character (BPC)1.31Unverified
6Recurrent Highway NetworksBit per Character (BPC)1.27Unverified
7Large FS-LSTM-4Bit per Character (BPC)1.25Unverified
8Large mLSTMBit per Character (BPC)1.24Unverified
9AWD-LSTM (3 layers)Bit per Character (BPC)1.23Unverified
10Cluster-Former (#C=512)Bit per Character (BPC)1.22Unverified
#ModelMetricClaimedVerifiedStatus
1Smaller Transformer 126M (pre-trained)Test perplexity33Unverified
2OPT 125MTest perplexity32.26Unverified
3Larger Transformer 771M (pre-trained)Test perplexity28.1Unverified
4OPT 1.3BTest perplexity19.55Unverified
5GPT-Neo 125MTest perplexity17.83Unverified
6OPT 2.7BTest perplexity17.81Unverified
7Smaller Transformer 126M (fine-tuned)Test perplexity12Unverified
8GPT-Neo 1.3BTest perplexity11.46Unverified
9Transformer 125MTest perplexity10.7Unverified
10GPT-Neo 2.7BTest perplexity10.44Unverified