SOTAVerified

Language Modelling

A language model is a model of natural language. Language models are useful for a variety of tasks, including speech recognition, machine translation, natural language generation (generating more human-like text), optical character recognition, route optimization, handwriting recognition, grammar induction, and information retrieval.

Large language models (LLMs), currently their most advanced form, are predominantly based on transformers trained on larger datasets (frequently using words scraped from the public internet). They have superseded recurrent neural network-based models, which had previously superseded the purely statistical models, such as word n-gram language model.

Source: Wikipedia

Papers

Showing 98519900 of 17610 papers

TitleStatusHype
SLaDe: A Portable Small Language Model Decompiler for Optimized Assembly0
HIINT: Historical, Intra- and Inter- personal Dynamics Modeling with Cross-person Memory Transformer0
Augmenting Autotelic Agents with Large Language Models0
Abstract Meaning Representation-Based Logic-Driven Data Augmentation for Logical ReasoningCode1
DPIC: Decoupling Prompt and Intrinsic Characteristics for LLM Generated Text Detection0
ESCOXLM-R: Multilingual Taxonomy-driven Pre-training for the Job Market DomainCode1
Glot500: Scaling Multilingual Corpora and Language Models to 500 LanguagesCode1
DisCo: Distilled Student Models Co-training for Semi-supervised Text MiningCode1
Collaborative Development of NLP modelsCode0
Patton: Language Model Pretraining on Text-Rich Networks0
Pengi: An Audio Language Model for Audio TasksCode2
Unsupervised ASR via Cross-Lingual Pseudo-Labeling0
Eye-SpatialNet: Spatial Information Extraction from Ophthalmology Notes0
Self-QA: Unsupervised Knowledge Guided Language Model AlignmentCode3
Interpretable Word Sense Representations via Definition Generation: The Case of Semantic Change AnalysisCode0
Cross-Lingual Supervision improves Large Language Models Pre-training0
AttriCLIP: A Non-Incremental Learner for Incremental Knowledge Learning0
Extending Memory for Language Modelling0
From Alignment to Entailment: A Unified Textual Entailment Framework for Entity AlignmentCode0
Clinical Camel: An Open Expert-Level Medical Language Model with Dialogue-Based Knowledge EncodingCode1
A Sequence-to-Sequence Approach for Arabic Pronoun Resolution0
Constructing Word-Context-Coupled Space Aligned with Associative Knowledge Relations for Interpretable Language ModelingCode0
Analyzing and Reducing the Performance Gap in Cross-Lingual Transfer with Fine-tuning Slow and Fast0
Empower Large Language Model to Perform Better on Industrial Domain-Specific Question AnsweringCode1
Graphologue: Exploring Large Language Model Responses with Interactive Diagrams0
Decouple knowledge from parameters for plug-and-play language modelingCode1
Shattering the Agent-Environment Interface for Fine-Tuning Inclusive Language Models0
Syllable Discovery and Cross-Lingual Generalization in a Visually Grounded, Self-Supervised Speech ModelCode1
Multimodal Web Navigation with Instruction-Finetuned Foundation Models0
Introspective Tips: Large Language Model for In-Context Decision Making0
LIMA: Less Is More for AlignmentCode4
VisionLLM: Large Language Model is also an Open-Ended Decoder for Vision-Centric TasksCode4
DrugChat: Towards Enabling ChatGPT-Like Capabilities on Drug Molecule GraphsCode0
UMDFood: Vision-language models boost food composition compilation0
Vaxformer: Antigenicity-controlled Transformer for Vaccine Design Against SARS-CoV-2Code0
Comparing Machines and Children: Using Developmental Psychology Experiments to Assess the Strengths and Weaknesses of LaMDA Responses0
MCD: A Model-Agnostic Counterfactual Search Method For Multi-modal Design Modifications0
Instruct2Act: Mapping Multi-modality Instructions to Robotic Actions with Large Language ModelCode2
Ditto: A Simple and Efficient Approach to Improve Sentence Embeddings0
How does the task complexity of masked pretraining objectives affect downstream performance?Code0
Emergent Representations of Program Semantics in Language Models Trained on ProgramsCode1
Generalized Multiple Intent Conditioned Slot Filling0
Diffusion Language Models Generation Can Be Halted Early0
Listen, Think, and UnderstandCode2
SpeechGPT: Empowering Large Language Models with Intrinsic Cross-Modal Conversational AbilitiesCode3
MolXPT: Wrapping Molecules with Text for Generative Pre-trainingCode0
Language Model Tokenizers Introduce Unfairness Between LanguagesCode0
DoReMi: Optimizing Data Mixtures Speeds Up Language Model PretrainingCode2
Tree of Thoughts: Deliberate Problem Solving with Large Language ModelsCode5
Prompt Engineering for Transformer-based Chemical Similarity Search Identifies Structurally Distinct Functional AnaloguesCode0
Show:102550
← PrevPage 198 of 353Next →

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1Decay RNNValidation perplexity76.67Unverified
2GRUValidation perplexity53.78Unverified
3LSTMValidation perplexity52.73Unverified
4LSTMTest perplexity48.7Unverified
5Temporal CNNTest perplexity45.2Unverified
6TCNTest perplexity45.19Unverified
7GCNN-8Test perplexity44.9Unverified
8Neural cache model (size = 100)Test perplexity44.8Unverified
9Neural cache model (size = 2,000)Test perplexity40.8Unverified
10GPT-2 SmallTest perplexity37.5Unverified
#ModelMetricClaimedVerifiedStatus
1TCNTest perplexity108.47Unverified
2Seq-U-NetTest perplexity107.95Unverified
3GRU (Bai et al., 2018)Test perplexity92.48Unverified
4R-TransformerTest perplexity84.38Unverified
5Zaremba et al. (2014) - LSTM (medium)Test perplexity82.7Unverified
6Gal & Ghahramani (2016) - Variational LSTM (medium)Test perplexity79.7Unverified
7LSTM (Bai et al., 2018)Test perplexity78.93Unverified
8Zaremba et al. (2014) - LSTM (large)Test perplexity78.4Unverified
9Gal & Ghahramani (2016) - Variational LSTM (large)Test perplexity75.2Unverified
10Inan et al. (2016) - Variational RHNTest perplexity66Unverified
#ModelMetricClaimedVerifiedStatus
1LSTM (7 layers)Bit per Character (BPC)1.67Unverified
2HypernetworksBit per Character (BPC)1.34Unverified
3SHA-LSTM (4 layers, h=1024, no attention head)Bit per Character (BPC)1.33Unverified
4LN HM-LSTMBit per Character (BPC)1.32Unverified
5ByteNetBit per Character (BPC)1.31Unverified
6Recurrent Highway NetworksBit per Character (BPC)1.27Unverified
7Large FS-LSTM-4Bit per Character (BPC)1.25Unverified
8Large mLSTMBit per Character (BPC)1.24Unverified
9AWD-LSTM (3 layers)Bit per Character (BPC)1.23Unverified
10Cluster-Former (#C=512)Bit per Character (BPC)1.22Unverified
#ModelMetricClaimedVerifiedStatus
1Smaller Transformer 126M (pre-trained)Test perplexity33Unverified
2OPT 125MTest perplexity32.26Unverified
3Larger Transformer 771M (pre-trained)Test perplexity28.1Unverified
4OPT 1.3BTest perplexity19.55Unverified
5GPT-Neo 125MTest perplexity17.83Unverified
6OPT 2.7BTest perplexity17.81Unverified
7Smaller Transformer 126M (fine-tuned)Test perplexity12Unverified
8GPT-Neo 1.3BTest perplexity11.46Unverified
9Transformer 125MTest perplexity10.7Unverified
10GPT-Neo 2.7BTest perplexity10.44Unverified