SOTAVerified

multimodal generation

Multimodal generation refers to the process of generating outputs that incorporate multiple modalities, such as images, text, and sound. This can be done using deep learning models that are trained on data that includes multiple modalities, allowing the models to generate output that is informed by more than one type of data.

For example, a multimodal generation model could be trained to generate captions for images that incorporate both text and visual information. The model could learn to identify objects in the image and generate descriptions of them in natural language, while also taking into account contextual information and the relationships between the objects in the image.

Multimodal generation can also be used in other applications, such as generating realistic images from textual descriptions or generating audio descriptions of video content. By combining multiple modalities in this way, multimodal generation models can produce more accurate and comprehensive output, making them useful for a wide range of applications.

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 51–98 of 98 papers

TitleStatusHype
ProtTeX: Structure-In-Context Reasoning and Editing of Proteins with Large Language Models—0
ARMOR v0.1: Empowering Autoregressive Multimodal Understanding Model with Interleaved Multimodal Generation via Asymmetric Synergy—0
Unlocking Pretrained LLMs for Motion-Related Multimodal Generation: A Fine-Tuning Approach to Unify Diffusion and Next-Token Prediction—0
From Principles to Applications: A Comprehensive Survey of Discrete Tokenizers in Generation, Comprehension, Recommendation, and Information Retrieval—0
A Survey on Bridging EEG Signals and Generative AI: From Image and Text to Beyond—0
Artificial Intelligence in Creative Industries: Advances Prior to 2025—0
RDPM: Solve Diffusion Probabilistic Models via Recurrent Token Prediction—0
D-Judge: How Far Are We? Evaluating the Discrepancies Between AI-synthesized Images and Natural Images through Multimodal GuidanceCode0
LMFusion: Adapting Pretrained Language Models for Multimodal Generation—0
Multimodal Latent Language Modeling with Next-Token DiffusionCode0
Visatronic: A Multimodal Decoder-Only Model for Speech Synthesis—0
Benchmarking Multimodal Models for Ukrainian Language Understanding Across Academic and Cultural Domains—0
A Survey on Vision Autoregressive Model—0
A Survey of Emerging Approaches and Advances in Video Generation—0
Both Ears Wide Open: Towards Language-Driven Spatial Audio Generation—0
ACDC: Autoregressive Coherent Multimodal Generation using Diffusion Correction—0
Characterizing and Efficiently Accelerating Multimodal Generation Model Inference—0
PixelBytes: Catching Unified Representation for Multimodal GenerationCode0
PixelBytes: Catching Unified Embedding for Multimodal GenerationCode0
Multimodal ELBO with Diffusion Decoders—0
Learning Multimodal Latent Space with EBM Prior and MCMC Inference—0
Pre-trained Encoder Inference: Revealing Upstream Encoders In Downstream Machine Learning ServicesCode0
Diffusion Models For Multi-Modal Generative Modeling—0
Empathic Grounding: Explorations using Multimodal Interaction and Large Language Models with Conversational AgentsCode0
The Evolution of Multimodal Model Architectures—0
C3LLM: Conditional Multimodal Content Generation Using Large Language Models—0
Multimodal Pretraining and Generation for Recommendation: A Tutorial—0
PCQA: A Strong Baseline for AIGC Quality Assessment Based on Prompt Condition—0
Food Development through Co-creation with AI: bread with a "taste of love"—0
3D-VLA: A 3D Vision-Language-Action Generative World Model—0
CLIP Model for Images to Textual Prompts Based on Top-k Neighbors—0
CoDi-2: In-Context, Interleaved, and Interactive Any-to-Any Generation—0
C3Net: Compound Conditioned ControlNet for Multimodal Content Generation—0
Towards Vision Enhancing LLMs: Empowering Multimodal Knowledge Storage and Sharing in LLMs—0
LiveChat: Video Comment Generation from Audio-Visual Multimodal Contexts—0
Consistent Multimodal Generation via A Unified GAN FrameworkCode0
SPAE: Semantic Pyramid AutoEncoder for Multimodal Generation with Frozen LLMs—0
Multi-modal Latent DiffusionCode0
DiffuSIA: A Spiral Interaction Architecture for Encoder-Decoder Text Diffusion—0
Accountable Textual-Visual Chat Learns to Reject Human Instructions in Image Re-creationCode0
Multimedia Generative Script Learning for Task PlanningCode0
Multimodal Generation of Novel Action Appearances for Synthetic-to-Real Recognition of Activities of Daily LivingCode0
Unconditional Image-Text Pair Generation with Multimodal Cross QuantizerCode0
Enabling Multimodal Generation on CLIP via Vision-Language Knowledge Distillation—0
Enabling Multimodal Generation on CLIP via Vision-Language Knowledge Distillation—0
I Want This Product but Different : Multimodal Retrieval with Synthetic Query Expansion—0
Continual and Multi-Task Architecture SearchCode0
Latent Dirichlet Allocation in Generative Adversarial Networks—0
Show:102550
← PrevPage 2 of 2Next →

No leaderboard results yet.