SOTAVerified

multimodal generation

Multimodal generation refers to the process of generating outputs that incorporate multiple modalities, such as images, text, and sound. This can be done using deep learning models that are trained on data that includes multiple modalities, allowing the models to generate output that is informed by more than one type of data.

For example, a multimodal generation model could be trained to generate captions for images that incorporate both text and visual information. The model could learn to identify objects in the image and generate descriptions of them in natural language, while also taking into account contextual information and the relationships between the objects in the image.

Multimodal generation can also be used in other applications, such as generating realistic images from textual descriptions or generating audio descriptions of video content. By combining multiple modalities in this way, multimodal generation models can produce more accurate and comprehensive output, making them useful for a wide range of applications.

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 76–98 of 98 papers

TitleStatusHype
Multimodal ELBO with Diffusion Decoders—0
Enabling Multimodal Generation on CLIP via Vision-Language Knowledge Distillation—0
Enabling Multimodal Generation on CLIP via Vision-Language Knowledge Distillation—0
SPAE: Semantic Pyramid AutoEncoder for Multimodal Generation with Frozen LLMs—0
Food Development through Co-creation with AI: bread with a "taste of love"—0
From Principles to Applications: A Comprehensive Survey of Discrete Tokenizers in Generation, Comprehension, Recommendation, and Information Retrieval—0
Stance-Driven Multimodal Controlled Statement Generation: New Dataset and Task—0
CoDi-2: In-Context, Interleaved, and Interactive Any-to-Any Generation—0
Generating Multimodal Driving Scenes via Next-Scene Prediction—0
Characterizing and Efficiently Accelerating Multimodal Generation Model Inference—0
CLIP Model for Images to Textual Prompts Based on Top-k Neighbors—0
Have we unified image generation and understanding yet? An empirical study of GPT-4o's image generation ability—0
I Want This Product but Different : Multimodal Retrieval with Synthetic Query Expansion—0
Latent Dirichlet Allocation in Generative Adversarial Networks—0
Learning Multimodal Latent Space with EBM Prior and MCMC Inference—0
The Evolution of Multimodal Model Architectures—0
LiveChat: Video Comment Generation from Audio-Visual Multimodal Contexts—0
LMFusion: Adapting Pretrained Language Models for Multimodal Generation—0
C3Net: Compound Conditioned ControlNet for Multimodal Content Generation—0
MADFormer: Mixed Autoregressive and Diffusion Transformers for Continuous Image Generation—0
C3LLM: Conditional Multimodal Content Generation Using Large Language Models—0
Towards Vision Enhancing LLMs: Empowering Multimodal Knowledge Storage and Sharing in LLMs—0
Both Ears Wide Open: Towards Language-Driven Spatial Audio Generation—0
Show:102550
← PrevPage 4 of 4Next →

No leaderboard results yet.