SOTAVerified

Multimodal Deep Learning

Multimodal deep learning is a type of deep learning that combines information from multiple modalities, such as text, image, audio, and video, to make more accurate and comprehensive predictions. It involves training deep neural networks on data that includes multiple types of information and using the network to make predictions based on this combined data.

One of the key challenges in multimodal deep learning is how to effectively combine information from multiple modalities. This can be done using a variety of techniques, such as fusing the features extracted from each modality, or using attention mechanisms to weight the contribution of each modality based on its importance for the task at hand.

Multimodal deep learning has many applications, including image captioning, speech recognition, natural language processing, and autonomous vehicles. By combining information from multiple modalities, multimodal deep learning can improve the accuracy and robustness of models, enabling them to perform better in real-world scenarios where multiple types of information are present.

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 51–75 of 213 papers

TitleStatusHype
In the Search for Optimal Multi-view Learning Models for Crop Classification with Global Remote Sensing DataCode0
Describe-and-Dissect: Interpreting Neurons in Vision Networks with Language Models—0
Integrating Wearable Sensor Data and Self-reported Diaries for Personalized Affect Forecasting—0
MoPE: Mixture of Prompt Experts for Parameter-Efficient and Scalable Multimodal FusionCode1
A Multimodal Intermediate Fusion Network with Manifold Learning for Stress Detection—0
Restoring Ancient Ideograph: A Multimodal Multitask Neural Network ApproachCode0
Multimodal deep learning approach to predicting neurological recovery from coma after cardiac arrest—0
DeepSeek-VL: Towards Real-World Vision-Language UnderstandingCode7
Cultural-Aware AI Model for Emotion RecognitionCode0
Multimodal Learning To Improve Cardiac Late Mechanical Activation Detection From Cine MR Images—0
Robust CLIP: Unsupervised Adversarial Fine-Tuning of Vision Embeddings for Robust Large Vision-Language ModelsCode2
Multimodal-Enhanced Objectness Learner for Corner Case Detection in Autonomous DrivingCode1
InstructIR: High-Quality Image Restoration Following Human InstructionsCode4
Multimodal Deep Learning of Word-of-Mouth Text and Demographics to Predict Customer Rating: Handling Consumer Heterogeneity in Marketing—0
Uncertainty-Aware Hardware Trojan Detection Using Multimodal Deep LearningCode0
Multimodal Urban Areas of Interest Generation via Remote Sensing Imagery and Geographical Prior—0
Predicting the Skies: A Novel Model for Flight-Level Passenger Traffic Forecasting—0
Multimodal self-supervised learning for lesion localization—0
Linguistic-Aware Patch Slimming Framework for Fine-grained Cross-Modal AlignmentCode2
Integrating Chemical Language and Molecular Graph in Multimodal Fused Deep Learning for Drug Property Prediction—0
A graph-based multimodal framework to predict gentrification—0
SynthScribe: Deep Multimodal Tools for Synthesizer Sound Retrieval and Exploration—0
TextAug: Test time Text Augmentation for Multimodal Person Re-identification—0
Enhancing Scene Graph Generation with Hierarchical Relationships and Commonsense KnowledgeCode1
Multimodal deep learning for mapping forest dominant height by fusing GEDI with earth observation data—0
Show:102550
← PrevPage 3 of 9Next →

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1Two Branch Network (Text - Bert + Image - Nts-Net)Accuracy96.81—Unverified