Multimodal Deep Learning

Multimodal deep learning is a type of deep learning that combines information from multiple modalities, such as text, image, audio, and video, to make more accurate and comprehensive predictions. It involves training deep neural networks on data that includes multiple types of information and using the network to make predictions based on this combined data.

One of the key challenges in multimodal deep learning is how to effectively combine information from multiple modalities. This can be done using a variety of techniques, such as fusing the features extracted from each modality, or using attention mechanisms to weight the contribution of each modality based on its importance for the task at hand.

Multimodal deep learning has many applications, including image captioning, speech recognition, natural language processing, and autonomous vehicles. By combining information from multiple modalities, multimodal deep learning can improve the accuracy and robustness of models, enabling them to perform better in real-world scenarios where multiple types of information are present.

Papers

Recently Added Most Hyped Most Active Needs Verification Most Verified

Showing 131–140 of 213 papers

Title	Date	Tasks	Status
Multimodal deep learning for mapping forest dominant height by fusing GEDI with earth observation data	Nov 20, 2023	Earth ObservationMultimodal Deep Learning	—Unverified
Multimodal Deep Learning for Mental Disorders Prediction from Audio Speech Samples	Sep 3, 2019	Multimodal Deep LearningRepresentation Learning	—Unverified
Timing Is Everything: Finding the Optimal Fusion Points in Multimodal Medical Imaging	May 5, 2025	DiagnosticMultimodal Deep Learning	—Unverified
A Comprehensive and Versatile Multimodal Deep Learning Approach for Predicting Diverse Properties of Advanced Materials	Mar 29, 2023	Deep LearningMultimodal Deep Learning	—Unverified
Multimodal Deep Learning for Scientific Imaging Interpretation	Sep 21, 2023	ArticlesBenchmarking	—Unverified
Multimodal deep learning for short-term stock volatility prediction	Dec 25, 2018	Deep LearningMultimodal Deep Learning	—Unverified
Multimodal Deep Learning for Stroke Prediction and Detection using Retinal Imaging and Clinical Data	May 5, 2025	Multimodal Deep LearningSelf-Supervised Learning	—Unverified
A C++ library for Multimodal Deep Learning	Dec 22, 2015	Deep LearningMultimodal Deep Learning	—Unverified
Multimodal Deep Learning Framework for Image Popularity Prediction on Social Media	May 18, 2021	Image popularity predictionMultimodal Deep Learning	—Unverified
Multimodal Deep Learning of Word-of-Mouth Text and Demographics to Predict Customer Rating: Handling Consumer Heterogeneity in Marketing	Jan 22, 2024	MarketingMultimodal Deep Learning	—Unverified

Show:10 25 50

← PrevPage 14 of 22Next →

Benchmark Results

#	Model	Metric	Claimed	Verified	Status
1	Two Branch Network (Text - Bert + Image - Nts-Net)	Accuracy	96.81	—	Unverified