SOTAVerified

Multimodal Deep Learning

Multimodal deep learning is a type of deep learning that combines information from multiple modalities, such as text, image, audio, and video, to make more accurate and comprehensive predictions. It involves training deep neural networks on data that includes multiple types of information and using the network to make predictions based on this combined data.

One of the key challenges in multimodal deep learning is how to effectively combine information from multiple modalities. This can be done using a variety of techniques, such as fusing the features extracted from each modality, or using attention mechanisms to weight the contribution of each modality based on its importance for the task at hand.

Multimodal deep learning has many applications, including image captioning, speech recognition, natural language processing, and autonomous vehicles. By combining information from multiple modalities, multimodal deep learning can improve the accuracy and robustness of models, enabling them to perform better in real-world scenarios where multiple types of information are present.

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 151–160 of 213 papers

TitleStatusHype
The Role of Emotions in Informational Support Question-Response Pairs in Online Health Communities: A Multimodal Deep Learning Approach—0
Timing Is Everything: Finding the Optimal Fusion Points in Multimodal Medical Imaging—0
Toxicity Prediction by Multimodal Deep Learning—0
Unified Cross-Modal Attention-Mixer Based Structural-Functional Connectomics Fusion for Neuropsychiatric Disorder Diagnosis—0
Using Neural Architecture Search for Improving Software Flaw Detection in Multimodal Deep Learning Models—0
Validation & Exploration of Multimodal Deep-Learning Camera-Lidar Calibration models—0
Variational methods for Conditional Multimodal Deep Learning—0
Vision-Aided Frame-Capture-Based CSI Recomposition for WiFi Sensing: A Multimodal Approach—0
Where and When: Space-Time Attention for Audio-Visual Explanations—0
Multimodal Co-learning: Challenges, Applications with Datasets, Recent Advances and Future Directions—0
Show:102550
← PrevPage 16 of 22Next →

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1Two Branch Network (Text - Bert + Image - Nts-Net)Accuracy96.81—Unverified