Multi-QuAD: Multi-Level Quality-Adaptive Dynamic Network for Reliable Multimodal Classification

Abstract

Multimodal machine learning has achieved remarkable progress in many scenarios, but its reliability is undermined by varying sample quality. This paper finds that existing reliable multimodal classification methods not only fail to provide robust estimation of data quality, but also lack dynamic networks for sample-specific depth and parameters to achieve reliable inference. To this end, a novel framework for multimodal reliable classification termed Multi-level Quality-Adaptive Dynamic multimodal network (Multi-QuAD) is proposed. Multi-QuAD first adopts a novel approach based on noise-free prototypes and a classifier-free design to reliably estimate the quality of each sample at both modality and feature levels. It then achieves sample-specific network depth via the Global Confidence Normalized Depth (GCND) mechanism. By normalizing depth across modalities and samples, GCND effectively mitigates the impact of challenging modality inputs on dynamic depth reliability. Furthermore, Multi-QuAD provides sample-adaptive network parameters via the Layer-wise Greedy Parameter (LGP) mechanism driven by feature-level quality. The cross-modality layer-wise greedy strategy in LGP designs a reliable parameter prediction paradigm for multimodal networks with variable architecture for the first time. Experiments conducted on four datasets demonstrate that Multi-QuAD significantly outperforms state-of-the-art methods in classification performance and reliability, exhibiting strong adaptability to data with diverse quality.

Tasks

Reproductions