Dynamic Fusion Strategies for Federated Multimodal Recommendations

Abstract

Delivering deeply personalized recommendations necessitates understanding user interactions with diverse multimedia features, but achieving this within the constraints of Federated Recommendation Systems (FedRec) is severely hampered by communication bottlenecks, user heterogeneity, and the complexity of privacy-preserving multimodal fusion. To this end, we propose FedMR, a novel multimodal FedRec framework centered around the Mixing Feature Fusion Module (MFFM). FedMR employs a two-stage process: (1) Server-side centralized multimedia content processing provides rich, shared item context using pre-trained models, mitigating limitations from client sparsity and resource constraints efficiently. (2) Client-Side Personalized Refinement, where the MFFM dynamically adapts these server-provided multimodal representations based on client-specific interaction patterns, effectively tailoring recommendations and resolving heterogeneity in user preferences towards different modalities. Extensive experiments validate that FedMR seamlessly enhances existing ID-based FedRecs, effectively transforming them into high-performing federated multimodal systems.

Tasks

Reproductions