SOTAVerified

Multispectral Object Detection

Only using RGB cameras for automatic outdoor scene analysis is challenging when, for example, facing insufficient illumination or adverse weather. To improve the recognition reliability, multispectral systems add additional cameras (e.g. infra-red) and perform object detection from multispectral data. Although multispectral scene analysis with deep learning has be shown to have a great potential, there are still many open research questions and it has not been widely deployed in industrial contexts.

Papers

Showing 139 of 39 papers

TitleStatusHype
YOLOv11-RGBT: Towards a Comprehensive Single-Stage Multispectral Object Detection FrameworkCode4
When Pedestrian Detection Meets Multi-Modal Learning: Generalist Model and Benchmark DatasetCode2
UniRGB-IR: A Unified Framework for RGB-Infrared Semantic Tasks via Adapter TuningCode2
CFMW: Cross-modality Fusion Mamba for Multispectral Object Detection under Adverse Weather ConditionsCode2
Removal then Selection: A Coarse-to-Fine Fusion Perspective for RGB-Infrared Object DetectionCode2
ICAFusion: Iterative Cross-Attention Guided Feature Fusion for Multispectral Object DetectionCode2
CMX: Cross-Modal Fusion for RGB-X Semantic Segmentation with TransformersCode2
Rethinking Early-Fusion Strategies for Improved Multispectral Object DetectionCode1
MiPa: Mixed Patch Infrared-Visible Modality Agnostic Object DetectionCode1
INSANet: INtra-INter Spectral Attention Network for Effective Feature Fusion of Multispectral Pedestrian DetectionCode1
RGB-X Object Detection via Scene-Specific Fusion ModulesCode1
C^2Former: Calibrated and Complementary Transformer for RGB-Infrared Object DetectionCode1
TFDet: Target-Aware Fusion for RGB-T Pedestrian DetectionCode1
ADJUST: A Dictionary-Based Joint Reconstruction and Unmixing Method for Spectral TomographyCode1
Cross-Modality Fusion Transformer for Multispectral Object DetectionCode1
LLVIP: A Visible-infrared Paired Dataset for Low-light VisionCode1
MLPD: Multi-Label Pedestrian Detector in Multispectral DomainCode1
Guided Attentive Feature Fusion for Multispectral Pedestrian DetectionCode1
Multispectral Fusion for Object Detection with Cyclic Fuse-and-Refine BlocksCode1
Improving Multispectral Pedestrian Detection by Addressing Modality Imbalance ProblemsCode1
Multispectral Deep Neural Networks for Pedestrian DetectionCode1
Fully Convolutional Networks for Semantic SegmentationCode1
Multispectral Detection Transformer with Infrared-Centric Sensor FusionCode0
Optimizing Multispectral Object Detection: A Bag of Tricks and Comprehensive Benchmarks0
CAFF-DINO: Multi-spectral object detection transformers with cross-attention features fusion0
Surveying You Only Look Once (YOLO) Multispectral Object Detection Advancements, Applications And Challenges0
RGB-T Object Detection via Group Shuffled Multi-receptive Attention and Multi-modal Supervision0
A Multispectral Automated Transfer Technique (MATT) for machine-driven image labeling utilizing the Segment Anything Model (SAM)0
Multimodal Object Detection by Channel Switching and Spatial Attention0
Translation, Scale and Rotation: Cross-Modal Alignment Meets RGB-Infrared Vehicle Detection0
Deep learning with RGB and thermal images onboard a drone for monitoring operations0
Confidence-aware Fusion using Dempster-Shafer Theory for Multispectral Pedestrian DetectionCode0
A Comparison of Deep Saliency Map Generators on Multispectral Data in Object Detection0
Multispectral Object Detection with Deep Learning0
Weakly Aligned Cross-Modal Learning for Multispectral Pedestrian DetectionCode0
CIAN: Cross-Image Affinity Net for Weakly Supervised Semantic SegmentationCode0
Multispectral Pedestrian Detection via Simultaneous Detection and SegmentationCode0
Illumination-aware Faster R-CNN for Robust Multispectral Pedestrian Detection0
Fusion of Multispectral Data Through Illumination-aware Deep Neural Networks for Pedestrian Detection0
Show:102550

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1MMPedestronmAP5086.4Unverified
2RGB-X Scene Adaptive CBAMmAP5086.16Unverified
3CAFF-DINOmAP5085.5Unverified
4RSDetmAP5083.9Unverified
5CMXmAP5082.2Unverified
6UniRGB-IRmAP5081.4Unverified
7MiPamAP5081.3Unverified
8CSSAmAP5079.2Unverified
9CFTmAP5077.7Unverified
10ProbEnmAP5075.5Unverified
#ModelMetricClaimedVerifiedStatus
1FusionRPN+BFAll Miss Rate51.7Unverified
2Halfway FusionAll Miss Rate49.18Unverified
3IATDNN+IASSAll Miss Rate48.96Unverified
4IAFR-CNNAll Miss Rate44.23Unverified
5CIANAll Miss Rate35.53Unverified
6AR-CNNAll Miss Rate34.95Unverified
7MSDS-R-CNNAll Miss Rate34.15Unverified
8MBNetAll Miss Rate31.87Unverified
9TSFADetAll Miss Rate30.74Unverified
10CMPDAll Miss Rate28.98Unverified
#ModelMetricClaimedVerifiedStatus
1YOLOv3-4‐channelmAP@0.5:0.9564.4Unverified
2YOLOv3-EnsemblemAP@0.5:0.9553.4Unverified
#ModelMetricClaimedVerifiedStatus
1CFTmAP5097.5Unverified