SOTAVerified

Image Captioning

Image Captioning is the task of describing the content of an image in words. This task lies at the intersection of computer vision and natural language processing. Most image captioning systems use an encoder-decoder framework, where an input image is encoded into an intermediate representation of the information in the image, and then decoded into a descriptive text sequence. The most popular benchmarks are nocaps and COCO, and models are typically evaluated according to a BLEU or CIDER metric.

( Image credit: Reflective Decoding Network for Image Captioning, ICCV'19)

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 651–700 of 1878 papers

TitleStatusHype
Mindstorms in Natural Language-Based Societies of Mind—0
HAAV: Hierarchical Aggregation of Augmented Views for Image Captioning—0
Exploring Diverse In-Context Configurations for Image CaptioningCode1
EmbodiedGPT: Vision-Language Pre-Training via Embodied Chain of Thought—0
An Examination of the Robustness of Reference-Free Image Captioning Evaluation MetricsCode0
Visually-Situated Natural Language Understanding with Contrastive Reading Model and Frozen Large Language ModelsCode1
Gender Biases in Automatic Evaluation Metrics for Image CaptioningCode0
Exploring Affordance and Situated Meaning in Image Captions: A Multimodal Analysis—0
Alt-Text with Context: Improving Accessibility for Images on Twitter—0
Text encoders bottleneck compositionality in contrastive vision-language modelsCode1
PIC-XAI: Post-hoc Image Captioning Explanation using SegmentationCode0
MemeCap: A Dataset for Captioning and Interpreting MemesCode1
Text-based Person Search without Parallel Image-Text Data—0
What Makes for Good Visual Tokenizers for Large Language Models?Code1
A request for clarity over the End of Sequence token in the Self-Critical Sequence TrainingCode0
DiffCap: Exploring Continuous Diffusion on Image Captioning—0
Cross2StrA: Unpaired Cross-lingual Image Captioning with Cross-lingual Cross-modal Structure-pivoted Alignment—0
Enhancing Vision-Language Pre-Training with Jointly Learned Questioner and Dense CaptionerCode1
Brain Captioning: Decoding human brain activity into images and textCode1
Bridging the Domain Gap: Self-Supervised 3D Scene Understanding with Foundation ModelsCode1
Semantic Composition in Visually Grounded Language Models—0
IMAGINATOR: Pre-Trained Image+Text Joint Embeddings using Word-Level Grounding of ImagesCode0
Simple Token-Level Confidence Improves Caption Correctness—0
Towards L-System Captioning for Tree Reconstruction—0
InfoMetIC: An Informative Metric for Reference-free Image Caption EvaluationCode1
WikiWeb2M: A Page-Level Multimodal Wikipedia DatasetCode3
Vision-Language Models in Remote Sensing: Current Progress and Future TrendsCode1
Exploiting Pseudo Image Captions for Multimodal Summarization—0
UIT-OpenViIC: A Novel Benchmark for Evaluating Image Captioning in Vietnamese—0
The Role of Data Curation in Image CaptioningCode0
A Suite of Generative Tasks for Multi-Level Multimodal Webpage UnderstandingCode0
Image Captioners Sometimes Tell More Than Images They See—0
Caption Anything: Interactive Image Description with Diverse Multimodal ControlsCode3
Making the Most of What You Have: Adapting Pre-trained Visual Language Models in the Low-data Regime—0
Multimodal Data Augmentation for Image Captioning using Diffusion ModelsCode0
Fairness in AI Systems: Mitigating gender bias from language-vision models—0
Transforming Visual Scene Graphs to Image CaptionsCode1
Quality-agnostic Image Captioning to Safely Assist People with Vision Impairment—0
Learning Human-Human Interactions in Images from Weak Textual Supervision—0
From Association to Generation: Text-only Captioning by Unsupervised Cross-modal MappingCode1
TTIDA: Controllable Generative Data Augmentation via Text-to-Text and Text-to-Image ModelsCode0
VALOR: Vision-Audio-Language Omni-Perception Pretraining Model and DatasetCode2
A-CAP: Anticipation Captioning with Commonsense Knowledge—0
Advancing Medical Imaging with Language Models: A Journey from N-grams to ChatGPT—0
Boosting Cross-task Transferability of Adversarial Patches with Visual Relations—0
ImageCaptioner^2: Image Captioner for Image Captioning Bias Amplification Assessment—0
Model-Agnostic Gender Debiased Image CaptioningCode0
Uncurated Image-Text Datasets: Shedding Light on Demographic BiasCode1
Towards Self-Explainability of Deep Neural Networks with Heatmap Captioning and Large-Language Models—0
Scalable and Accurate Self-supervised Multimodal Representation Learning without Aligned Video and Text Data—0
Show:102550
← PrevPage 14 of 38Next →
All datasetsVizWiz 2020 test-devCOCO Captionsnocaps in-domainnocaps near-domainnocaps out-of-domainnocaps entireCOCO (Common Objects in Context)VizWiz 2020 testnocaps-XD entirenocaps-val-in-domainnocaps-val-overallnocaps-XD in-domain

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1IBM Research AICIDEr80.67—Unverified
2CASIA_IVACIDEr79.15—Unverified
3feixiangCIDEr77.31—Unverified
4wocaoCIDEr77.21—Unverified
5lamiwab172CIDEr75.93—Unverified
6RUC_AIM3CIDEr73.52—Unverified
7funasCIDEr73.51—Unverified
8SRC-B_VCLabCIDEr73.47—Unverified
9spartaCIDEr73.41—Unverified
10x-vizCIDEr73.26—Unverified
#ModelMetricClaimedVerifiedStatus
1VALORCIDER152.5—Unverified
2VASTCIDER149—Unverified
3Virtex (ResNet-101)CIDER94—Unverified
4KOSMOS-1 (1.6B) (zero-shot)CIDER84.7—Unverified
5BLIP-FuseCapCLIPScore78.5—Unverified
6mPLUGBLEU-446.5—Unverified
7OFABLEU-444.9—Unverified
8GITBLEU-444.1—Unverified
9BLIP-2 ViT-G OPT 2.7B (zero-shot)BLEU-443.7—Unverified
10BLIP-2 ViT-G OPT 6.7B (zero-shot)BLEU-443.5—Unverified
#ModelMetricClaimedVerifiedStatus
1PaLICIDEr149.1—Unverified
2GIT2, Single ModelCIDEr124.18—Unverified
3GIT, Single ModelCIDEr122.4—Unverified
4PaLICIDEr121.09—Unverified
5CoCa - Google BrainCIDEr117.9—Unverified
6Microsoft Cognitive Services teamCIDEr112.82—Unverified
7Single ModelCIDEr108.98—Unverified
8GRIT (zero-shot, no VL pretraining, no CBS)CIDEr105.9—Unverified
9FudanFVLCIDEr104.9—Unverified
10FudanWYZCIDEr104.25—Unverified
#ModelMetricClaimedVerifiedStatus
1GIT2, Single ModelCIDEr125.51—Unverified
2PaLICIDEr124.35—Unverified
3GIT, Single ModelCIDEr123.92—Unverified
4CoCa - Google BrainCIDEr120.73—Unverified
5Microsoft Cognitive Services teamCIDEr115.54—Unverified
6Single ModelCIDEr110.76—Unverified
7FudanFVLCIDEr109.33—Unverified
8FudanWYZCIDEr108.04—Unverified
9IEDA-LABCIDEr100.15—Unverified
10firetheholeCIDEr99.51—Unverified
#ModelMetricClaimedVerifiedStatus
1PaLICIDEr126.67—Unverified
2GIT2, Single ModelCIDEr122.27—Unverified
3GIT, Single ModelCIDEr122.04—Unverified
4CoCa - Google BrainCIDEr121.69—Unverified
5Microsoft Cognitive Services teamCIDEr110.14—Unverified
6Single ModelCIDEr109.49—Unverified
7FudanFVLCIDEr106.55—Unverified
8FudanWYZCIDEr103.75—Unverified
9HumanCIDEr91.62—Unverified
10firetheholeCIDEr88.54—Unverified