| GuideBench: Benchmarking Domain-Oriented Guideline Following for LLM Agents | May 16, 2025 | BenchmarkingInstruction Following | —Unverified | 0 |
| Benchmarking Critical Questions Generation: A Challenging Reasoning Task for Large Language Models | May 16, 2025 | Benchmarking | —Unverified | 0 |
| Massive-STEPS: Massive Semantic Trajectories for Understanding POI Check-ins -- Dataset and Benchmarks | May 16, 2025 | Benchmarking | CodeCode Available | 1 |
| Audio Turing Test: Benchmarking the Human-likeness of Large Language Model-based Text-to-Speech Systems in Chinese | May 16, 2025 | BenchmarkingLanguage Modeling | —Unverified | 0 |
| VitaGraph: Building a Knowledge Graph for Biologically Relevant Learning Tasks | May 16, 2025 | BenchmarkingLink Prediction | CodeCode Available | 0 |
| STEP: A Unified Spiking Transformer Evaluation Platform for Fair and Reproducible Benchmarking | May 16, 2025 | Benchmarking | CodeCode Available | 0 |
| CleanPatrick: A Benchmark for Image Data Cleaning | May 16, 2025 | BenchmarkingLabel Error Detection | CodeCode Available | 0 |
| Visual Anomaly Detection under Complex View-Illumination Interplay: A Large-Scale Benchmark | May 16, 2025 | Anomaly DetectionBenchmarking | —Unverified | 0 |
| Time Travel is Cheating: Going Live with DeepFund for Real-Time Fund Investment Benchmarking | May 16, 2025 | BenchmarkingManagement | CodeCode Available | 3 |
| MatTools: Benchmarking Large Language Models for Materials Science Tools | May 16, 2025 | BenchmarkingQuestion Answering | CodeCode Available | 1 |
| Relation Extraction Across Entire Books to Reconstruct Community Networks: The AffilKG Datasets | May 16, 2025 | BenchmarkingKnowledge Graphs | —Unverified | 0 |
| TCC-Bench: Benchmarking the Traditional Chinese Culture Understanding Capabilities of MLLMs | May 16, 2025 | BenchmarkingQuestion Answering | CodeCode Available | 0 |
| M4-SAR: A Multi-Resolution, Multi-Polarization, Multi-Scene, Multi-Source Dataset and Benchmark for Optical-SAR Fusion Object Detection | May 16, 2025 | Benchmarkingobject-detection | CodeCode Available | 1 |
| Benchmarking performance, explainability, and evaluation strategies of vision-language models for surgery: Challenges and opportunities | May 16, 2025 | Benchmarking | —Unverified | 0 |
| Words That Unite The World: A Unified Framework for Deciphering Central Bank Communications Globally | May 15, 2025 | BenchmarkingSentence | CodeCode Available | 1 |
| Model Performance-Guided Evaluation Data Selection for Effective Prompt Optimization | May 15, 2025 | BenchmarkingClustering | —Unverified | 0 |
| GNN-Suite: a Graph Neural Network Benchmarking Framework for Biomedical Informatics | May 15, 2025 | BenchmarkingGraph Neural Network | CodeCode Available | 0 |
| On the Evaluation of Engineering Artificial General Intelligence | May 15, 2025 | Benchmarking | —Unverified | 0 |
| Visual Fidelity Index for Generative Semantic Communications with Critical Information Embedding | May 15, 2025 | BenchmarkingSemantic Communication | —Unverified | 0 |
| PsOCR: Benchmarking Large Multimodal Models for Optical Character Recognition in Low-resource Pashto Language | May 15, 2025 | BenchmarkingOptical Character Recognition | CodeCode Available | 0 |
| Evaluating Robustness of Deep Reinforcement Learning for Autonomous Surface Vehicle Control in Field Tests | May 15, 2025 | BenchmarkingDeep Reinforcement Learning | CodeCode Available | 1 |
| Do LLMs Memorize Recommendation Datasets? A Preliminary Study on MovieLens-1M | May 15, 2025 | BenchmarkingMemorization | CodeCode Available | 0 |
| MMLongBench: Benchmarking Long-Context Vision-Language Models Effectively and Thoroughly | May 15, 2025 | 8kBenchmarking | CodeCode Available | 2 |
| DIF: A Framework for Benchmarking and Verifying Implicit Bias in LLMs | May 15, 2025 | BenchmarkingFairness | —Unverified | 0 |
| What Does Neuro Mean to Cardio? Investigating the Role of Clinical Specialty Data in Medical LLMs | May 15, 2025 | AllBenchmarking | —Unverified | 0 |
| JointDistill: Adaptive Multi-Task Distillation for Joint Depth Estimation and Scene Segmentation | May 15, 2025 | BenchmarkingDepth Estimation | —Unverified | 0 |
| Real-World fNIRS-Based Brain-Computer Interfaces: Benchmarking Deep Learning and Classical Models in Interactive Gaming | May 15, 2025 | BenchmarkingData Augmentation | —Unverified | 0 |
| Towards scalable surrogate models based on Neural Fields for large scale aerodynamic simulations | May 14, 2025 | Benchmarking | CodeCode Available | 1 |
| TARGET: Benchmarking Table Retrieval for Generative Tasks | May 14, 2025 | BenchmarkingRepresentation Learning | —Unverified | 0 |
| VeriFact: Enhancing Long-Form Factuality Evaluation with Refined Fact Extraction and Reference Facts | May 14, 2025 | BenchmarkingForm | —Unverified | 0 |
| KRISTEVA: Close Reading as a Novel Task for Benchmarking Interpretive Reasoning | May 14, 2025 | BenchmarkingMMLU | —Unverified | 0 |
| ManipBench: Benchmarking Vision-Language Models for Low-Level Robot Manipulation | May 14, 2025 | BenchmarkingDeformable Object Manipulation | —Unverified | 0 |
| WorldView-Bench: A Benchmark for Evaluating Global Cultural Perspectives in Large Language Models | May 14, 2025 | Benchmarking | —Unverified | 0 |
| OpenLKA: An Open Dataset of Lane Keeping Assist from Recent Car Models under Real-world Driving Conditions | May 14, 2025 | Autonomous DrivingBenchmarking | CodeCode Available | 1 |
| A Standardized Benchmark Set of Clustering Problem Instances for Comparing Black-Box Optimizers | May 14, 2025 | BenchmarkingClustering | —Unverified | 0 |
| RobustSpring: Benchmarking Robustness to Image Corruptions for Optical Flow, Scene Flow and Stereo | May 14, 2025 | BenchmarkingOptical Flow Estimation | —Unverified | 0 |
| BioVFM-21M: Benchmarking and Scaling Self-Supervised Vision Foundation Models for Biomedical Image Analysis | May 14, 2025 | BenchmarkingComputational Efficiency | CodeCode Available | 0 |
| How Hungry is AI? Benchmarking Energy, Water, and Carbon Footprint of LLM Inference | May 14, 2025 | Benchmarking | —Unverified | 0 |
| Load-independent Metrics for Benchmarking Force Controllers | May 13, 2025 | Benchmarking | —Unverified | 0 |
| Grounding Synthetic Data Evaluations of Language Models in Unsupervised Document Corpora | May 13, 2025 | BenchmarkingDiagnostic | CodeCode Available | 0 |
| Granite-speech: open-source speech-aware LLMs with strong English ASR capabilities | May 13, 2025 | automatic-speech-translationBenchmarking | —Unverified | 0 |
| ExEBench: Benchmarking Foundation Models on Extreme Earth Events | May 13, 2025 | BenchmarkingManagement | CodeCode Available | 0 |
| A Large-scale Benchmark on Geological Fault Delineation Models: Domain Shift, Training Dynamics, Generalizability, Evaluation and Inferential Behavior | May 13, 2025 | BenchmarkingSeismic Interpretation | —Unverified | 0 |
| Benchmarking AI scientists in omics data-driven biological research | May 13, 2025 | BenchmarkingMultiple-choice | CodeCode Available | 1 |
| Large Language Model Psychometrics: A Systematic Review of Evaluation, Validation, and Enhancement | May 13, 2025 | BenchmarkingLanguage Modeling | CodeCode Available | 2 |
| Benchmarking Graph Neural Networks for Document Layout Analysis in Public Affairs | May 12, 2025 | BenchmarkingDocument Layout Analysis | —Unverified | 0 |
| PRISM: Complete Online Decentralized Multi-Agent Pathfinding with Rapid Information Sharing using Motion Constraints | May 12, 2025 | Benchmarking | —Unverified | 0 |
| Benchmarking Ethical and Safety Risks of Healthcare LLMs in China-Toward Systemic Governance under Healthy China 2030 | May 12, 2025 | BenchmarkingEthics | —Unverified | 0 |
| FalseReject: A Resource for Improving Contextual Safety and Mitigating Over-Refusals in LLMs via Structured Reasoning | May 12, 2025 | 16kBenchmarking | —Unverified | 0 |
| From raw affiliations to organization identifiers | May 12, 2025 | BenchmarkingMetadata quality | CodeCode Available | 0 |