| An OpenMind for 3D medical vision self-supervised learning | Dec 22, 2024 | BenchmarkingSelf-Supervised Learning | CodeCode Available | 2 |
| First-frame Supervised Video Polyp Segmentation via Propagative and Semantic Dual-teacher Network | Dec 21, 2024 | BenchmarkingTransfer Learning | CodeCode Available | 0 |
| HammerBench: Fine-Grained Function-Calling Evaluation in Real Mobile Device Scenarios | Dec 21, 2024 | Benchmarking | CodeCode Available | 0 |
| Patherea: Cell Detection and Classification for the 2020s | Dec 21, 2024 | BenchmarkingCell Detection | —Unverified | 0 |
| A Classification Benchmark for Artificial Intelligence Detection of Laryngeal Cancer from Patient Voice | Dec 20, 2024 | BenchmarkingDiagnostic | CodeCode Available | 0 |
| Toward Robust Hyper-Detailed Image Captioning: A Multiagent Approach and Dual Evaluation Metrics for Factuality and Coverage | Dec 20, 2024 | AttributeBenchmarking | —Unverified | 0 |
| Benchmarking LLMs and SLMs for patient reported outcomes | Dec 20, 2024 | BenchmarkingPrivacy Preserving | —Unverified | 0 |
| Enriching Social Science Research via Survey Item Linking | Dec 20, 2024 | BenchmarkingEntity Disambiguation | CodeCode Available | 0 |
| XRAG: eXamining the Core -- Benchmarking Foundational Components in Advanced Retrieval-Augmented Generation | Dec 20, 2024 | BenchmarkingDiagnostic | CodeCode Available | 2 |
| TelcoLM: collecting data, adapting, and benchmarking language models for the telecommunication domain | Dec 20, 2024 | Benchmarking | —Unverified | 0 |
| Deciphering the Underserved: Benchmarking LLM OCR for Low-Resource Scripts | Dec 20, 2024 | BenchmarkingOptical Character Recognition | CodeCode Available | 0 |
| AI-generated Image Quality Assessment in Visual Communication | Dec 20, 2024 | BenchmarkingImage Quality Assessment | CodeCode Available | 0 |
| Generative CKM Construction using Partially Observed Data with Diffusion Model | Dec 19, 2024 | Benchmarking | CodeCode Available | 1 |
| TOMG-Bench: Evaluating LLMs on Text-based Open Molecule Generation | Dec 19, 2024 | BenchmarkingDescription-guided molecule generation | CodeCode Available | 1 |
| AutoTrust: Benchmarking Trustworthiness in Large Vision Language Models for Autonomous Driving | Dec 19, 2024 | Autonomous DrivingBenchmarking | CodeCode Available | 2 |
| Pitfalls of topology-aware image segmentation | Dec 19, 2024 | BenchmarkingImage Segmentation | —Unverified | 0 |
| Autonomous Microscopy Experiments through Large Language Model Agents | Dec 18, 2024 | BenchmarkingExperimental Design | CodeCode Available | 1 |
| Mind Your Theory: Theory of Mind Goes Deeper Than Reasoning | Dec 18, 2024 | BenchmarkingPosition | —Unverified | 0 |
| AntiLeak-Bench: Preventing Data Contamination by Automatically Constructing Benchmarks with Updated Real-World Knowledge | Dec 18, 2024 | BenchmarkingWorld Knowledge | CodeCode Available | 0 |
| TheAgentCompany: Benchmarking LLM Agents on Consequential Real World Tasks | Dec 18, 2024 | Benchmarking | CodeCode Available | 1 |
| Benchmarking and Improving Large Vision-Language Models for Fundamental Visual Graph Understanding and Reasoning | Dec 18, 2024 | BenchmarkingGraph Learning | CodeCode Available | 1 |
| Generation of Large District Heating System Models Using Open-Source Data and Tools: An Exemplary Workflow | Dec 18, 2024 | Benchmarking | —Unverified | 0 |
| Open Universal Arabic ASR Leaderboard | Dec 18, 2024 | Benchmarking | CodeCode Available | 2 |
| RAG-RewardBench: Benchmarking Reward Models in Retrieval Augmented Generation for Preference Alignment | Dec 18, 2024 | BenchmarkingRAG | CodeCode Available | 1 |
| DateLogicQA: Benchmarking Temporal Biases in Large Language Models | Dec 17, 2024 | Benchmarking | CodeCode Available | 0 |
| Multi-Dimensional Insights: Benchmarking Real-World Personalization in Large Multimodal Models | Dec 17, 2024 | Benchmarking | —Unverified | 0 |
| Benchmarking and Understanding Compositional Relational Reasoning of LLMs | Dec 17, 2024 | BenchmarkingRelational Reasoning | CodeCode Available | 0 |
| Selective Shot Learning for Code Explanation | Dec 17, 2024 | Benchmarking | —Unverified | 0 |
| C-FedRAG: A Confidential Federated Retrieval-Augmented Generation System | Dec 17, 2024 | BenchmarkingRAG | —Unverified | 0 |
| AI PERSONA: Towards Life-long Personalization of LLMs | Dec 17, 2024 | Benchmarking | —Unverified | 0 |
| F-Bench: Rethinking Human Preference Evaluation Metrics for Benchmarking Face Generation, Customization, and Restoration | Dec 17, 2024 | BenchmarkingFace Generation | —Unverified | 0 |
| A Scalable Approach to Benchmarking the In-Conversation Differential Diagnostic Accuracy of a Health AI | Dec 17, 2024 | BenchmarkingChatbot | —Unverified | 0 |
| ShiftedBronzes: Benchmarking and Analysis of Domain Fine-Grained Classification in Open-World Settings | Dec 17, 2024 | Benchmarking | —Unverified | 0 |
| How Different AI Chatbots Behave? Benchmarking Large Language Models in Behavioral Economics Games | Dec 16, 2024 | BenchmarkingChatbot | —Unverified | 0 |
| CharacterBench: Benchmarking Character Customization of Large Language Models | Dec 16, 2024 | Benchmarking | CodeCode Available | 1 |
| SciFaultyQA: Benchmarking LLMs on Faulty Science Question Detection with a GAN-Inspired Approach to Synthetic Dataset Generation | Dec 16, 2024 | BenchmarkingDataset Generation | CodeCode Available | 0 |
| MT-LENS: An all-in-one Toolkit for Better Machine Translation Evaluation | Dec 16, 2024 | AllBenchmarking | CodeCode Available | 1 |
| QUENCH: Measuring the gap between Indic and Non-Indic Contextual General Reasoning in LLMs | Dec 16, 2024 | BenchmarkingCommon Sense Reasoning | CodeCode Available | 0 |
| PunchBench: Benchmarking MLLMs in Multimodal Punchline Comprehension | Dec 16, 2024 | BenchmarkingImage Captioning | —Unverified | 0 |
| AD-LLM: Benchmarking Large Language Models for Anomaly Detection | Dec 15, 2024 | Anomaly DetectionBenchmarking | CodeCode Available | 1 |
| RoLargeSum: A Large Dialect-Aware Romanian News Dataset for Summary, Headline, and Keyword Generation | Dec 15, 2024 | ArticlesBenchmarking | CodeCode Available | 0 |
| Sequence-Level Leakage Risk of Training Data in Large Language Models | Dec 15, 2024 | Benchmarking | —Unverified | 0 |
| Benchmarking and Learning Multi-Dimensional Quality Evaluator for Text-to-3D Generation | Dec 15, 2024 | 3D GenerationBenchmarking | —Unverified | 0 |
| NoisyEQA: Benchmarking Embodied Question Answering Against Noisy Queries | Dec 14, 2024 | BenchmarkingEmbodied Question Answering | —Unverified | 0 |
| NeuralPLexer3: Accurate Biomolecular Complex Structure Prediction with Flow Models | Dec 14, 2024 | BenchmarkingDrug Design | CodeCode Available | 2 |
| EvalGIM: A Library for Evaluating Generative Image Models | Dec 13, 2024 | BenchmarkingDiversity | CodeCode Available | 2 |
| CRS Arena: Crowdsourced Benchmarking of Conversational Recommender Systems | Dec 13, 2024 | BenchmarkingRecommendation Systems | —Unverified | 0 |
| Benchmarking Table Comprehension In The Wild | Dec 13, 2024 | BenchmarkingQuestion Answering | —Unverified | 0 |
| Benchmarking Linguistic Diversity of Large Language Models | Dec 13, 2024 | BenchmarkingDiversity | CodeCode Available | 0 |
| Benchmarking large language models for materials synthesis: the case of atomic layer deposition | Dec 13, 2024 | BenchmarkingHallucination | —Unverified | 0 |