| CODESIM: Multi-Agent Code Generation and Problem Solving through Simulation-Driven Planning and Debugging | Feb 8, 2025 | Code GenerationHumanEval | CodeCode Available | 2 |
| Reasoning-as-Logic-Units: Scaling Test-Time Reasoning in Large Language Models Through Logic Unit Alignment | Feb 5, 2025 | GSM8KHumanEval | —Unverified | 0 |
| Large Language Model Guided Self-Debugging Code Generation | Feb 5, 2025 | Code GenerationComputational Efficiency | —Unverified | 0 |
| Learning to Generate Unit Tests for Automated Debugging | Feb 3, 2025 | HumanEvalLarge Language Model | CodeCode Available | 1 |
| ACECODER: Acing Coder RL via Automated Test-Case Synthesis | Feb 3, 2025 | HumanEvalmbpp | —Unverified | 0 |
| Importing Phantoms: Measuring LLM Package Hallucination Vulnerabilities | Jan 31, 2025 | Code GenerationHallucination | —Unverified | 0 |
| How to Select Datapoints for Efficient Human Evaluation of NLG Models? | Jan 30, 2025 | HumanEvalMachine Translation | CodeCode Available | 1 |
| CoCoNUT: Structural Code Understanding does not fall out of a tree | Jan 27, 2025 | Code GenerationHumanEval | CodeCode Available | 0 |
| QualityFlow: An Agentic Workflow for Program Synthesis Controlled by LLM Quality Checks | Jan 20, 2025 | Code GenerationHumanEval | —Unverified | 0 |
| MyGO Multiplex CoT: A Method for Self-Reflection in Large Language Models via Double Chain of Thought Thinking | Jan 20, 2025 | Decision MakingGSM8K | CodeCode Available | 1 |
| Leveraging Metamemory Mechanisms for Enhanced Data-Free Code Generation in LLMs | Jan 14, 2025 | Code GenerationHumanEval | —Unverified | 0 |
| Guided Code Generation with LLMs: A Multi-Agent Framework for Complex Code Tasks | Jan 11, 2025 | Code GenerationHumanEval | —Unverified | 0 |
| Dafny as Verification-Aware Intermediate Language for Code Generation | Jan 10, 2025 | Code GenerationHumanEval | —Unverified | 0 |
| InfiFusion: A Unified Framework for Enhanced Cross-Model Reasoning via LLM Fusion | Jan 6, 2025 | GSM8KHumanEval | —Unverified | 0 |
| Dynamic Scaling of Unit Tests for Code Reward Modeling | Jan 2, 2025 | Code GenerationHumanEval | —Unverified | 0 |
| Thinking Before Running! Efficient Code Generation with Thorough Exploration and Optimal Refinement | Dec 30, 2024 | Code GenerationHumanEval | —Unverified | 0 |
| HumanEval Pro and MBPP Pro: Evaluating Large Language Models on Self-invoking Code Generation | Dec 30, 2024 | Code GenerationHumanEval | CodeCode Available | 1 |
| SecBench: A Comprehensive Multi-Dimensional Benchmarking Dataset for LLMs in Cybersecurity | Dec 30, 2024 | BenchmarkingCode Generation | —Unverified | 0 |
| Dovetail: A CPU/GPU Heterogeneous Speculative Decoding for LLM inference | Dec 25, 2024 | CPUGPU | —Unverified | 0 |
| Inference-Aware Fine-Tuning for Best-of-N Sampling in Large Language Models | Dec 18, 2024 | HumanEvalImitation Learning | —Unverified | 0 |
| PERC: Plan-As-Query Example Retrieval for Underrepresented Code Generation | Dec 17, 2024 | Code GenerationHumanEval | —Unverified | 0 |
| Falcon: Faster and Parallel Inference of Large Language Models through Enhanced Semi-Autoregressive Drafting and Custom-Designed Decoding Tree | Dec 17, 2024 | GSM8KHumanEval | —Unverified | 0 |
| Learning to Reason via Self-Iterative Process Feedback for Small Language Models | Dec 11, 2024 | Domain GeneralizationGSM8K | —Unverified | 0 |
| AlphaVerus: Bootstrapping Formally Verified Code Generation through Self-Improving Translation and Treefinement | Dec 9, 2024 | Code GenerationHumanEval | —Unverified | 0 |
| Does Few-Shot Learning Help LLM Performance in Code Synthesis? | Dec 3, 2024 | Code GenerationFew-Shot Learning | —Unverified | 0 |
| Addressing Data Leakage in HumanEval Using Combinatorial Test Design | Dec 2, 2024 | HumanEval | —Unverified | 0 |
| Inference Scaling fLaws: The Limits of LLM Resampling with Imperfect Verifiers | Nov 26, 2024 | HumanEvalmbpp | CodeCode Available | 0 |
| A Preliminary Study of Multilingual Code Language Models for Code Generation Task Using Translated Benchmarks | Nov 23, 2024 | Code GenerationHumanEval | —Unverified | 0 |
| Planning-Driven Programming: A Large Language Model Programming Workflow | Nov 21, 2024 | Code GenerationHumanEval | CodeCode Available | 1 |
| DSTC: Direct Preference Learning with Only Self-Generated Tests and Code to Improve Code LMs | Nov 20, 2024 | Code GenerationHumanEval | —Unverified | 0 |
| PerfCodeGen: Improving Performance of LLM Generated Code with Execution Feedback | Nov 18, 2024 | HumanEvalmbpp | CodeCode Available | 1 |
| VALTEST: Automated Validation of Language Model Generated Test Cases | Nov 13, 2024 | HumanEvalLanguage Modeling | —Unverified | 0 |
| Synthesize, Partition, then Adapt: Eliciting Diverse Samples from Foundation Models | Nov 11, 2024 | Code GenerationHumanEval | —Unverified | 0 |
| CodeTree: Agent-guided Tree Search for Code Generation with Large Language Models | Nov 7, 2024 | Code GenerationDecision Making | —Unverified | 0 |
| InterTrans: Leveraging Transitive Intermediate Translations to Enhance LLM-based Code Translation | Nov 1, 2024 | Code TranslationHumanEval | CodeCode Available | 0 |
| SelfCodeAlign: Self-Alignment for Code Generation | Oct 31, 2024 | Code GenerationHumanEval | CodeCode Available | 3 |
| Demo-Craft: Using In-Context Learning to Improve Code Generation in Large Language Models | Oct 30, 2024 | Code GenerationHumanEval | —Unverified | 0 |
| Can Language Models Replace Programmers for Coding? REPOCOD Says 'Not Yet' | Oct 29, 2024 | Code CompletionCode Generation | CodeCode Available | 1 |
| FALCON: Feedback-driven Adaptive Long/short-term memory reinforced Coding Optimization system | Oct 28, 2024 | Code GenerationHumanEval | CodeCode Available | 0 |
| Aligning CodeLLMs with Direct Preference Optimization | Oct 24, 2024 | Decision MakingHumanEval | —Unverified | 0 |
| Adaptive Dense Reward: Understanding the Gap Between Action and Reward Space in Alignment | Oct 23, 2024 | GSM8KHumanEval | —Unverified | 0 |
| MojoBench: Language Modeling and Benchmarks for Mojo | Oct 23, 2024 | Code GenerationHumanEval | —Unverified | 0 |
| Scattered Forest Search: Smarter Code Space Exploration with LLMs | Oct 22, 2024 | Code GenerationDiversity | —Unverified | 0 |
| Self-Evolving Multi-Agent Collaboration Networks for Software Development | Oct 22, 2024 | HumanEval | —Unverified | 0 |
| Semantic-guided Search for Efficient Program Repair with Large Language Models | Oct 22, 2024 | GPUHumanEval | —Unverified | 0 |
| Self-Explained Keywords Empower Large Language Models for Code Generation | Oct 21, 2024 | Code GenerationHumanEval | —Unverified | 0 |
| mHumanEval -- A Multilingual Benchmark to Evaluate Large Language Models for Code Generation | Oct 19, 2024 | Code GenerationDiversity | CodeCode Available | 0 |
| CELI: Controller-Embedded Language Model Interactions | Oct 18, 2024 | ArticlesCode Generation | —Unverified | 0 |
| HumanEval-V: Evaluating Visual Understanding and Reasoning Abilities of Large Multimodal Models Through Coding Tasks | Oct 16, 2024 | Code GenerationHumanEval | CodeCode Available | 1 |
| G-Designer: Architecting Multi-agent Communication Topologies via Graph Neural Networks | Oct 15, 2024 | HumanEvalLanguage Modelling | —Unverified | 0 |