SOTAVerified

Code Generation

Code Generation is an important field to predict explicit code or program structure from multimodal data sources such as incomplete code, programs in another programming language, natural language descriptions or execution examples. Code Generation tools can assist the development of automatic programming tools to improve programming productivity.

Source: Deep Learning for Source Code Modeling and Generation

Image source: Measuring Coding Challenge Competence With APPS

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 251–300 of 1697 papers

TitleStatusHype
Rethinking the Generation of High-Quality CoT Data from the Perspective of LLM-Adaptive Question Difficulty Grading—0
Themisto: Jupyter-Based Runtime Benchmark—0
A Dual-Space Framework for General Knowledge Distillation of Large Language ModelsCode1
The Future of MLLM Prompting is Adaptive: A Comprehensive Experimental Evaluation of Prompt Engineering Methods for Robust Multimodal Performance—0
Draw with Thought: Unleashing Multimodal Reasoning for Scientific Diagram Generation—0
Iterative Self-Training for Code Generation via Reinforced Re-Ranking—0
Type-Constrained Code Generation with Language Models—0
LSR-MCTS: Alleviating Long Range Dependency in Code Generation—0
Boosting Universal LLM Reward Design through the Heuristic Reward Observation Space Evolution—0
PR-Attack: Coordinated Prompt-RAG Attacks on Retrieval-Augmented Generation in Large Language Models via Bilevel Optimization—0
LoRI: Reducing Cross-Task Interference in Multi-Task Low-Rank AdaptationCode2
MDIT: A Model-free Data Interpolation Method for Diverse Instruction Tuning—0
Prism: Dynamic and Flexible Benchmarking of LLMs Code Generation with Monte Carlo Tree Search—0
How Accurately Do Large Language Models Understand Code?—0
DDPT: Diffusion-Driven Prompt Tuning for Large Language Model Code Generation—0
OpenCodeInstruct: A Large-scale Instruction Tuning Dataset for Code LLMs—0
Do LLM Evaluators Prefer Themselves for a Reason?Code0
NAACL2025 Tutorial: Adaptation of Large Language Models—0
Pel, A Programming Language for Orchestrating AI Agents—0
Enhancing Chart-to-Code Generation in Multimodal Large Language Models via Iterative Dual Preference LearningCode0
The Self-Learning Agent with a Progressive Neural Network Integrated Transformer—0
MG-Gen: Single Image to Motion Graphics Generation with Layer Decomposition—0
On Simulation-Guided LLM-based Code Generation for Safe Autonomous Driving Software—0
From Code Generation to Software Testing: AI Copilot with Context-Based RAG—0
How Difficulty-Aware Staged Reinforcement Learning Enhances LLMs' Reasoning Capabilities: A Preliminary Experimental Study—0
SRLCG: Self-Rectified Large-Scale Code Generation with Multidimensional Chain-of-Thought and Dynamic Backtracking—0
TuRTLe: A Unified Evaluation of LLMs for RTL GenerationCode1
MaintainCoder: Maintainable Code Generation Under Dynamic RequirementsCode1
Rubric Is All You Need: Enhancing LLM-based Code Evaluation With Question-Specific Rubrics—0
Carbon Footprint Evaluation of Code Generation through LLM as a Service—0
CCCI: Code Completion with Contextual Information for Complex Data Transfer Tasks Using Large Language Models—0
RobuNFR: Evaluating the Robustness of Large Language Models on Non-Functional Requirements Aware Code Generation—0
Challenges and Paths Towards AI for Software Engineering—0
Why Stop at One Error? Benchmarking LLMs as Data Science Code Debuggers for Multi-Hop and Multi-Bug ErrorsCode0
ObscuraCoder: Powering Efficient Code LM Pre-Training Via Obfuscation GroundingCode0
Malicious and Unintentional Disclosure Risks in Large Language Models for Code Generation—0
Unlocking the Potential of Past Research: Using Generative AI to Reconstruct Healthcare Simulation Models—0
Optimizing Language Models for Inference Time Objectives using Reinforcement Learning—0
VecTrans: Enhancing Compiler Auto-Vectorization through LLM-Assisted Code Transformations—0
AssertionForge: Enhancing Formal Verification Assertion Generation with Structured Representation of Specifications and RTL—0
Verbal Process Supervision Elicits Better Coding Agents—0
ModiGen: A Large Language Model-Based Workflow for Multi-Task Modelica Code Generation—0
Every Sample Matters: Leveraging Mixture-of-Experts and High-Quality Data for Efficient and Accurate Code LLM—0
A Study on the Improvement of Code Generation Quality Using Large Language Models Leveraging Product Documentation—0
LaMOuR: Leveraging Language Models for Out-of-Distribution Recovery in Reinforcement Learning—0
LLMs Love Python: A Study of LLMs' Bias for Programming Languages and Libraries—0
RustEvo^2: An Evolving Benchmark for API Evolution in LLM-based Rust Code GenerationCode0
On Explaining (Large) Language Models For Code Using Global Code-Based Explanations—0
CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models—0
Aligning Crowd-sourced Human Feedback for Reinforcement Learning on Code Generation by Large Language Models—0
Show:102550
← PrevPage 6 of 34Next →
All datasetsMBPPAPPSCoNaLaDjangoWikiSQLRES-QCodeContestsHumanEvalPECCWebApp1K-ReactCoNaLa-ExtWebApp1k-Duo-React

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1EG-CFG (DeepSeek-V3-0324)Accuracy96.6—Unverified
2QualityFlow (Sonnet-3.5)Accuracy94.2—Unverified
3o1-mini + MapCoder (Hamming.ai)Accuracy93.2—Unverified
4MGDebugger (DeepSeek-V3-0324)Accuracy92.4—Unverified
5GPT-4 + AgentCoderAccuracy91.8—Unverified
6CodeSim (GPT4o)Accuracy90.7—Unverified
7Jiutian-大模型Accuracy90—Unverified
8GPT-3.5 Turbo (ChatGPT) + AgentCoderAccuracy89.9—Unverified
9MapCoder (GPT-4o)Accuracy89.7—Unverified
10GPT-4 (ChatGPT Plus)Accuracy87.5—Unverified
#ModelMetricClaimedVerifiedStatus
1LPW (GPT-4o)Introductory Pass@187.2—Unverified
2MoTCoder-32B-V1.5Introductory Pass@168.44—Unverified
3MoTCoder-7B-V1.5Introductory Pass@154.26—Unverified
4code-davinci-002 175B (CodeT)Introductory Pass@147.3—Unverified
5deepseek-ai/deepseek-coder-6.7b-instructIntroductory Pass@133.8—Unverified
6code-davinci-002 175BIntroductory Pass@131.92—Unverified
7CodeChain+WizardCoder-15bIntroductory Pass@129.3—Unverified
8WizardCoder-15bIntroductory Pass@126.29—Unverified
9CodeSim (GPT4)Introductory Pass@126.04—Unverified
10AlphaCode 1B Filtered from 50000Competition Pass@any22—Unverified
#ModelMetricClaimedVerifiedStatus
1PanGu-Coder-FT-IBLEU44.32—Unverified
2RoBERTaMarianBLEU35.74—Unverified
3MarianCGBLEU34.43—Unverified
4TranX + BERT w/minedBLEU34.2—Unverified
5BERT + TAEBLEU33.41—Unverified
6BERTMarianBLEU32.46—Unverified
7External Knowledge With API + RerankingBLEU32.26—Unverified
8External Knowledge With APIBLEU30.69—Unverified
9BART W/ MinedBLEU30.55—Unverified
10ELECTRAMarianBLEU30.18—Unverified
#ModelMetricClaimedVerifiedStatus
1MarianCGAccuracy81.83—Unverified
2BERT + TAEAccuracy81.03—Unverified
3TranX + BERT w/minedAccuracy81.03—Unverified
4RerankerAccuracy80.2—Unverified
5LUKEMarianAccuracy78.5—Unverified
6RoBERTaMarianAccuracy77.95—Unverified
7BERTMarianAccuracy76.68—Unverified
8TranxAccuracy73.7—Unverified
9ELECTRAMarianAccuracy65.32—Unverified
10lpn (Ling et al., 2016)Accuracy62.3—Unverified
#ModelMetricClaimedVerifiedStatus
1NL2SQL-RULEExecution Accuracy89.2—Unverified
2TypeSQL+TC (Yu et al., 2018)+Execution Accuracy82.6—Unverified
3TranxExecution Accuracy78.6—Unverified
4STAMP+RL (Sun et al., 2018)+Execution Accuracy74.6—Unverified
5STAMP (Sun et al., 2018)+Execution Accuracy74.4—Unverified
6TypeSQL (Yu et al., 2018)Execution Accuracy73.5—Unverified
7PT-MAML (Huang et al., 2018)Execution Accuracy68—Unverified
8Bidirectional Attention for SQL GenerationExecution Accuracy62.5—Unverified
9Seq2SQL (Zhong et al., 2017)Execution Accuracy59.4—Unverified
10Seq2Seq (Zhong et al., 2017)Execution Accuracy35.9—Unverified
#ModelMetricClaimedVerifiedStatus
1QurrentOS-coder + Claude 3.5 Sonnetpass@158—Unverified
2QurrentOS-coder + GPT-4opass@146—Unverified
3QurrentOS-coder + GPT-4 Turbopass@137—Unverified
4QurrentOS-coder + Claude 3 Opuspass@136—Unverified
5QurrentOS-coder + GPT-4pass@130—Unverified
6QurrentOS-coder + Gemini 1.5 Propass@130—Unverified
7QurrentOS-coder + DeepSeek-Coder-V2pass@129—Unverified
8QurrentOS-coder + Llama 3 70bpass@120—Unverified
9QurrentOS-coder + Qwen-72B-Instructpass@118—Unverified
#ModelMetricClaimedVerifiedStatus
1EG-CFG (DeepSeek-V3-0324)Test Set pass@158.18—Unverified
2LPW (GPT-4o)Test Set pass@134.7—Unverified
3MapCoder (GPT-4)Test Set pass@128.5—Unverified
4CodeSim (GPT4)Test Set pass@128.4—Unverified
5MoTCoder-15BTest Set pass@126.34—Unverified
6MoTCoder-7B-v1.5Test Set pass@120.77—Unverified
7CodeChain + WizardCoder-15BTest Set pass@12.35—Unverified
8WizardCoder-15BTest Set pass@11.11—Unverified
#ModelMetricClaimedVerifiedStatus
1DeepSeek-R1 (MGDebugger)Pass@1100—Unverified
2LLaMA 3Pass@199.4—Unverified
3QualityFlow (Sonnet-3.5)Pass@198.8—Unverified
4Phi-2Pass@198.2—Unverified
5EG-CFG (DeepSeek-V3-0324)Pass@196.95—Unverified
6Mistral 7BPass@193.9—Unverified
7Claude Sonnet 3.5Pass@190.85—Unverified
8L2MAC (GPT-4)Pass@190.2—Unverified
#ModelMetricClaimedVerifiedStatus
1Claude 3 HaikuPass@327.67—Unverified
2GPT-3.5 TurboPass@323.75—Unverified
3codechat-bisonPass@311.39—Unverified
4chat-bisonPass@38.48—Unverified
5Mixtral-8x7B-InstructPass@38.35—Unverified
6Phi-3-mini-128k-instructPass@37.18—Unverified
7WizardLM-2-7BPass@33.72—Unverified
8Llama-3-8B-InstructPass@33.1—Unverified
#ModelMetricClaimedVerifiedStatus
1o1-previewpass@10.95—Unverified
2o1-minipass@10.94—Unverified
3gpt-4o-2024-08-06pass@10.89—Unverified
4claude-3.5-sonnetpass@10.88—Unverified
5deepseek-v2.5pass@10.83—Unverified
6mistral-large-2pass@10.78—Unverified
7deepseek-coder-v2-instructpass@10.7—Unverified
8llama-v3p1-405b-instructpass@10.3—Unverified
#ModelMetricClaimedVerifiedStatus
1BART W/ MinedBLEU35.32—Unverified
2BART BaseBLEU34.35—Unverified
3External Knowledge With API + RerankingBLEU20.54—Unverified
4External Knowledge With APIBLEU20.37—Unverified
5RerankerBLEU19.85—Unverified
6TranXBLEU18.85—Unverified
#ModelMetricClaimedVerifiedStatus
1claude-3-5-sonnetpass@10.68—Unverified
2o1-minipass@10.67—Unverified