Code Generation

Code Generation is an important field to predict explicit code or program structure from multimodal data sources such as incomplete code, programs in another programming language, natural language descriptions or execution examples. Code Generation tools can assist the development of automatic programming tools to improve programming productivity.

Source: Deep Learning for Source Code Modeling and Generation

Image source: Measuring Coding Challenge Competence With APPS

Papers

Recently Added Most Hyped Most Active Needs Verification Most Verified

Showing 226–250 of 1697 papers

Title	Date	Tasks	Status	Hype
Evaluating Grounded Reasoning by Code-Assisted Large Language Models for Mathematics	Apr 24, 2025	Code GenerationMath	—Unverified	0
Towards Machine-Generated Code for the Resolution of User Intentions	Apr 24, 2025	Code Generation	CodeCode Available	0
Paper2Code: Automating Code Generation from Scientific Papers in Machine Learning	Apr 24, 2025	Code Generation	CodeCode Available	7
High-Fidelity And Complex Test Data Generation For Real-World SQL Code Generation Services	Apr 24, 2025	Code Generation	—Unverified	0
On Developers' Self-Declaration of AI-Generated Code: An Analysis of Practices	Apr 23, 2025	Code Generation	CodeCode Available	0
EduBot -- Can LLMs Solve Personalized Learning and Programming Assignments?	Apr 23, 2025	Code CompletionCode Generation	—Unverified	0
ClarifyCoder: Clarification-Aware Fine-Tuning for Programmatic Problem Solving	Apr 23, 2025	Code GenerationSynthetic Data Generation	—Unverified	0
A Large-scale Class-level Benchmark Dataset for Code Generation with LLMs	Apr 22, 2025	BenchmarkingClass-level Code Generation	—Unverified	0
VeriCoder: Enhancing LLM-Based RTL Code Generation through Functional Correctness Validation	Apr 22, 2025	Code Generation	—Unverified	0
Insights from Verification: Training a Verilog Generation LLM with Reinforcement Learning with Testbench Feedback	Apr 22, 2025	Code GenerationHallucination	—Unverified	0
Evaluating Judges as Evaluators: The JETTS Benchmark of LLM-as-Judges as Test-Time Scaling Evaluators	Apr 21, 2025	Code GenerationInstruction Following	CodeCode Available	0
Evaluating Code Generation of LLMs in Advanced Computer Science Problems	Apr 21, 2025	Code Generation	—Unverified	0
Empowering AI to Generate Better AI Code: Guided Generation of Deep Learning Projects with LLMs	Apr 21, 2025	Code GenerationDeep Learning	—Unverified	0
LeetCodeDataset: A Temporal Dataset for Robust Evaluation and Efficient Training of Code LLMs	Apr 20, 2025	Code Generation	CodeCode Available	0
Towards Optimal Circuit Generation: Multi-Agent Collaboration Meets Collective Intelligence	Apr 20, 2025	Code GenerationRetrieval-augmented Generation	CodeCode Available	0
ReasoningV: Efficient Verilog Code Generation with Adaptive Hybrid Reasoning Model	Apr 20, 2025	Code GenerationComputational Efficiency	CodeCode Available	0
Improving RL Exploration for LLM Reasoning through Retrospective Replay	Apr 19, 2025	Code GenerationMathematical Reasoning	—Unverified	0
Towards End-to-End Network Intent Management with Large Language Models	Apr 18, 2025	Code GenerationManagement	—Unverified	0
Do Prompt Patterns Affect Code Quality? A First Empirical Assessment of ChatGPT-Generated Code	Apr 18, 2025	Code GenerationPrompt Engineering	—Unverified	0
CodeVisionary: An Agent-based Framework for Evaluating Large Language Models in Code Generation	Apr 18, 2025	Code Generation	—Unverified	0
Chinese-Vicuna: A Chinese Instruction-following Llama-based Model	Apr 17, 2025	Code GenerationCPU	CodeCode Available	7
Syntactic and Semantic Control of Large Language Models via Sequential Monte Carlo	Apr 17, 2025	Code GenerationProbabilistic Programming	—Unverified	0
RoboTwin: Dual-Arm Robot Benchmark with Generative Digital Twins	Apr 17, 2025	Code Generation	CodeCode Available	0
Code Copycat Conundrum: Demystifying Repetition in LLM-based Code Generation	Apr 17, 2025	Code Generation	—Unverified	0
Data-efficient LLM Fine-tuning for Code Generation	Apr 17, 2025	Code GenerationGPU	CodeCode Available	1

Show:10 25 50

← PrevPage 10 of 68Next →

All datasets MBPP APPS CoNaLa Django WikiSQL RES-Q CodeContests HumanEval PECC WebApp1K-React CoNaLa-Ext WebApp1k-Duo-React

Benchmark Results

#	Model	Metric	Claimed	Verified	Status
1	EG-CFG (DeepSeek-V3-0324)	Accuracy	96.6	—	Unverified
2	QualityFlow (Sonnet-3.5)	Accuracy	94.2	—	Unverified
3	o1-mini + MapCoder (Hamming.ai)	Accuracy	93.2	—	Unverified
4	MGDebugger (DeepSeek-V3-0324)	Accuracy	92.4	—	Unverified
5	GPT-4 + AgentCoder	Accuracy	91.8	—	Unverified
6	CodeSim (GPT4o)	Accuracy	90.7	—	Unverified
7	Jiutian-大模型	Accuracy	90	—	Unverified
8	GPT-3.5 Turbo (ChatGPT) + AgentCoder	Accuracy	89.9	—	Unverified
9	MapCoder (GPT-4o)	Accuracy	89.7	—	Unverified
10	GPT-4 (ChatGPT Plus)	Accuracy	87.5	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	LPW (GPT-4o)	Introductory Pass@1	87.2	—	Unverified
2	MoTCoder-32B-V1.5	Introductory Pass@1	68.44	—	Unverified
3	MoTCoder-7B-V1.5	Introductory Pass@1	54.26	—	Unverified
4	code-davinci-002 175B (CodeT)	Introductory Pass@1	47.3	—	Unverified
5	deepseek-ai/deepseek-coder-6.7b-instruct	Introductory Pass@1	33.8	—	Unverified
6	code-davinci-002 175B	Introductory Pass@1	31.92	—	Unverified
7	CodeChain+WizardCoder-15b	Introductory Pass@1	29.3	—	Unverified
8	WizardCoder-15b	Introductory Pass@1	26.29	—	Unverified
9	CodeSim (GPT4)	Introductory Pass@1	26.04	—	Unverified
10	AlphaCode 1B Filtered from 50000	Competition Pass@any	22	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	PanGu-Coder-FT-I	BLEU	44.32	—	Unverified
2	RoBERTaMarian	BLEU	35.74	—	Unverified
3	MarianCG	BLEU	34.43	—	Unverified
4	TranX + BERT w/mined	BLEU	34.2	—	Unverified
5	BERT + TAE	BLEU	33.41	—	Unverified
6	BERTMarian	BLEU	32.46	—	Unverified
7	External Knowledge With API + Reranking	BLEU	32.26	—	Unverified
8	External Knowledge With API	BLEU	30.69	—	Unverified
9	BART W/ Mined	BLEU	30.55	—	Unverified
10	ELECTRAMarian	BLEU	30.18	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	MarianCG	Accuracy	81.83	—	Unverified
2	BERT + TAE	Accuracy	81.03	—	Unverified
3	TranX + BERT w/mined	Accuracy	81.03	—	Unverified
4	Reranker	Accuracy	80.2	—	Unverified
5	LUKEMarian	Accuracy	78.5	—	Unverified
6	RoBERTaMarian	Accuracy	77.95	—	Unverified
7	BERTMarian	Accuracy	76.68	—	Unverified
8	Tranx	Accuracy	73.7	—	Unverified
9	ELECTRAMarian	Accuracy	65.32	—	Unverified
10	lpn (Ling et al., 2016)	Accuracy	62.3	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	NL2SQL-RULE	Execution Accuracy	89.2	—	Unverified
2	TypeSQL+TC (Yu et al., 2018)+	Execution Accuracy	82.6	—	Unverified
3	Tranx	Execution Accuracy	78.6	—	Unverified
4	STAMP+RL (Sun et al., 2018)+	Execution Accuracy	74.6	—	Unverified
5	STAMP (Sun et al., 2018)+	Execution Accuracy	74.4	—	Unverified
6	TypeSQL (Yu et al., 2018)	Execution Accuracy	73.5	—	Unverified
7	PT-MAML (Huang et al., 2018)	Execution Accuracy	68	—	Unverified
8	Bidirectional Attention for SQL Generation	Execution Accuracy	62.5	—	Unverified
9	Seq2SQL (Zhong et al., 2017)	Execution Accuracy	59.4	—	Unverified
10	Seq2Seq (Zhong et al., 2017)	Execution Accuracy	35.9	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	QurrentOS-coder + Claude 3.5 Sonnet	pass@1	58	—	Unverified
2	QurrentOS-coder + GPT-4o	pass@1	46	—	Unverified
3	QurrentOS-coder + GPT-4 Turbo	pass@1	37	—	Unverified
4	QurrentOS-coder + Claude 3 Opus	pass@1	36	—	Unverified
5	QurrentOS-coder + GPT-4	pass@1	30	—	Unverified
6	QurrentOS-coder + Gemini 1.5 Pro	pass@1	30	—	Unverified
7	QurrentOS-coder + DeepSeek-Coder-V2	pass@1	29	—	Unverified
8	QurrentOS-coder + Llama 3 70b	pass@1	20	—	Unverified
9	QurrentOS-coder + Qwen-72B-Instruct	pass@1	18	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	EG-CFG (DeepSeek-V3-0324)	Test Set pass@1	58.18	—	Unverified
2	LPW (GPT-4o)	Test Set pass@1	34.7	—	Unverified
3	MapCoder (GPT-4)	Test Set pass@1	28.5	—	Unverified
4	CodeSim (GPT4)	Test Set pass@1	28.4	—	Unverified
5	MoTCoder-15B	Test Set pass@1	26.34	—	Unverified
6	MoTCoder-7B-v1.5	Test Set pass@1	20.77	—	Unverified
7	CodeChain + WizardCoder-15B	Test Set pass@1	2.35	—	Unverified
8	WizardCoder-15B	Test Set pass@1	1.11	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	DeepSeek-R1 (MGDebugger)	Pass@1	100	—	Unverified
2	LLaMA 3	Pass@1	99.4	—	Unverified
3	QualityFlow (Sonnet-3.5)	Pass@1	98.8	—	Unverified
4	Phi-2	Pass@1	98.2	—	Unverified
5	EG-CFG (DeepSeek-V3-0324)	Pass@1	96.95	—	Unverified
6	Mistral 7B	Pass@1	93.9	—	Unverified
7	Claude Sonnet 3.5	Pass@1	90.85	—	Unverified
8	L2MAC (GPT-4)	Pass@1	90.2	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Claude 3 Haiku	Pass@3	27.67	—	Unverified
2	GPT-3.5 Turbo	Pass@3	23.75	—	Unverified
3	codechat-bison	Pass@3	11.39	—	Unverified
4	chat-bison	Pass@3	8.48	—	Unverified
5	Mixtral-8x7B-Instruct	Pass@3	8.35	—	Unverified
6	Phi-3-mini-128k-instruct	Pass@3	7.18	—	Unverified
7	WizardLM-2-7B	Pass@3	3.72	—	Unverified
8	Llama-3-8B-Instruct	Pass@3	3.1	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	o1-preview	pass@1	0.95	—	Unverified
2	o1-mini	pass@1	0.94	—	Unverified
3	gpt-4o-2024-08-06	pass@1	0.89	—	Unverified
4	claude-3.5-sonnet	pass@1	0.88	—	Unverified
5	deepseek-v2.5	pass@1	0.83	—	Unverified
6	mistral-large-2	pass@1	0.78	—	Unverified
7	deepseek-coder-v2-instruct	pass@1	0.7	—	Unverified
8	llama-v3p1-405b-instruct	pass@1	0.3	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	BART W/ Mined	BLEU	35.32	—	Unverified
2	BART Base	BLEU	34.35	—	Unverified
3	External Knowledge With API + Reranking	BLEU	20.54	—	Unverified
4	External Knowledge With API	BLEU	20.37	—	Unverified
5	Reranker	BLEU	19.85	—	Unverified
6	TranX	BLEU	18.85	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	claude-3-5-sonnet	pass@1	0.68	—	Unverified
2	o1-mini	pass@1	0.67	—	Unverified