Code Generation

Code Generation is an important field to predict explicit code or program structure from multimodal data sources such as incomplete code, programs in another programming language, natural language descriptions or execution examples. Code Generation tools can assist the development of automatic programming tools to improve programming productivity.

Source: Deep Learning for Source Code Modeling and Generation

Image source: Measuring Coding Challenge Competence With APPS

Papers

Recently Added Most Hyped Most Active Needs Verification Most Verified

Showing 451–475 of 1697 papers

Title	Date	Tasks	Status	Hype
C codegen considered unnecessary: go directly to binary, do not pass C. Compilation of Julia code for deployment in model-based engineering	Feb 3, 2025	C++ codeCode Generation	CodeCode Available	1
Analysis of LLMs vs Human Experts in Requirements Engineering	Jan 31, 2025	Code Generation	—Unverified	0
Importing Phantoms: Measuring LLM Package Hallucination Vulnerabilities	Jan 31, 2025	Code GenerationHallucination	—Unverified	0
Towards Adaptive Self-Improvement for Smarter Energy Systems	Jan 31, 2025	Code GenerationDecision Making	—Unverified	0
Cogito, ergo sum: A Neurobiologically-Inspired Cognition-Memory-Growth System for Code Generation	Jan 30, 2025	Code GenerationHippocampus	CodeCode Available	0
Enhancing Large Language Model Efficiencyvia Symbolic Compression: A Formal Approach Towards Interpretability	Jan 30, 2025	Code GenerationLanguage Modeling	—Unverified	0
o3-mini vs DeepSeek-R1: Which One is Safer?	Jan 30, 2025	Code GenerationProgram Repair	CodeCode Available	1
Statistical multi-metric evaluation and visualization of LLM system predictive performance	Jan 30, 2025	Code GenerationDecision Making	—Unverified	0
Towards Making Flowchart Images Machine Interpretable	Jan 29, 2025	Code GenerationOptical Character Recognition (OCR)	CodeCode Available	1
Using Code Generation to Solve Open Instances of Combinatorial Design Problems	Jan 29, 2025	Code Generationvalid	CodeCode Available	0
GLLM: Self-Corrective G-Code Generation using Large Language Models with User Feedback	Jan 29, 2025	Code GenerationRAG	—Unverified	0
Programming by Examples Meets Historical Linguistics: A Large Language Model Based Approach to Sound Law Induction	Jan 27, 2025	Code GenerationInductive Bias	—Unverified	0
CoCoNUT: Structural Code Understanding does not fall out of a tree	Jan 27, 2025	Code GenerationHumanEval	CodeCode Available	0
Advancing Generative Artificial Intelligence and Large Language Models for Demand Side Management with Internet of Electric Vehicles	Jan 26, 2025	Code Generationenergy management	—Unverified	0
ENTER: Event Based Interpretable Reasoning for VideoQA	Jan 24, 2025	Code GenerationEgoSchema	—Unverified	0
Chain of Grounded Objectives: Bridging Process and Goal-oriented Prompting for Code Generation	Jan 23, 2025	Code Generation	—Unverified	0
Pseudocode-Injection Magic: Enabling LLMs to Tackle Graph Computational Tasks	Jan 23, 2025	Code GenerationComputational Efficiency	—Unverified	0
Revisit Self-Debugging with Self-Generated Tests for Code Generation	Jan 22, 2025	Code Generation	—Unverified	0
Correctness Assessment of Code Generated by Large Language Models Using Internal Representations	Jan 22, 2025	Code Generation	CodeCode Available	0
QualityFlow: An Agentic Workflow for Program Synthesis Controlled by LLM Quality Checks	Jan 20, 2025	Code GenerationHumanEval	—Unverified	0
Consolidating TinyML Lifecycle with Large Language Models: Reality, Illusion, or Opportunity?	Jan 20, 2025	Code GenerationModel Optimization	—Unverified	0
Towards Advancing Code Generation with Large Language Models: A Research Roadmap	Jan 20, 2025	Code Generation	—Unverified	0
GREEN-CODE: Learning to Optimize Energy Efficiency in LLM-based Code Generation	Jan 19, 2025	Bug fixingCode Completion	CodeCode Available	0
ChaosEater: Fully Automating Chaos Engineering with Large Language Models	Jan 19, 2025	Code Generation	CodeCode Available	1
SOP-Agent: Empower General Purpose AI Agent with Domain-Specific SOPs	Jan 16, 2025	AI AgentCode Generation	—Unverified	0

Show:10 25 50

← PrevPage 19 of 68Next →

All datasets MBPP APPS CoNaLa Django WikiSQL RES-Q CodeContests HumanEval PECC WebApp1K-React CoNaLa-Ext WebApp1k-Duo-React

Benchmark Results

#	Model	Metric	Claimed	Verified	Status
1	EG-CFG (DeepSeek-V3-0324)	Accuracy	96.6	—	Unverified
2	QualityFlow (Sonnet-3.5)	Accuracy	94.2	—	Unverified
3	o1-mini + MapCoder (Hamming.ai)	Accuracy	93.2	—	Unverified
4	MGDebugger (DeepSeek-V3-0324)	Accuracy	92.4	—	Unverified
5	GPT-4 + AgentCoder	Accuracy	91.8	—	Unverified
6	CodeSim (GPT4o)	Accuracy	90.7	—	Unverified
7	Jiutian-大模型	Accuracy	90	—	Unverified
8	GPT-3.5 Turbo (ChatGPT) + AgentCoder	Accuracy	89.9	—	Unverified
9	MapCoder (GPT-4o)	Accuracy	89.7	—	Unverified
10	GPT-4 (ChatGPT Plus)	Accuracy	87.5	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	LPW (GPT-4o)	Introductory Pass@1	87.2	—	Unverified
2	MoTCoder-32B-V1.5	Introductory Pass@1	68.44	—	Unverified
3	MoTCoder-7B-V1.5	Introductory Pass@1	54.26	—	Unverified
4	code-davinci-002 175B (CodeT)	Introductory Pass@1	47.3	—	Unverified
5	deepseek-ai/deepseek-coder-6.7b-instruct	Introductory Pass@1	33.8	—	Unverified
6	code-davinci-002 175B	Introductory Pass@1	31.92	—	Unverified
7	CodeChain+WizardCoder-15b	Introductory Pass@1	29.3	—	Unverified
8	WizardCoder-15b	Introductory Pass@1	26.29	—	Unverified
9	CodeSim (GPT4)	Introductory Pass@1	26.04	—	Unverified
10	AlphaCode 1B Filtered from 50000	Competition Pass@any	22	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	PanGu-Coder-FT-I	BLEU	44.32	—	Unverified
2	RoBERTaMarian	BLEU	35.74	—	Unverified
3	MarianCG	BLEU	34.43	—	Unverified
4	TranX + BERT w/mined	BLEU	34.2	—	Unverified
5	BERT + TAE	BLEU	33.41	—	Unverified
6	BERTMarian	BLEU	32.46	—	Unverified
7	External Knowledge With API + Reranking	BLEU	32.26	—	Unverified
8	External Knowledge With API	BLEU	30.69	—	Unverified
9	BART W/ Mined	BLEU	30.55	—	Unverified
10	ELECTRAMarian	BLEU	30.18	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	MarianCG	Accuracy	81.83	—	Unverified
2	BERT + TAE	Accuracy	81.03	—	Unverified
3	TranX + BERT w/mined	Accuracy	81.03	—	Unverified
4	Reranker	Accuracy	80.2	—	Unverified
5	LUKEMarian	Accuracy	78.5	—	Unverified
6	RoBERTaMarian	Accuracy	77.95	—	Unverified
7	BERTMarian	Accuracy	76.68	—	Unverified
8	Tranx	Accuracy	73.7	—	Unverified
9	ELECTRAMarian	Accuracy	65.32	—	Unverified
10	lpn (Ling et al., 2016)	Accuracy	62.3	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	NL2SQL-RULE	Execution Accuracy	89.2	—	Unverified
2	TypeSQL+TC (Yu et al., 2018)+	Execution Accuracy	82.6	—	Unverified
3	Tranx	Execution Accuracy	78.6	—	Unverified
4	STAMP+RL (Sun et al., 2018)+	Execution Accuracy	74.6	—	Unverified
5	STAMP (Sun et al., 2018)+	Execution Accuracy	74.4	—	Unverified
6	TypeSQL (Yu et al., 2018)	Execution Accuracy	73.5	—	Unverified
7	PT-MAML (Huang et al., 2018)	Execution Accuracy	68	—	Unverified
8	Bidirectional Attention for SQL Generation	Execution Accuracy	62.5	—	Unverified
9	Seq2SQL (Zhong et al., 2017)	Execution Accuracy	59.4	—	Unverified
10	Seq2Seq (Zhong et al., 2017)	Execution Accuracy	35.9	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	QurrentOS-coder + Claude 3.5 Sonnet	pass@1	58	—	Unverified
2	QurrentOS-coder + GPT-4o	pass@1	46	—	Unverified
3	QurrentOS-coder + GPT-4 Turbo	pass@1	37	—	Unverified
4	QurrentOS-coder + Claude 3 Opus	pass@1	36	—	Unverified
5	QurrentOS-coder + Gemini 1.5 Pro	pass@1	30	—	Unverified
6	QurrentOS-coder + GPT-4	pass@1	30	—	Unverified
7	QurrentOS-coder + DeepSeek-Coder-V2	pass@1	29	—	Unverified
8	QurrentOS-coder + Llama 3 70b	pass@1	20	—	Unverified
9	QurrentOS-coder + Qwen-72B-Instruct	pass@1	18	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	EG-CFG (DeepSeek-V3-0324)	Test Set pass@1	58.18	—	Unverified
2	LPW (GPT-4o)	Test Set pass@1	34.7	—	Unverified
3	MapCoder (GPT-4)	Test Set pass@1	28.5	—	Unverified
4	CodeSim (GPT4)	Test Set pass@1	28.4	—	Unverified
5	MoTCoder-15B	Test Set pass@1	26.34	—	Unverified
6	MoTCoder-7B-v1.5	Test Set pass@1	20.77	—	Unverified
7	CodeChain + WizardCoder-15B	Test Set pass@1	2.35	—	Unverified
8	WizardCoder-15B	Test Set pass@1	1.11	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	DeepSeek-R1 (MGDebugger)	Pass@1	100	—	Unverified
2	LLaMA 3	Pass@1	99.4	—	Unverified
3	QualityFlow (Sonnet-3.5)	Pass@1	98.8	—	Unverified
4	Phi-2	Pass@1	98.2	—	Unverified
5	EG-CFG (DeepSeek-V3-0324)	Pass@1	96.95	—	Unverified
6	Mistral 7B	Pass@1	93.9	—	Unverified
7	Claude Sonnet 3.5	Pass@1	90.85	—	Unverified
8	L2MAC (GPT-4)	Pass@1	90.2	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Claude 3 Haiku	Pass@3	27.67	—	Unverified
2	GPT-3.5 Turbo	Pass@3	23.75	—	Unverified
3	codechat-bison	Pass@3	11.39	—	Unverified
4	chat-bison	Pass@3	8.48	—	Unverified
5	Mixtral-8x7B-Instruct	Pass@3	8.35	—	Unverified
6	Phi-3-mini-128k-instruct	Pass@3	7.18	—	Unverified
7	WizardLM-2-7B	Pass@3	3.72	—	Unverified
8	Llama-3-8B-Instruct	Pass@3	3.1	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	o1-preview	pass@1	0.95	—	Unverified
2	o1-mini	pass@1	0.94	—	Unverified
3	gpt-4o-2024-08-06	pass@1	0.89	—	Unverified
4	claude-3.5-sonnet	pass@1	0.88	—	Unverified
5	deepseek-v2.5	pass@1	0.83	—	Unverified
6	mistral-large-2	pass@1	0.78	—	Unverified
7	deepseek-coder-v2-instruct	pass@1	0.7	—	Unverified
8	llama-v3p1-405b-instruct	pass@1	0.3	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	BART W/ Mined	BLEU	35.32	—	Unverified
2	BART Base	BLEU	34.35	—	Unverified
3	External Knowledge With API + Reranking	BLEU	20.54	—	Unverified
4	External Knowledge With API	BLEU	20.37	—	Unverified
5	Reranker	BLEU	19.85	—	Unverified
6	TranX	BLEU	18.85	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	claude-3-5-sonnet	pass@1	0.68	—	Unverified
2	o1-mini	pass@1	0.67	—	Unverified