SOTAVerified

Code Generation

Code Generation is an important field to predict explicit code or program structure from multimodal data sources such as incomplete code, programs in another programming language, natural language descriptions or execution examples. Code Generation tools can assist the development of automatic programming tools to improve programming productivity.

Source: Deep Learning for Source Code Modeling and Generation

Image source: Measuring Coding Challenge Competence With APPS

Papers

Showing 110 of 1697 papers

TitleStatusHype
CUDA-L1: Improving CUDA Optimization via Contrastive Reinforcement Learning0
Towards Formal Verification of LLM-Generated Code from Natural Language Prompts0
MERA Code: A Unified Framework for Evaluating Code Generation Across Tasks0
Scaling Up RL: Unlocking Diverse Reasoning in LLMs via Prolonged Training0
The Devil behind the mask: An emergent safety vulnerability of Diffusion LLMsCode2
Turning the Tide: Repository-based Code Reflection0
CodeAssistBench (CAB): Dataset & Benchmarking for Multi-turn Chat-Based Code Assistance0
CodeJudgeBench: Benchmarking LLM-as-a-Judge for Coding Tasks0
Kodezi Chronos: A Debugging-First Language Model for Repository-Scale, Memory-Driven Code UnderstandingCode9
Multilingual Multimodal Software Developer for Code Generation0
Show:102550
← PrevPage 1 of 170Next →

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1NL2SQL-RULEExecution Accuracy89.2Unverified
2TypeSQL+TC (Yu et al., 2018)+Execution Accuracy82.6Unverified
3TranxExecution Accuracy78.6Unverified
4STAMP+RL (Sun et al., 2018)+Execution Accuracy74.6Unverified
5STAMP (Sun et al., 2018)+Execution Accuracy74.4Unverified
6TypeSQL (Yu et al., 2018)Execution Accuracy73.5Unverified
7PT-MAML (Huang et al., 2018)Execution Accuracy68Unverified
8Bidirectional Attention for SQL GenerationExecution Accuracy62.5Unverified
9Seq2SQL (Zhong et al., 2017)Execution Accuracy59.4Unverified
10Seq2Seq (Zhong et al., 2017)Execution Accuracy35.9Unverified