SOTAVerified

Code Completion

Papers

Showing 8190 of 212 papers

TitleStatusHype
GenCodeSearchNet: A Benchmark Test Suite for Evaluating Generalization in Programming Language UnderstandingCode0
Code Completion with Neural Attention and Pointer NetworksCode0
On the Embeddings of Variables in Recurrent Neural Networks for Source CodeCode0
MERGE: Fast Private Text GenerationCode0
Don't Complete It! Preventing Unhelpful Code Completion for Productive and Sustainable Neural Code Completion SystemsCode0
Learning to Execute Programs with Instruction Pointer Attention Graph Neural NetworksCode0
Neural Software AnalysisCode0
Enriching Source Code with Contextual Data for Code Completion Models: An Empirical StudyCode0
Large Language Models of Code Fail at Completing Code with Potential BugsCode0
Enhancing High-Quality Code Generation in Large Language Models with Comparative Prefix-TuningCode0
Show:102550
← PrevPage 9 of 22Next →

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1deepseek-coder-33b-baseAverage69.01Unverified
2deepseek-coder-6.7b-baseAverage63.4Unverified
3starcoderbaseAverage55.54Unverified
4gpt-4-1106-previewAverage53.28Unverified
5CodeLlama-13b-hfAverage52.78Unverified
6deepseek-coder-1.3b-baseAverage52.63Unverified
7CodeLlama-34b-hfAverage49.66Unverified
8CodeLlama-7b-hfAverage45Unverified
9gpt-3.5-turbo-0301Average40.86Unverified
10incoder-6BAverage33.79Unverified
#ModelMetricClaimedVerifiedStatus
1CodeGPT-adaptedAccuracy (token-level)77.13Unverified
2CodeT5+ 770MEM (line-level)37.9Unverified
3CodeT5+ 220MEM (line-level)35.17Unverified
#ModelMetricClaimedVerifiedStatus
1CodeGPT-adaptedAccuracy (token-level)75.11Unverified
2CodeT5+ 770MEM (line-level)44.86Unverified
3CodeT5+ 220MEM (line-level)43.42Unverified
#ModelMetricClaimedVerifiedStatus
1SantaCoder-MGDCompilation Rate73.03Unverified
2SantaCoderCompilation Rate59.97Unverified
3SantaCoderCompilation Rate59.79Unverified
#ModelMetricClaimedVerifiedStatus
1RamboCompilation Rate76.47Unverified
2RepoCoderCompilation Rate74.02Unverified
#ModelMetricClaimedVerifiedStatus
1RamboCompilation Rate61.7Unverified
2RepoCoderCompilation Rate58.09Unverified