SOTAVerified

Code Completion

Papers

Showing 176200 of 212 papers

TitleStatusHype
What Do They Capture? -- A Structural Analysis of Pre-Trained Language Models for Source Code0
Examining Zero-Shot Vulnerability Repair with Large Language Models0
Long-Range Modeling of Source Code Files with eWASH: Extended Window Access by Syntax Hierarchy0
Time-Efficient Code Completion Model for the R Programming LanguageCode0
Toward Less Hidden Cost of Code Completion with Acceptance and Ranking Models0
Learning to Complete Code with Sketches0
A Syntax-Guided Edit Decoder for Neural Program RepairCode1
Energy-Based Models for Code Generation under Compilability ConstraintsCode1
Learning to Extend Program Graphs to Work-in-Progress Code0
Improving Code Autocompletion with Transfer Learning0
Code Completion by Modeling Flattened Abstract Syntax Trees as Graphs0
CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and GenerationCode1
Neural Software AnalysisCode0
A Simple Approach for Handling Out-of-Vocabulary Identifiers in Deep Learning for Source CodeCode1
On the Embeddings of Variables in Recurrent Neural Networks for Source CodeCode0
Learning to Execute Programs with Instruction Pointer Attention Graph Neural NetworksCode0
Empirical Study of Transformers for Source CodeCode1
Towards Full-line Code Completion with Neural Language Models0
GraphCodeBERT: Pre-training Code Representations with Data Flow0
You Autocomplete Me: Poisoning Vulnerabilities in Neural Code Completion0
IntelliCode Compose: Code Generation Using Transformer0
LambdaNet: Probabilistic Type Inference using Graph Neural NetworksCode1
Fast and Memory-Efficient Neural Code Completion0
Code Completion using Neural Attention and Byte Pair Encoding0
Sequence Model Design for Code Completion in the Modern IDE0
Show:102550
← PrevPage 8 of 9Next →

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1deepseek-coder-33b-baseAverage69.01Unverified
2deepseek-coder-6.7b-baseAverage63.4Unverified
3starcoderbaseAverage55.54Unverified
4gpt-4-1106-previewAverage53.28Unverified
5CodeLlama-13b-hfAverage52.78Unverified
6deepseek-coder-1.3b-baseAverage52.63Unverified
7CodeLlama-34b-hfAverage49.66Unverified
8CodeLlama-7b-hfAverage45Unverified
9gpt-3.5-turbo-0301Average40.86Unverified
10incoder-6BAverage33.79Unverified
#ModelMetricClaimedVerifiedStatus
1CodeGPT-adaptedAccuracy (token-level)77.13Unverified
2CodeT5+ 770MEM (line-level)37.9Unverified
3CodeT5+ 220MEM (line-level)35.17Unverified
#ModelMetricClaimedVerifiedStatus
1CodeGPT-adaptedAccuracy (token-level)75.11Unverified
2CodeT5+ 770MEM (line-level)44.86Unverified
3CodeT5+ 220MEM (line-level)43.42Unverified
#ModelMetricClaimedVerifiedStatus
1SantaCoder-MGDCompilation Rate73.03Unverified
2SantaCoderCompilation Rate59.97Unverified
3SantaCoderCompilation Rate59.79Unverified
#ModelMetricClaimedVerifiedStatus
1RamboCompilation Rate76.47Unverified
2RepoCoderCompilation Rate74.02Unverified
#ModelMetricClaimedVerifiedStatus
1RamboCompilation Rate61.7Unverified
2RepoCoderCompilation Rate58.09Unverified