SOTAVerified

Code Completion

Papers

Showing 8190 of 212 papers

TitleStatusHype
Comparative Analysis of Large Language Models for Context-Aware Code Completion using SAFIM Framework0
Coeditor: Leveraging Contextual Changes for Multi-round Code Auto-editing0
Beyond Autocomplete: Designing CopilotLens Towards Transparent and Explainable AI Coding Agents0
Better Context Makes Better Code Language Models: A Case Study on Function Call Argument Completion0
A Review of Repository Level Prompting for LLMs0
CodeMixBench: Evaluating Large Language Models on Code Generation with Code-Mixed Prompts0
Benchmarking Causal Study to Interpret Large Language Models for Source Code0
Insights from the Usage of the Ansible Lightspeed Code Completion Service0
Identifying and Mitigating the Security Risks of Generative AI0
Identifying and Mitigating Vulnerabilities in LLM-Integrated Applications0
Show:102550
← PrevPage 9 of 22Next →

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1deepseek-coder-33b-baseAverage69.01Unverified
2deepseek-coder-6.7b-baseAverage63.4Unverified
3starcoderbaseAverage55.54Unverified
4gpt-4-1106-previewAverage53.28Unverified
5CodeLlama-13b-hfAverage52.78Unverified
6deepseek-coder-1.3b-baseAverage52.63Unverified
7CodeLlama-34b-hfAverage49.66Unverified
8CodeLlama-7b-hfAverage45Unverified
9gpt-3.5-turbo-0301Average40.86Unverified
10incoder-6BAverage33.79Unverified
#ModelMetricClaimedVerifiedStatus
1CodeGPT-adaptedAccuracy (token-level)77.13Unverified
2CodeT5+ 770MEM (line-level)37.9Unverified
3CodeT5+ 220MEM (line-level)35.17Unverified
#ModelMetricClaimedVerifiedStatus
1CodeGPT-adaptedAccuracy (token-level)75.11Unverified
2CodeT5+ 770MEM (line-level)44.86Unverified
3CodeT5+ 220MEM (line-level)43.42Unverified
#ModelMetricClaimedVerifiedStatus
1SantaCoder-MGDCompilation Rate73.03Unverified
2SantaCoderCompilation Rate59.97Unverified
3SantaCoderCompilation Rate59.79Unverified
#ModelMetricClaimedVerifiedStatus
1RamboCompilation Rate76.47Unverified
2RepoCoderCompilation Rate74.02Unverified
#ModelMetricClaimedVerifiedStatus
1RamboCompilation Rate61.7Unverified
2RepoCoderCompilation Rate58.09Unverified