SOTAVerified

Code Completion

Papers

Showing 201212 of 212 papers

TitleStatusHype
Serenity: Library Based Python Code Analysis for Code Completion and Automated Machine Learning0
Stable Code Technical Report0
Statically Contextualizing Large Language Models with Typed Holes0
Structure-Aware Corpus Construction and User-Perception-Aligned Metrics for Large-Language-Model Code Completion0
TPIA: Towards Target-specific Prompt Injection Attack against Code-oriented Large Language Models0
TASTY: A Transformer based Approach to Space and Time complexity0
Text and Patterns: For Effective Chain of Thought, It Takes Two to Tango0
Token Alignment via Character Matching for Subword Completion0
Toward Less Hidden Cost of Code Completion with Acceptance and Ranking Models0
Towards Full-line Code Completion with Neural Language Models0
User-Interactive Machine Learning Model for Identifying Structural Relationships of Code Features0
TaskEval: Assessing Difficulty of Code Generation Tasks for Large Language Models0
Show:102550
← PrevPage 5 of 5Next →

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1deepseek-coder-33b-baseAverage69.01Unverified
2deepseek-coder-6.7b-baseAverage63.4Unverified
3starcoderbaseAverage55.54Unverified
4gpt-4-1106-previewAverage53.28Unverified
5CodeLlama-13b-hfAverage52.78Unverified
6deepseek-coder-1.3b-baseAverage52.63Unverified
7CodeLlama-34b-hfAverage49.66Unverified
8CodeLlama-7b-hfAverage45Unverified
9gpt-3.5-turbo-0301Average40.86Unverified
10incoder-6BAverage33.79Unverified
#ModelMetricClaimedVerifiedStatus
1CodeGPT-adaptedAccuracy (token-level)77.13Unverified
2CodeT5+ 770MEM (line-level)37.9Unverified
3CodeT5+ 220MEM (line-level)35.17Unverified
#ModelMetricClaimedVerifiedStatus
1CodeGPT-adaptedAccuracy (token-level)75.11Unverified
2CodeT5+ 770MEM (line-level)44.86Unverified
3CodeT5+ 220MEM (line-level)43.42Unverified
#ModelMetricClaimedVerifiedStatus
1SantaCoder-MGDCompilation Rate73.03Unverified
2SantaCoderCompilation Rate59.97Unverified
3SantaCoderCompilation Rate59.79Unverified
#ModelMetricClaimedVerifiedStatus
1RamboCompilation Rate76.47Unverified
2RepoCoderCompilation Rate74.02Unverified
#ModelMetricClaimedVerifiedStatus
1RamboCompilation Rate61.7Unverified
2RepoCoderCompilation Rate58.09Unverified