SOTAVerified

Code Completion

Papers

Showing 171–180 of 212 papers

TitleStatusHype
Stable Code Technical Report—0
Statically Contextualizing Large Language Models with Typed Holes—0
Structure-Aware Corpus Construction and User-Perception-Aligned Metrics for Large-Language-Model Code Completion—0
TPIA: Towards Target-specific Prompt Injection Attack against Code-oriented Large Language Models—0
TASTY: A Transformer based Approach to Space and Time complexity—0
Text and Patterns: For Effective Chain of Thought, It Takes Two to Tango—0
Token Alignment via Character Matching for Subword Completion—0
Toward Less Hidden Cost of Code Completion with Acceptance and Ranking Models—0
Towards Full-line Code Completion with Neural Language Models—0
User-Interactive Machine Learning Model for Identifying Structural Relationships of Code Features—0
Show:102550
← PrevPage 18 of 22Next →

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1deepseek-coder-33b-baseAverage69.01—Unverified
2deepseek-coder-6.7b-baseAverage63.4—Unverified
3starcoderbaseAverage55.54—Unverified
4gpt-4-1106-previewAverage53.28—Unverified
5CodeLlama-13b-hfAverage52.78—Unverified
6deepseek-coder-1.3b-baseAverage52.63—Unverified
7CodeLlama-34b-hfAverage49.66—Unverified
8CodeLlama-7b-hfAverage45—Unverified
9gpt-3.5-turbo-0301Average40.86—Unverified
10incoder-6BAverage33.79—Unverified
#ModelMetricClaimedVerifiedStatus
1CodeGPT-adaptedAccuracy (token-level)77.13—Unverified
2CodeT5+ 770MEM (line-level)37.9—Unverified
3CodeT5+ 220MEM (line-level)35.17—Unverified
#ModelMetricClaimedVerifiedStatus
1CodeGPT-adaptedAccuracy (token-level)75.11—Unverified
2CodeT5+ 770MEM (line-level)44.86—Unverified
3CodeT5+ 220MEM (line-level)43.42—Unverified
#ModelMetricClaimedVerifiedStatus
1SantaCoder-MGDCompilation Rate73.03—Unverified
2SantaCoderCompilation Rate59.97—Unverified
3SantaCoderCompilation Rate59.79—Unverified
#ModelMetricClaimedVerifiedStatus
1RamboCompilation Rate76.47—Unverified
2RepoCoderCompilation Rate74.02—Unverified
#ModelMetricClaimedVerifiedStatus
1RamboCompilation Rate61.7—Unverified
2RepoCoderCompilation Rate58.09—Unverified