SOTAVerified

Code Completion

Papers

Showing 51100 of 212 papers

TitleStatusHype
Learning Deep Semantics for Test CompletionCode1
LayoutNUWA: Revealing the Hidden Layout Expertise of Large Language ModelsCode1
CoCoMIC: Code Completion By Jointly Modeling In-file and Cross-file ContextCode1
RepoBench: Benchmarking Repository-Level Code Auto-Completion SystemsCode1
How to Get Your LLM to Generate Challenging Problems for EvaluationCode1
Empirical Study of Transformers for Source CodeCode1
LLMSecEval: A Dataset of Natural Language Prompts for Security EvaluationsCode1
How Effective Are Neural Networks for Fixing Security VulnerabilitiesCode1
UniXcoder: Unified Cross-Modal Pre-training for Code RepresentationCode1
CrossCodeEval: A Diverse and Multilingual Benchmark for Cross-File Code CompletionCode1
LambdaNet: Probabilistic Type Inference using Graph Neural NetworksCode1
VersiCode: Towards Version-controllable Code GenerationCode1
Ada-Instruct: Adapting Instruction Generators for Complex ReasoningCode1
LogQuant: Log-Distributed 2-Bit Quantization of KV Cache with Superior Accuracy PreservationCode1
DataSculpt: Crafting Data Landscapes for Long-Context LLMs through Multi-Objective PartitioningCode1
Dataflow-Guided Retrieval Augmentation for Repository-Level Code CompletionCode1
A Syntax-Guided Edit Decoder for Neural Program RepairCode1
IRCoder: Intermediate Representations Make Language Models Robust Multilingual Code GeneratorsCode1
CASTLE: Benchmarking Dataset for Static Code Analyzers and LLMs towards CWE DetectionCode1
Can Large Language Models Write Parallel Code?Code1
Security Attacks on LLM-based Code Completion ToolsCode1
GitChameleon: Unmasking the Version-Switching Capabilities of Code Generation ModelsCode1
Breaking the Silence: the Threats of Using LLMs in Software EngineeringCode0
Structural Language Models of CodeCode0
CodeT5+: Open Code Large Language Models for Code Understanding and GenerationCode0
Pythia: AI-assisted Code Completion SystemCode0
CodeMark: Imperceptible Watermarking for Code Datasets against Neural Code Completion ModelsCode0
CodeKGC: Code Language Model for Generative Knowledge Graph ConstructionCode0
CodeGRU: Context-aware Deep Learning with Gated Recurrent Unit for Source Code ModelingCode0
Automating Code-Related Tasks Through Transformers: The Impact of Pre-trainingCode0
ObscuraCoder: Powering Efficient Code LM Pre-Training Via Obfuscation GroundingCode0
On the Embeddings of Variables in Recurrent Neural Networks for Source CodeCode0
Code Completion with Neural Attention and Pointer NetworksCode0
Neural Software AnalysisCode0
MERGE: Fast Private Text GenerationCode0
LongCoder: A Long-Range Pre-trained Language Model for Code CompletionCode0
Exploring ChatGPT's Ability to Rank Content: A Preliminary Study on Consistency with Human PreferencesCode0
Open Vocabulary Learning on Source Code with a Graph-Structured CacheCode0
Learning to Execute Programs with Instruction Pointer Attention Graph Neural NetworksCode0
GenCodeSearchNet: A Benchmark Test Suite for Evaluating Generalization in Programming Language UnderstandingCode0
Enriching Source Code with Contextual Data for Code Completion Models: An Empirical StudyCode0
GraphCodeBERT: Pre-training Code Representations with Data FlowCode0
GREEN-CODE: Learning to Optimize Energy Efficiency in LLM-based Code GenerationCode0
Don't Complete It! Preventing Unhelpful Code Completion for Productive and Sustainable Neural Code Completion SystemsCode0
Enhancing High-Quality Code Generation in Large Language Models with Comparative Prefix-TuningCode0
A Transformer-Based Approach for Smart Invocation of Automatic Code CompletionCode0
Large Language Models of Code Fail at Completing Code with Potential BugsCode0
Eclipse CDT code analysis and unit testingCode0
KV Prediction for Improved Time to First TokenCode0
Can You Really Trust Code Copilots? Evaluating Large Language Models from a Code Security PerspectiveCode0
Show:102550
← PrevPage 2 of 5Next →

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1deepseek-coder-33b-baseAverage69.01Unverified
2deepseek-coder-6.7b-baseAverage63.4Unverified
3starcoderbaseAverage55.54Unverified
4gpt-4-1106-previewAverage53.28Unverified
5CodeLlama-13b-hfAverage52.78Unverified
6deepseek-coder-1.3b-baseAverage52.63Unverified
7CodeLlama-34b-hfAverage49.66Unverified
8CodeLlama-7b-hfAverage45Unverified
9gpt-3.5-turbo-0301Average40.86Unverified
10incoder-6BAverage33.79Unverified
#ModelMetricClaimedVerifiedStatus
1CodeGPT-adaptedAccuracy (token-level)77.13Unverified
2CodeT5+ 770MEM (line-level)37.9Unverified
3CodeT5+ 220MEM (line-level)35.17Unverified
#ModelMetricClaimedVerifiedStatus
1CodeGPT-adaptedAccuracy (token-level)75.11Unverified
2CodeT5+ 770MEM (line-level)44.86Unverified
3CodeT5+ 220MEM (line-level)43.42Unverified
#ModelMetricClaimedVerifiedStatus
1SantaCoder-MGDCompilation Rate73.03Unverified
2SantaCoderCompilation Rate59.97Unverified
3SantaCoderCompilation Rate59.79Unverified
#ModelMetricClaimedVerifiedStatus
1RamboCompilation Rate76.47Unverified
2RepoCoderCompilation Rate74.02Unverified
#ModelMetricClaimedVerifiedStatus
1RamboCompilation Rate61.7Unverified
2RepoCoderCompilation Rate58.09Unverified