SOTAVerified

Code Completion

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 151–200 of 212 papers

TitleStatusHype
LLMSecEval: A Dataset of Natural Language Prompts for Security EvaluationsCode1
Exploring ChatGPT's Ability to Rank Content: A Preliminary Study on Consistency with Human PreferencesCode0
From Copilot to Pilot: Towards AI Supported Software Development—0
Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt InjectionCode4
Learning Deep Semantics for Test CompletionCode1
Generation Probabilities Are Not Enough: Uncertainty Highlighting in AI Code Completions—0
Automating Code-Related Tasks Through Transformers: The Impact of Pre-trainingCode0
Execution-based Code Generation using Deep Reinforcement LearningCode1
Serenity: Library Based Python Code Analysis for Code Completion and Automated Machine Learning—0
Unveiling Code Pre-Trained Models: Investigating Syntax and Semantics Capacities—0
CoCoMIC: Code Completion By Jointly Modeling In-file and Cross-file ContextCode1
MultiCoder: Multi-Programming-Lingual Pre-Training for Low-Resource Code Completion—0
Syntax-Aware On-the-Fly Code CompletionCode0
Multi-lingual Evaluation of Code Generation ModelsCode1
Reading Between the Lines: Modeling User Behavior and Costs in AI-Assisted ProgrammingCode1
Text and Patterns: For Effective Chain of Thought, It Takes Two to Tango—0
Don't Complete It! Preventing Unhelpful Code Completion for Productive and Sustainable Neural Code Completion SystemsCode0
MetaTPTrans: A Meta Learning Approach for Multilingual Code Representation LearningCode1
HierarchyNet: Learning to Summarize Source Code with Heterogeneous Representations—0
All You Need Is Logs: Improving Code Completion by Learning from Anonymous IDE Usage Logs—0
Productivity Assessment of Neural Code CompletionCode1
ReACC: A Retrieval-Augmented Code Completion FrameworkCode1
Compilable Neural Code Generation with Compiler Feedback—0
UniXcoder: Unified Cross-Modal Pre-training for Code RepresentationCode1
CodeFill: Multi-token Code Completion by Jointly Learning from Structure and Naming SequencesCode1
What Do They Capture? -- A Structural Analysis of Pre-Trained Language Models for Source CodeCode0
Examining Zero-Shot Vulnerability Repair with Large Language Models—0
Long-Range Modeling of Source Code Files with eWASH: Extended Window Access by Syntax Hierarchy—0
Time-Efficient Code Completion Model for the R Programming LanguageCode0
Toward Less Hidden Cost of Code Completion with Acceptance and Ranking Models—0
Learning to Complete Code with Sketches—0
A Syntax-Guided Edit Decoder for Neural Program RepairCode1
Energy-Based Models for Code Generation under Compilability ConstraintsCode1
Learning to Extend Program Graphs to Work-in-Progress Code—0
Improving Code Autocompletion with Transfer Learning—0
Code Completion by Modeling Flattened Abstract Syntax Trees as Graphs—0
CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and GenerationCode1
Neural Software AnalysisCode0
A Simple Approach for Handling Out-of-Vocabulary Identifiers in Deep Learning for Source CodeCode1
On the Embeddings of Variables in Recurrent Neural Networks for Source CodeCode0
Learning to Execute Programs with Instruction Pointer Attention Graph Neural NetworksCode0
Empirical Study of Transformers for Source CodeCode1
Towards Full-line Code Completion with Neural Language Models—0
GraphCodeBERT: Pre-training Code Representations with Data FlowCode0
You Autocomplete Me: Poisoning Vulnerabilities in Neural Code Completion—0
IntelliCode Compose: Code Generation Using Transformer—0
LambdaNet: Probabilistic Type Inference using Graph Neural NetworksCode1
Fast and Memory-Efficient Neural Code Completion—0
Code Completion using Neural Attention and Byte Pair Encoding—0
Sequence Model Design for Code Completion in the Modern IDE—0
Show:102550
← PrevPage 4 of 5Next →
All datasetsSAFIMCodeXGLUE - Github Java CorpusCodeXGLUE - PY150DotPromptsDefects4JRambo Benchmark

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1deepseek-coder-33b-baseAverage69.01—Unverified
2deepseek-coder-6.7b-baseAverage63.4—Unverified
3starcoderbaseAverage55.54—Unverified
4gpt-4-1106-previewAverage53.28—Unverified
5CodeLlama-13b-hfAverage52.78—Unverified
6deepseek-coder-1.3b-baseAverage52.63—Unverified
7CodeLlama-34b-hfAverage49.66—Unverified
8CodeLlama-7b-hfAverage45—Unverified
9gpt-3.5-turbo-0301Average40.86—Unverified
10incoder-6BAverage33.79—Unverified
#ModelMetricClaimedVerifiedStatus
1CodeGPT-adaptedAccuracy (token-level)77.13—Unverified
2CodeT5+ 770MEM (line-level)37.9—Unverified
3CodeT5+ 220MEM (line-level)35.17—Unverified
#ModelMetricClaimedVerifiedStatus
1CodeGPT-adaptedAccuracy (token-level)75.11—Unverified
2CodeT5+ 770MEM (line-level)44.86—Unverified
3CodeT5+ 220MEM (line-level)43.42—Unverified
#ModelMetricClaimedVerifiedStatus
1SantaCoder-MGDCompilation Rate73.03—Unverified
2SantaCoderCompilation Rate59.97—Unverified
3SantaCoderCompilation Rate59.79—Unverified
#ModelMetricClaimedVerifiedStatus
1RamboCompilation Rate76.47—Unverified
2RepoCoderCompilation Rate74.02—Unverified
#ModelMetricClaimedVerifiedStatus
1RamboCompilation Rate61.7—Unverified
2RepoCoderCompilation Rate58.09—Unverified