OpenAPI code completion
Code Completion in the OpenAPI format which is a widely used machine and human-readable format for describing RESTful APIs in YAML or JSON.
The benchmark source code is available on GitHub.
Papers
Showing 1–1 of 1 papers
| Title | Status | Hype |
|---|---|---|
| Optimizing Large Language Models for OpenAPI Code Completion | Code | 2 |
Benchmark Results
| # | Model | Metric | Claimed | Verified | Status |
|---|---|---|---|---|---|
| 1 | Code Llama 7B, fine-tuned with document splitting | Correctness, avg., % | 34 | — | Unverified |
| 2 | Code Llama 7B, fine-tuned at 4096 tokens | Correctness, avg., % | 32 | — | Unverified |
| 3 | Code Llama 7B | Correctness, avg., % | 31.1 | — | Unverified |
| 4 | GitHub Copilot | Correctness, avg., % | 29 | — | Unverified |