Exploring the Verifiability of Code Generated by GitHub Copilot

Autor: Wong, Dakota, Kothig, Austin, Lam, Patrick
Rok vydání: 2022
Předmět:
Druh dokumentu: Working Paper
Popis: GitHub's Copilot generates code quickly. We investigate whether it generates good code. Our approach is to identify a set of problems, ask Copilot to generate solutions, and attempt to formally verify these solutions with Dafny. Our formal verification is with respect to hand-crafted specifications. We have carried out this process on 6 problems and succeeded in formally verifying 4 of the created solutions. We found evidence which corroborates the current consensus in the literature: Copilot is a powerful tool; however, it should not be "flying the plane" by itself.
Comment: HATRA workshop at SPLASH 2022
Databáze: arXiv