A Search-Based Testing Framework for Deep Neural Networks of Source Code Embedding
| Autor: | Maryam Vahdat Pour, Hadi Hemmati, Lei Ma, Zhuo Li |
|---|---|
| Rok vydání: | 2021 |
| Předmět: |
FOS: Computer and information sciences
Source code Artificial neural network Computer science business.industry media_common.quotation_subject Context (language use) Machine learning computer.software_genre Speech processing Software Engineering (cs.SE) Computer Science - Software Engineering Code refactoring Robustness (computer science) Code (cryptography) Artificial intelligence business computer media_common Test data |
| Zdroj: | ICST |
| DOI: | 10.48550/arxiv.2101.07910 |
| Popis: | Over the past few years, deep neural networks (DNNs) have been continuously expanding their real-world applications for source code processing tasks across the software engineering domain, e.g., clone detection, code search, comment generation. Although quite a few recent works have been performed on testing of DNNs in the context of image and speech processing, limited progress has been achieved so far on DNN testing in the context of source code processing, that exhibits rather unique characteristics and challenges. In this paper, we propose a search-based testing framework for DNNs of source code embedding and its downstream processing tasks like Code Search. To generate new test inputs, we adopt popular source code refactoring tools to generate the semantically equivalent variants. For more effective testing, we leverage the DNN mutation testing to guide the testing direction. To demonstrate the usefulness of our technique, we perform a large-scale evaluation on popular DNNs of source code processing based on multiple state-of-the-art code embedding methods (i.e., Code2vec, Code2seq and CodeBERT). The testing results show that our generated adversarial samples can on average reduce the performance of these DNNs from 5.41% to 9.58%. Through retraining the DNNs with our generated adversarial samples, the robustness of DNN can improve by 23.05% on average. The evaluation results also show that our adversarial test generation strategy has the least negative impact (median of 3.56%), on the performance of the DNNs for regular test data, compared to the other methods. Comment: ICST 2021 |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|