A Test Architecture for Machine Learning Product
| Autor: | Hideto Ogawa, Yasuharu Nishi, Satoshi Masuda, Keiji Uetsuki |
|---|---|
| Rok vydání: | 2018 |
| Předmět: |
Functional safety
Test design business.industry Computer science Product testing 020207 software engineering 02 engineering and technology Machine learning computer.software_genre Software Robustness (computer science) 0202 electrical engineering electronic engineering information engineering Systems architecture Snapshot (computer storage) 020201 artificial intelligence & image processing Artificial intelligence business Quality assurance computer |
| Zdroj: | ICST Workshops |
| DOI: | 10.1109/icstw.2018.00060 |
| Popis: | As machine learning (ML) technology continues to spread by rapid evolution, the system or service using Machine Learning technology, called ML product, makes big impact on our life, society and economy. Meanwhile, Quality Assurance (QA) for ML product is quite more difficult than hardware, non-ML software and service because performance of ML technology is much better than non-ML technology in exchange for the characteristics of ML product, e.g. low explainability. We must keep rapid evolution and reduce quality risk of ML product simultaneously. In this paper, we show a Quality Assurance Framework for Machine Learning product. Scope of QA in this paper is limited to product evaluation. First, a policy of QA for ML Product is proposed. General principles of product evaluation is introduced and applied to ML product evaluation as a part of the policy. They are composed of A-ARAI: Allowability, Achievability, Robustness, Avoidability and Improvability. A strategy of ML Product Evaluation is constructed as another part of the policy. Quality Integrity Level for ML product is also modelled. Second, we propose a test architecture of ML product testing. It consists of test levels and fundamental test types of ML product testing, including snapshot testing, learning testing and confrontation testing. Finally, we defines QA activity levels for ML product. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|