Experiences with Formal Engineering
| Autor: | Sijtema, M., Salaün, G., Schätz, B., Belinfante, Axel, Stoelinga, Mariëlle Ida Antoinette, Marinelli, L. |
|---|---|
| Jazyk: | angličtina |
| Rok vydání: | 2011 |
| Předmět: |
Computer science
Model-Based Testing White-box testing Code coverage 02 engineering and technology JTorX computer.software_genre METIS-303971 0202 electrical engineering electronic engineering information engineering Programming language IR-78451 Formal methods Formal engineering lps2torx 020201 artificial intelligence & image processing EC Grant Agreement nr.: FP7-ICT-2007-1 Model checking CADP LPS Process (engineering) IR-88365 evaluator4 Formal Methods mCRL2 Formal specification Component (UML) EC Grant Agreement nr.: FP7/214755 Regression testing EC Grant Agreement nr.: FP7/2007-2013 Code (cryptography) Software verification and validation EC Grant Agreement nr.: FP7/318490 Simulation LTSMIN METIS-279685 EWI-23994 Model-based testing business.industry Software bus 020207 software engineering MCL IOCO EWI-20768 EC Grant Agreement nr.: FP7/318003 Software construction Software reliability testing Software engineering business computer Software |
| Zdroj: | Science of computer programming, 80(Part A), 188-209. Elsevier 16th International Workshop on Formal Methods for Industrial Critical Systems, FMICS 2011, 117-133 STARTPAGE=117;ENDPAGE=133;TITLE=16th International Workshop on Formal Methods for Industrial Critical Systems, FMICS 2011 Formal Methods for Industrial Critical Systems ISBN: 9783642244308 FMICS |
| ISSN: | 0302-9743 0167-6423 |
| DOI: | 10.1007/978-3-642-24431-5_10 |
| Popis: | We report on the actual industrial use of formal methods during the development of a software bus. During an internship at Neopost Inc., of 14 weeks, we developed the server component of a software bus, called the XBus, using formal methods during the design, validation and testing phase: we modeled our design of the XBus in the process algebra mCRL2, validated the design using the mCRL2-simulator, and fully automatically tested our implementation with the model-based test tool JTorX. This resulted in a well-tested software bus with a maintainable architecture. Writing the model (m"d"e"v), simulating it, and testing the implementation with JTorX only took 17% of the total development time. Moreover, the errors found with model-based testing would have been hard to find with conventional test methods. Thus, we show that formal engineering can be feasible, beneficial and cost-effective. The findings above, reported earlier by us in (Sijtema et al., 2011) [1], were well-received, also in industrially oriented conferences (Ferreira and Romanenko, 2010) [2] and [3]. In this paper, we look back on the case study, and carefully analyze its merits and shortcomings. We reflect on (1) the added benefits of model checking, (2) model completeness and (3) the quality and performance of the test process. Thus, in a second phase, after the internship, we model checked the XBus protocol-this was not done in [1] since the Neopost business process required a working implementation after 14 weeks. We used the CADP tool evaluator4 to check the behavioral requirements obtained during the development. Model checking did not uncover errors in model m"d"e"v, but revealed that model m"d"e"v was neither complete nor optimized: in particular, requirements to the so-called bad weather behavior (exceptions, unexpected inputs, etc.) were missing. Therefore, we created several improved models, checked that we could validate them, and used them to analyze quality and performance of the test process. Model checking was expensive: it took us approx. 4 weeks in total, compared to 3 weeks for the entire model-based testing approach during the internship. In the second phase, we analyzed the quality and performance of the test process, where we looked at both code and model coverage. We found that high code coverage (almost 100%) is in most cases obtained within 1000 test steps and 2 minutes, which matches the fact that the faults in the XBus were discovered within a few minutes. Summarizing, we firmly believe that the formal engineering approach is cost-effective, and produces high quality software products. Model checking does yield significantly better models, but is also costly. Thus, system developers should trade off higher model quality against higher costs. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|
Full Text from ScienceDirect