CAVE: Configuration Assessment, Visualization and Evaluation
| Autor: | Joshua Marben, Marius Lindauer, André Biedenkapp, Frank Hutter |
|---|---|
| Rok vydání: | 2018 |
| Předmět: |
Empirical data
021103 operations research Computer science 0211 other engineering and technologies Contrast (statistics) 02 engineering and technology computer.software_genre Visualization Set (abstract data type) Configurator 0202 electrical engineering electronic engineering information engineering Benchmark (computing) 020201 artificial intelligence & image processing Data mining Boolean satisfiability problem computer Software verification |
| Zdroj: | Lecture Notes in Computer Science ISBN: 9783030053475 LION |
| DOI: | 10.1007/978-3-030-05348-2_10 |
| Popis: | To achieve peak performance of an algorithm (in particular for problems in AI), algorithm configuration is often necessary to determine a well-performing parameter configuration. So far, most studies in algorithm configuration focused on proposing better algorithm configuration procedures or on improving a particular algorithm’s performance. In contrast, we use all the collected empirical performance data gathered during algorithm configuration runs to generate extensive insights into an algorithm, given problem instances and the used configurator. To this end, we provide a tool, called CAVE, that automatically generates comprehensive reports and insightful figures from all available empirical data. CAVE aims to help algorithm and configurator developers to better understand their experimental setup in an automated fashion. We showcase its use by thoroughly analyzing the well studied SAT solver spear on a benchmark of software verification instances and by empirically verifying two long-standing assumptions in algorithm configuration and parameter importance: (i) Parameter importance changes depending on the instance set at hand and (ii) Local and global parameter importance analysis do not necessarily agree with each other. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|