Explaining repaired data with CFDs
| Autor: | Joeri Rammelaere, Floris Geerts |
|---|---|
| Jazyk: | angličtina |
| Rok vydání: | 2018 |
| Předmět: |
Computer. Automation
Computer science General Engineering 02 engineering and technology computer.software_genre Constraint (information theory) Business process discovery Set (abstract data type) 020204 information systems 0202 electrical engineering electronic engineering information engineering 020201 artificial intelligence & image processing Noise (video) Data mining Tuple computer |
| Zdroj: | Proceedings of the VLDB Endowment |
| ISSN: | 2150-8097 |
| Popis: | Many popular data cleaning approaches are rule-based: Constraints are formulated in a logical framework, and data is considered dirty if constraints are violated. These constraints are often discovered from data, but to ascertain their validity, user verification is necessary. Since the full set of discovered constraints is typically too large for manual inspection, recent research integrates user feedback into the discovery process. We propose a different approach that employs user interaction only at the start of the algorithm: a user manually cleans a small set of dirty tuples, and we infer the constraint underlying those repairs, called an explanation . We make use of conditional functional dependencies (CFDs) as the constraint formalism. We introduce XP lode , an on-demand algorithm which discovers the best explanation for a given repair. Guided by this explanation, data can then be cleaned using state-of-the-art CFD-based cleaning algorithms. Experiments on synthetic and real-world datasets show that the best explanation can typically be inferred using a limited number of modifications. Moreover, XP lode is substantially faster than discovering all CFDs that hold on a dataset, and is robust to noise in the modifications. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|