Data intensive high energy physics analysis in a distributed cloud
| Autor: | Ian Gable, Andre Charbonneau, Randall Sobie, D Harris, M Anderson, A Agarwal, M Vliet, Michael Paterson, Colin Leavett-Brown, W Podaima, Roger Impey, K Fransham, P Armstrong |
|---|---|
| Rok vydání: | 2012 |
| Předmět: |
History
Particle physics Computer science Distributed computing Cloud computing computer.software_genre Education High-throughput computing Input datas High energy physics Central database Throughput (business) Running applications business.industry Cloud systems Physics Process (computing) Data intensive Computer Science Applications Batch jobs Calibration data Virtual machine Operating system Batch processing System use Virtual machines business Scale (map) computer |
| Zdroj: | Journal of Physics: Conference Series. 341:012003 |
| ISSN: | 1742-6596 |
| DOI: | 10.1088/1742-6596/341/1/012003 |
| Popis: | We show that distributed Infrastructure-as-a-Service (IaaS) compute clouds can be effectively used for the analysis of high energy physics data. We have designed a distributed cloud system that works with any application using large input data sets requiring a high throughput computing environment. The system uses IaaS-enabled science and commercial clusters in Canada and the United States. We describe the process in which a user prepares an analysis virtual machine (VM) and submits batch jobs to a central scheduler. The system boots the user-specific VM on one of the IaaS clouds, runs the jobs and returns the output to the user. The user application accesses a central database for calibration data during the execution of the application. Similarly, the data is located in a central location and streamed by the running application. The system can easily run one hundred simultaneous jobs in an efficient manner and should scale to many hundreds and possibly thousands of user jobs. © IOP Publishing Ltd. 25th High Performance Computing Symposium 2011, HPCS 2011, June 15-17 2011, Montreal, Quebec, Canada |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|