Analysis of empty ATLAS pilot jobs

Autor: ATLAS Collaboration, Love, P. A., Alef, Manfred, Dal Pra, S., Di Girolamo, A., Forti, A., Templon, J., Vamvakopoulos, E.
Přispěvatelé: Centre de Calcul de l'IN2P3 (CC-IN2P3), Centre National de la Recherche Scientifique (CNRS)-Institut National de Physique Nucléaire et de Physique des Particules du CNRS (IN2P3), ATLAS, Institut National de Physique Nucléaire et de Physique des Particules du CNRS (IN2P3)-Centre National de la Recherche Scientifique (CNRS)
Jazyk: angličtina
Rok vydání: 2017
Předmět:
Zdroj: J.Phys.Conf.Ser.
22nd International Conference on Computing in High Energy and Nuclear Physics
22nd International Conference on Computing in High Energy and Nuclear Physics, Oct 2016, San Francisco, United States. pp.092005, ⟨10.1088/1742-6596/898/9/092005⟩
Journal of physics / Conference Series, 898, 092005
ISSN: 1742-6588
1742-6596
DOI: 10.5445/ir/1000076866#verlagsausgabe
Popis: In this analysis we quantify the wallclock time used by short empty pilot jobs on a number of WLCG compute resources. Pilot factory logs and site batch logs are used to provide independent accounts of the usage. Results show a wide variation of wallclock time used by short jobs depending on the site and queue, and changing with time. For a reference dataset of all jobs in August 2016, the fraction of wallclock time used by empty jobs per studied site ranged from 0.1% to 0.8%. The variation in wallclock usage may be explained by different workloads for each resource with a greater fraction when the workload is low. Aside from the wall time used by empty pilots, we also looked at how many pilots were empty as a fraction of all pilots sent. Binning the August dataset into days, empty fractions between 2% and 90% were observed. The higher fractions correlate well with periods of few actual payloads being sent to the site. In this analysis we quantify the wallclock time used by short empty pilot jobs on a number of WLCG compute resources. Pilot factory logs and site batch logs are used to provide independent accounts of the usage. Results show a wide variation of wallclock time used by short jobs depending on the site and queue, and changing with time. For a reference dataset of all jobs in August 2016, the fraction of wallclock time used by empty jobs per studied site ranged from 0.1% to 0.8%. Aside from the wall time used by empty pilots, we also looked at how many pilots were empty as a fraction of all pilots sent. Binning the August dataset into days, empty fractions between 2% and 90% were observed. The higher fractions correlate well with periods of few actual payloads being sent to the site.
Databáze: OpenAIRE
Externí odkaz: