iBench: a Distributed Inference Simulation and Benchmark Suite
| Autor: | Ben Parsons, Robert P. Trevino, Wesley Emeneker, Wesley H. Brewer, Alan Scheinine, Greg Behm |
|---|---|
| Rok vydání: | 2020 |
| Předmět: |
Computer science
Distributed computing Node (networking) Payload (computing) Inference 020206 networking & telecommunications 02 engineering and technology Benchmarking Supercomputer Server 0202 electrical engineering electronic engineering information engineering Benchmark (computing) 020201 artificial intelligence & image processing Edge computing |
| Zdroj: | HPEC |
| Popis: | We present a novel distributed inference benchmarking system, called “iBench”, that provides relevant performance metrics for high-performance edge computing systems using trained deep learning models. The proposed benchmark is unique in that it includes data transfer performance through a distributed system, such as a supercomputer, using clients and servers to provide a system-level benchmark. iBench is flexible and robust enough to allow for the benchmarking of custom-built inference servers. This was demonstrated through the development of a custom Flask-based inference server to serve MLPerf's official ResNet50v1.5 model. In this paper, we compare iBench against MLPerf inference performance on an 8-V100 GPU node. iBench is shown to provide two primary advantages over MLPerf: (1) the ability to measure distributed inference performance, and (2) a more realistic measure of benchmark performance for inference servers on HPC by taking into account additional factors to inference time, such as HTTP request-response time, payload pre-processing and packing time, and invest time. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|