NLP-Fast: A Fast, Scalable, and Flexible System to Accelerate Large-Scale Heterogeneous NLP Models
| Autor: | Eunbok Lee, Joonsung Kim, Seung Ho Lee, Suyeon Hur, Jangwoo Kim |
|---|---|
| Rok vydání: | 2021 |
| Předmět: |
Flexibility (engineering)
Computer science business.industry Scale (chemistry) InformationSystems_INFORMATIONSTORAGEANDRETRIEVAL computer.software_genre ComputingMethodologies_ARTIFICIALINTELLIGENCE ComputingMethodologies_PATTERNRECOGNITION Scalability Artificial intelligence business Field-programmable gate array Throughput (business) computer Natural language processing |
| Zdroj: | PACT |
| DOI: | 10.1109/pact52795.2021.00013 |
| Popis: | Emerging natural language processing (NLP) models have become more complex and bigger to provide more sophisticated NLP services. Accordingly, there is also a strong demand for scalable and flexible computer infrastructure to support these large-scale, complex, and diverse NLP models. However, existing proposals cannot provide enough scalability and flexibility as they neither identify nor optimize a wide spectrum of performance-critical operations appearing in recent NLP models and only focus on optimizing specific operations. In this paper, we propose NLP-Fast, a novel system solution to accelerate a wide spectrum of large-scale NLP models. NLP-Fast mainly consists of two parts: (1) NLP-Perf: an in-depth performance analysis tool to identify critical operations in emerging NLP models and (2) NLP-Opt: three end-to-end optimization techniques to accelerate the identified performance-critical operations on various hardware platforms (e.g., CPU, GPU, FPGA). In this way, NLP-Fast can accelerate various types of NLP models on different hardware platforms by identifying their critical operations through NLP-Perf and applying the NLP-Opt's holistic optimizations. We evaluate NLP-Fast on CPU, GPU, and FPGA, and the overall throughputs are increased by up to 2.92×, 1.59×, and 4.47× over each platform's baseline. We release NLP-Fast to the community so that users are easily able to conduct the NLP-Fast's analysis and apply NLP-Fast's optimizations for their own NLP applications. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|