GPU-Accelerated Viterbi Exact Lattice Decoder for Batched Online and Offline Speech Recognition
| Autor: | Justin Luitjens, Tim Kaldewey, Ryan Leary, Daniel Povey, Hugo Braun |
|---|---|
| Rok vydání: | 2019 |
| Předmět: |
FOS: Computer and information sciences
Speedup Computer Science - Computation and Language Computer science 020206 networking & telecommunications 02 engineering and technology Parallel computing Viterbi algorithm 030507 speech-language pathology & audiology 03 medical and health sciences symbols.namesake Audio and Speech Processing (eess.AS) 0202 electrical engineering electronic engineering information engineering Memory footprint symbols FOS: Electrical engineering electronic engineering information engineering 0305 other medical science Computation and Language (cs.CL) Decoding methods Electrical Engineering and Systems Science - Audio and Speech Processing Computer Science::Information Theory |
| Zdroj: | ICASSP |
| DOI: | 10.48550/arxiv.1910.10032 |
| Popis: | We present an optimized weighted finite-state transducer (WFST) decoder capable of online streaming and offline batch processing of audio using Graphics Processing Units (GPUs). The decoder is efficient in memory utilization, input/output (I/O) bandwidth, and uses a novel Viterbi implementation designed to maximize parallelism. The reduced memory footprint allows the decoder to process significantly larger graphs than previously possible, while optimizing I/O increases the number of simultaneous streams supported. GPU preprocessing of lattice segments enables intermediate lattice results to be returned to the requestor during streaming inference. Collectively, the proposed algorithm yields up to a 240x speedup over single core CPU decoding, and up to 40x faster decoding than the current state-of-the-art GPU decoder, while returning equivalent results. This decoder design enables deployment of production-grade ASR models on a large spectrum of systems, ranging from large data center servers to low-power edge devices. Comment: Accepted to ICASSP 2020 |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|