Výsledky vyhledávání

Report

Tuning Large language model for End-to-end Speech Translation

Autor: Zhang, Hao, Si, Nianwen, Chen, Yaqi, Zhang, Wenlin, Yang, Xukui, Qu, Dan, Jiao, Xiaolin

With the emergence of large language models (LLMs), multimodal models based on LLMs have demonstrated significant potential. Models such as LLaSM, X-LLM, and SpeechGPT exhibit an impressive ability to comprehend and generate human instructions. Howev

Externí odkaz: http://arxiv.org/abs/2310.02050

Zobrazit plný text záznamu

Report

DropDim: A Regularization Method for Transformer Networks

Autor: Zhang, Hao, Qu, Dan, Shao, Keji, Yang, Xukui

Publikováno v: IEEE SIGNAL PROCESSING LETTERS, VOL. 29, 2022

We introduceDropDim, a structured dropout method designed for regularizing the self-attention mechanism, which is a key component of the transformer. In contrast to the general dropout method, which randomly drops neurons, DropDim drops part of the e

Externí odkaz: http://arxiv.org/abs/2304.10321

Zobrazit plný text záznamu

Report

Improving Speech Translation by Cross-Modal Multi-Grained Contrastive Learning

Autor: Zhang, Hao, Si, Nianwen, Chen, Yaqi, Zhang, Wenlin, Yang, Xukui, Qu, Dan, Zhang, Wei-Qiang

Publikováno v: IEEE/ACM TRANSACTIONS ON AUDIO, SPEECH, AND LANGUAGE PROCESSING, VOL. 31, 2023

The end-to-end speech translation (E2E-ST) model has gradually become a mainstream paradigm due to its low latency and less error propagation. However, it is non-trivial to train such a model well due to the task complexity and data scarcity. The spe

Externí odkaz: http://arxiv.org/abs/2304.10309

Zobrazit plný text záznamu

Report

Decouple Non-parametric Knowledge Distillation For End-to-end Speech Translation

Autor: Zhang, Hao, Si, Nianwen, Chen, Yaqi, Zhang, Wenlin, Yang, Xukui, Qu, Dan, Li, Zhen

Existing techniques often attempt to make knowledge transfer from a powerful machine translation (MT) to speech translation (ST) model with some elaborate techniques, which often requires transcription as extra input during training. However, transcr

Externí odkaz: http://arxiv.org/abs/2304.10295

Zobrazit plný text záznamu