Cross-Model Comparative Loss for Enhancing Neuronal Utility in Language Understanding.

Autor: Zhu, Yunchang, Pang, Liang, Wu, Kangxi, Lan, Yanyan, Shen, Huawei, Cheng, Xueqi
Zdroj: ACM Transactions on Information Systems; Sep2024, Vol. 42 Issue 5, p1-29, 29p
Abstrakt: The article focuses on the limitations of current natural language understanding (NLU) models due to redundant hidden neurons and input noise, which hinder consistent performance improvements despite model scaling. Beyond existing approaches, the study proposes an intrinsic method to enhance neuron utility through a cross-model comparative loss.
Databáze: Complementary Index