Výsledky vyhledávání - "Leng, Jixuan"

Report

S$^{2}$FT: Efficient, Scalable and Generalizable LLM Fine-tuning by Structured Sparsity

Autor: Yang, Xinyu, Leng, Jixuan, Guo, Geyang, Zhao, Jiawei, Nakada, Ryumei, Zhang, Linjun, Yao, Huaxiu, Chen, Beidi

Current PEFT methods for LLMs can achieve either high quality, efficient training, or scalable serving, but not all three simultaneously. To address this limitation, we investigate sparse fine-tuning and observe a remarkable improvement in generaliza

Externí odkaz: http://arxiv.org/abs/2412.06289

Zobrazit plný text záznamu

Report

Taming Overconfidence in LLMs: Reward Calibration in RLHF

Autor: Leng, Jixuan, Huang, Chengsong, Zhu, Banghua, Huang, Jiaxin

Language model calibration refers to the alignment between the confidence of the model and the actual performance of its responses. While previous studies point out the overconfidence phenomenon in Large Language Models (LLMs) and show that LLMs trai

Externí odkaz: http://arxiv.org/abs/2410.09724

Zobrazit plný text záznamu

Report

Choosing Wisely and Learning Deeply: Selective Cross-Modality Distillation via CLIP for Domain Generalization

Autor: Leng, Jixuan, Li, Yijiang, Wang, Haohan

Domain Generalization (DG), a crucial research area, seeks to train models across multiple domains and test them on unseen ones. In this paper, we introduce a novel approach, namely, Selective Cross-Modality Distillation for Domain Generalization (SC

Externí odkaz: http://arxiv.org/abs/2311.15145

Zobrazit plný text záznamu