Výsledky vyhledávání - "SAKAI, YOSHIHIRO"

Report

Revisiting In-context Learning Inference Circuit in Large Language Models

Autor: Cho, Hakaze, Kato, Mariko, Sakai, Yoshihiro, Inoue, Naoya

In-context Learning (ICL) is an emerging few-shot learning paradigm on Language Models (LMs) with inner mechanisms un-explored. There are already existing works describing the inner processing of ICL, while they struggle to capture all the inference

Externí odkaz: http://arxiv.org/abs/2410.04468

Zobrazit plný text záznamu

Report

Token-based Decision Criteria Are Suboptimal in In-context Learning

Autor: Cho, Hakaze, Sakai, Yoshihiro, Kato, Mariko, Tanaka, Kenshiro, Ishii, Akira, Inoue, Naoya

In-Context Learning (ICL) typically utilizes classification criteria from output probabilities of manually selected label tokens. However, we argue that such token-based classification criteria lead to suboptimal decision boundaries, despite delicate

Externí odkaz: http://arxiv.org/abs/2406.16535

Zobrazit plný text záznamu

Report

Understanding Token Probability Encoding in Output Embeddings

Autor: Cho, Hakaze, Sakai, Yoshihiro, Tanaka, Kenshiro, Kato, Mariko, Inoue, Naoya

In this paper, we investigate the output token probability information in the output embedding of language models. We find an approximate common log-linear encoding of output token probabilities within the output embedding vectors and empirically dem

Externí odkaz: http://arxiv.org/abs/2406.01468

Zobrazit plný text záznamu

Report

NoisyICL: A Little Noise in Model Parameters Calibrates In-context Learning

Autor: Zhao, Yufeng, Sakai, Yoshihiro, Inoue, Naoya

In-Context Learning (ICL) is suffering from unsatisfactory performance and under-calibration due to high prior bias and unfaithful confidence. Some previous works fine-tuned language models for better ICL performance with enormous datasets and comput

Externí odkaz: http://arxiv.org/abs/2402.05515

Zobrazit plný text záznamu