Zobrazeno 1 - 10
of 386
pro vyhledávání: '"Zhang, Ziyin"'
Programming languages possess rich semantic information such as data flow that is represented by graphs and not available from the surface form of source code. Recent code language models have scaled to billions of parameters, but model source code s
Externí odkaz:
http://arxiv.org/abs/2409.04183
We present GSM-MC, a multiple-choice (MC) dataset constructed by collecting answers and incorrect predictions on GSM8K from 60 open-source models. Through extensive experiments, we show that LLMs' performance on the MC version of this popular benchma
Externí odkaz:
http://arxiv.org/abs/2405.11966
Autor:
Ai, Yiming, He, Zhiwei, Zhang, Ziyin, Zhu, Wenhong, Hao, Hongkun, Yu, Kai, Chen, Lingjun, Wang, Rui
In this study, we investigate the reliability of Large Language Models (LLMs) in professing human-like personality traits through responses to personality questionnaires. Our goal is to evaluate the consistency between LLMs' professed personality inc
Externí odkaz:
http://arxiv.org/abs/2402.14679
Autor:
Jiang, Zhaokun, Zhang, Ziyin
Large language models have demonstrated parallel and even superior translation performance compared to neural machine translation (NMT) systems. However, existing comparative studies between them mainly rely on automated metrics, raising questions in
Externí odkaz:
http://arxiv.org/abs/2401.05176
The growing popularity of neural machine translation (NMT) and LLMs represented by ChatGPT underscores the need for a deeper understanding of their distinct characteristics and relationships. Such understanding is crucial for language professionals a
Externí odkaz:
http://arxiv.org/abs/2312.10750
In this work, we present the largest benchmark to date on linguistic acceptability: Multilingual Evaluation of Linguistic Acceptability -- MELA, with 46K samples covering 10 languages from a diverse set of language families. We establish LLM baseline
Externí odkaz:
http://arxiv.org/abs/2311.09033
Autor:
Zhang, Ziyin, Chen, Chaoyu, Liu, Bingchang, Liao, Cong, Gong, Zi, Yu, Hang, Li, Jianguo, Wang, Rui
In this work we systematically review the recent advancements in software engineering with language models, covering 70+ models, 40+ evaluation tasks, 180+ datasets, and 900 related works. Unlike previous works, we integrate software engineering (SE)
Externí odkaz:
http://arxiv.org/abs/2311.07989
Text recognition methods are gaining rapid development. Some advanced techniques, e.g., powerful modules, language models, and un- and semi-supervised learning schemes, consecutively push the performance on public benchmarks forward. However, the pro
Externí odkaz:
http://arxiv.org/abs/2308.08806
Autor:
Hu, Hai, Zhang, Ziyin, Huang, Weifang, Lai, Jackie Yan-Ki, Li, Aini, Patterson, Yina, Huang, Jiahui, Zhang, Peng, Lin, Chien-Jer Charles, Wang, Rui
In this work, we revisit linguistic acceptability in the context of large language models. We introduce CoLAC - Corpus of Linguistic Acceptability in Chinese, the first large-scale acceptability dataset for a non-Indo-European language. It is verifie
Externí odkaz:
http://arxiv.org/abs/2305.14091
Autor:
Jiang, Zhaokun, Zhang, Ziyin
Hedges are widely studied across registers and disciplines, yet research on the translation of hedges in political texts is extremely limited. This contrastive study is dedicated to investigating whether there is a diachronic change in the frequencie
Externí odkaz:
http://arxiv.org/abs/2305.12146