Zobrazeno 1 - 9
of 9
pro vyhledávání: '"Gong, Linyuan"'
We introduce Syntax-Aware Fill-In-the-Middle (SAFIM), a new benchmark for evaluating Large Language Models (LLMs) on the code Fill-in-the-Middle (FIM) task. This benchmark focuses on syntax-aware completions of program structures such as code blocks
Externí odkaz:
http://arxiv.org/abs/2403.04814
Large language models (LLMs) have made significant advancements in code-related tasks, yet many LLMs treat code as simple sequences, neglecting its structured nature. We introduce AST-T5, a novel pretraining paradigm that leverages the Abstract Synta
Externí odkaz:
http://arxiv.org/abs/2401.03003
Autor:
Gong, Linyuan, Xiong, Chenyan, Liu, Xiaodong, Bajaj, Payal, Xie, Yiqing, Cheung, Alvin, Gao, Jianfeng, Song, Xia
This paper explores the effectiveness of model-generated signals in improving zero-shot generalization of text-to-text Transformers such as T5. We study various designs to pretrain T5 using an auxiliary model to construct more challenging token repla
Externí odkaz:
http://arxiv.org/abs/2305.12567
We propose the Adversarial DEep Learning Transpiler (ADELT), a novel approach to source-to-source transpilation between deep learning frameworks. ADELT uniquely decouples code skeleton transpilation and API keyword mapping. For code skeleton transpil
Externí odkaz:
http://arxiv.org/abs/2303.03593
The task of completing knowledge triplets has broad downstream applications. Both structural and semantic information plays an important role in knowledge graph completion. Unlike previous approaches that rely on either the structures or semantics of
Externí odkaz:
http://arxiv.org/abs/2209.08721
Autoregressive models are widely used for tasks such as image and audio generation. The sampling process of these models, however, does not allow interruptions and cannot adapt to real-time computational resources. This challenge impedes the deployme
Externí odkaz:
http://arxiv.org/abs/2102.11495
Autor:
Xu, Zhenhui, Gong, Linyuan, Ke, Guolin, He, Di, Zheng, Shuxin, Wang, Liwei, Bian, Jiang, Liu, Tie-Yan
Pre-trained contextual representations (e.g., BERT) have become the foundation to achieve state-of-the-art results on many NLP tasks. However, large-scale pre-training is computationally expensive. ELECTRA, an early attempt to accelerate pre-training
Externí odkaz:
http://arxiv.org/abs/2006.05744
Autor:
Xia, Yingce, Tan, Xu, Tian, Fei, Gao, Fei, Chen, Weicong, Fan, Yang, Gong, Linyuan, Leng, Yichong, Luo, Renqian, Wang, Yiren, Wu, Lijun, Zhu, Jinhua, Qin, Tao, Liu, Tie-Yan
We Microsoft Research Asia made submissions to 11 language directions in the WMT19 news translation tasks. We won the first place for 8 of the 11 directions and the second place for the other three. Our basic systems are built on Transformer, back tr
Externí odkaz:
http://arxiv.org/abs/1911.06191
Autor:
Gong, Linyuan, Ji, Ruyi
TextCNN, the convolutional neural network for text, is a useful deep learning algorithm for sentence classification tasks such as sentiment analysis and question classification. However, neural networks have long been known as black boxes because int
Externí odkaz:
http://arxiv.org/abs/1801.06287