Zobrazeno 1 - 10
of 19
pro vyhledávání: '"Li, Hongkang"'
Efficient training and inference algorithms, such as low-rank adaption and model pruning, have shown impressive performance for learning Transformer-based large foundation models. However, due to the technical challenges of the non-convex optimizatio
Externí odkaz:
http://arxiv.org/abs/2406.17167
Graph Transformers, which incorporate self-attention and positional encoding, have recently emerged as a powerful architecture for various graph learning tasks. Despite their impressive performance, the complex non-convex interactions across layers a
Externí odkaz:
http://arxiv.org/abs/2406.01977
Group imbalance has been a known problem in empirical risk minimization (ERM), where the achieved high average accuracy is accompanied by low accuracy in a minority group. Despite algorithmic efforts to improve the minority group accuracy, a theoreti
Externí odkaz:
http://arxiv.org/abs/2403.07310
Transformer-based large language models have displayed impressive in-context learning capabilities, where a pre-trained model can handle new tasks without fine-tuning by simply augmenting the query with some input-output examples from that task. Desp
Externí odkaz:
http://arxiv.org/abs/2402.15607
Autor:
Zhang, Shuai, Li, Hongkang, Wang, Meng, Liu, Miao, Chen, Pin-Yu, Lu, Songtao, Liu, Sijia, Murugesan, Keerthiram, Chaudhury, Subhajit
Publikováno v:
Neurips 2023
This paper provides a theoretical understanding of Deep Q-Network (DQN) with the $\varepsilon$-greedy exploration in deep reinforcement learning. Despite the tremendous empirical achievement of the DQN, its theoretical characterization remains undere
Externí odkaz:
http://arxiv.org/abs/2310.16173
The integration of external personalized context information into document-grounded conversational systems has significant potential business value, but has not been well-studied. Motivated by the concept of personalized context-aware document-ground
Externí odkaz:
http://arxiv.org/abs/2308.13760
Graph transformers need strong inductive biases to derive meaningful attention scores. Yet, current methods often fall short in capturing longer ranges, hierarchical structures, or community structures, which are common in various graphs such as mole
Externí odkaz:
http://arxiv.org/abs/2308.11129
Publikováno v:
ICLR 2023
Vision Transformers (ViTs) with self-attention modules have recently achieved great empirical success in many vision tasks. Due to non-convex interactions across layers, however, theoretical learning and generalization analysis is mostly elusive. Bas
Externí odkaz:
http://arxiv.org/abs/2302.06015
Learning and generalization of one-hidden-layer neural networks, going beyond standard Gaussian data
This paper analyzes the convergence and generalization of training a one-hidden-layer neural network when the input features follow the Gaussian mixture model consisting of a finite number of Gaussian distributions. Assuming the labels are generated
Externí odkaz:
http://arxiv.org/abs/2207.03615
Publikováno v:
ICML 2022
Graph convolutional networks (GCNs) have recently achieved great empirical success in learning graph-structured data. To address its scalability issue due to the recursive embedding of neighboring features, graph topology sampling has been proposed t
Externí odkaz:
http://arxiv.org/abs/2207.03584