Zobrazeno 1 - 10
of 65
pro vyhledávání: '"Huang, Xinmeng"'
Stochastic bilevel optimization tackles challenges involving nested optimization structures. Its fast-growing scale nowadays necessitates efficient distributed algorithms. In conventional distributed bilevel methods, each worker must transmit full-di
Externí odkaz:
http://arxiv.org/abs/2405.18858
The growing safety concerns surrounding Large Language Models (LLMs) raise an urgent need to align them with diverse human preferences to simultaneously enhance their helpfulness and safety. A promising approach is to enforce safety constraints throu
Externí odkaz:
http://arxiv.org/abs/2405.19544
Autor:
Huang, Xinmeng, Li, Shuo, Yu, Mengxin, Sesia, Matteo, Hassani, Hamed, Lee, Insup, Bastani, Osbert, Dobriban, Edgar
Language Models (LMs) have shown promising performance in natural language generation. However, as LMs often generate incorrect or hallucinated responses, it is crucial to correctly quantify their uncertainty in responding to given inputs. In additio
Externí odkaz:
http://arxiv.org/abs/2404.03163
Stochastic bilevel optimization (SBO) is becoming increasingly essential in machine learning due to its versatility in handling nested structures. To address large-scale SBO, decentralized approaches have emerged as effective paradigms in which nodes
Externí odkaz:
http://arxiv.org/abs/2402.03167
This paper investigates the influence of directed networks on decentralized stochastic non-convex optimization associated with column-stochastic mixing matrices. Surprisingly, we find that the canonical spectral gap, a widely used metric in undirecte
Externí odkaz:
http://arxiv.org/abs/2312.04928
Communication compression, a technique aiming to reduce the information volume to be transmitted over the air, has gained great interests in Federated Learning (FL) for the potential of alleviating its communication overhead. However, communication c
Externí odkaz:
http://arxiv.org/abs/2308.08165
Federated learning is a powerful paradigm for large-scale machine learning, but it faces significant challenges due to unreliable network connections, slow communication, and substantial data heterogeneity across clients. FedAvg and SCAFFOLD are two
Externí odkaz:
http://arxiv.org/abs/2306.16504
Large and complex datasets are often collected from several, possibly heterogeneous sources. Collaborative learning methods improve efficiency by leveraging commonalities across datasets while accounting for possible differences among them. Here we s
Externí odkaz:
http://arxiv.org/abs/2306.06291
Communication compression is a common technique in distributed optimization that can alleviate communication overhead by transmitting compressed gradients and model parameters. However, compression can introduce information distortion, which slows do
Externí odkaz:
http://arxiv.org/abs/2305.16297
Communication compression is an essential strategy for alleviating communication overhead by reducing the volume of information exchanged between computing nodes in large-scale distributed stochastic optimization. Although numerous algorithms with co
Externí odkaz:
http://arxiv.org/abs/2305.07612