Zobrazeno 1 - 10
of 156
pro vyhledávání: '"Jaiswal, Ajay"'
Autor:
Jaiswal, Ajay, Choudhary, Nurendra, Adkathimar, Ravinarayana, Alagappan, Muthu P., Hiranandani, Gaurush, Ding, Ying, Wang, Zhangyang, Huang, Edward W, Subbian, Karthik
Graph Neural Networks (GNNs) have attracted immense attention in the past decade due to their numerous real-world applications built around graph-structured data. On the other hand, Large Language Models (LLMs) with extensive pretrained knowledge and
Externí odkaz:
http://arxiv.org/abs/2407.14996
Autor:
Jaiswal, Ajay, Yin, Lu, Zhang, Zhenyu, Liu, Shiwei, Zhao, Jiawei, Tian, Yuandong, Wang, Zhangyang
Modern Large Language Models (LLMs) are composed of matrices with billions of elements, making their storage and processing quite demanding in terms of computational resources and memory usage. Being significantly large, such matrices can often be ex
Externí odkaz:
http://arxiv.org/abs/2407.11239
Autor:
Zhang, Zhenyu, Jaiswal, Ajay, Yin, Lu, Liu, Shiwei, Zhao, Jiawei, Tian, Yuandong, Wang, Zhangyang
Training Large Language Models (LLMs) is memory-intensive due to the large number of parameters and associated optimization states. GaLore, a recent method, reduces memory usage by projecting weight gradients into a low-rank subspace without compromi
Externí odkaz:
http://arxiv.org/abs/2407.08296
Autoregressive Large Language Models (e.g., LLaMa, GPTs) are omnipresent achieving remarkable success in language understanding and generation. However, such impressive capability typically comes with a substantial model size, which presents signific
Externí odkaz:
http://arxiv.org/abs/2404.03865
Autor:
Hong, Junyuan, Duan, Jinhao, Zhang, Chenhui, Li, Zhangheng, Xie, Chulin, Lieberman, Kelsey, Diffenderfer, James, Bartoldson, Brian, Jaiswal, Ajay, Xu, Kaidi, Kailkhura, Bhavya, Hendrycks, Dan, Song, Dawn, Wang, Zhangyang, Li, Bo
Compressing high-capability Large Language Models (LLMs) has emerged as a favored strategy for resource-efficient inferences. While state-of-the-art (SoTA) compression methods boast impressive advancements in preserving benign task performance, the p
Externí odkaz:
http://arxiv.org/abs/2403.15447
Graph Neural Networks (GNNs) have empowered the advance in graph-structured data analysis. Recently, the rise of Large Language Models (LLMs) like GPT-4 has heralded a new era in deep learning. However, their application to graph data poses distinct
Externí odkaz:
http://arxiv.org/abs/2402.08170
Autor:
Holste, Gregory, Zhou, Yiliang, Wang, Song, Jaiswal, Ajay, Lin, Mingquan, Zhuge, Sherry, Yang, Yuzhe, Kim, Dongkyun, Nguyen-Mau, Trong-Hieu, Tran, Minh-Triet, Jeong, Jaehyup, Park, Wongi, Ryu, Jongbin, Hong, Feng, Verma, Arsh, Yamagishi, Yosuke, Kim, Changhyun, Seo, Hyeryeong, Kang, Myungjoo, Celi, Leo Anthony, Lu, Zhiyong, Summers, Ronald M., Shih, George, Wang, Zhangyang, Peng, Yifan
Many real-world image recognition problems, such as diagnostic medical imaging exams, are "long-tailed" $\unicode{x2013}$ there are a few common findings followed by many more relatively rare conditions. In chest radiography, diagnosis is both a long
Externí odkaz:
http://arxiv.org/abs/2310.16112
Autor:
Yin, Lu, Wu, You, Zhang, Zhenyu, Hsieh, Cheng-Yu, Wang, Yaqing, Jia, Yiling, Li, Gen, Jaiswal, Ajay, Pechenizkiy, Mykola, Liang, Yi, Bendersky, Michael, Wang, Zhangyang, Liu, Shiwei
Large Language Models (LLMs), renowned for their remarkable performance across diverse domains, present a challenge when it comes to practical deployment due to their colossal model size. In response to this challenge, efforts have been directed towa
Externí odkaz:
http://arxiv.org/abs/2310.05175
Despite their remarkable achievements, modern Large Language Models (LLMs) face exorbitant computational and memory footprints. Recently, several works have shown significant success in training-free and data-free compression (pruning and quantizatio
Externí odkaz:
http://arxiv.org/abs/2310.01382
We present Junk DNA Hypothesis by adopting a novel task-centric angle for the pre-trained weights of large language models (LLMs). It has been believed that weights in LLMs contain significant redundancy, leading to the conception that a considerable
Externí odkaz:
http://arxiv.org/abs/2310.02277