Zobrazeno 1 - 10
of 29
pro vyhledávání: '"Hajimolahoseini, Habib"'
Tensor decomposition is a mathematically supported technique for data compression. It consists of applying some kind of a Low Rank Decomposition technique on the tensors or matrices in order to reduce the redundancy of the data. However, it is not a
Externí odkaz:
http://arxiv.org/abs/2407.20266
In this paper, we present a comprehensive study and propose several novel techniques for implementing 3D convolutional blocks using 2D and/or 1D convolutions with only 4D and/or 3D tensors. Our motivation is that 3D convolutions with 5D tensors are c
Externí odkaz:
http://arxiv.org/abs/2407.16514
This paper introduces a novel method of Progressive Low Rank Decomposition (PLRD) tailored for the compression of large language models. Our approach leverages a pre-trained model, which is then incrementally decompressed to smaller sizes using progr
Externí odkaz:
http://arxiv.org/abs/2406.19995
Autor:
Ataiefard, Foozhan, Ahmed, Walid, Hajimolahoseini, Habib, Asani, Saina, Javadi, Farnoosh, Hassanpour, Mohammad, Awad, Omar Mohamed, Wen, Austin, Liu, Kangling, Liu, Yang
Vision transformers are known to be more computationally and data-intensive than CNN models. These transformer models such as ViT, require all the input image tokens to learn the relationship among them. However, many of these tokens are not informat
Externí odkaz:
http://arxiv.org/abs/2401.15293
Autor:
Javadi, Farnoosh, Ahmed, Walid, Hajimolahoseini, Habib, Ataiefard, Foozhan, Hassanpour, Mohammad, Asani, Saina, Wen, Austin, Awad, Omar Mohamed, Liu, Kangling, Liu, Yang
Massive transformer-based models face several challenges, including slow and computationally intensive pre-training and over-parametrization. This paper addresses these challenges by proposing a versatile method called GQKVA, which generalizes query,
Externí odkaz:
http://arxiv.org/abs/2311.03426
Compression of a neural network can help in speeding up both the training and the inference of the network. In this research, we study applying compression using low rank decomposition on network layers. Our research demonstrates that to acquire a sp
Externí odkaz:
http://arxiv.org/abs/2309.12412
Autor:
Awad, Omar Mohamed, Hajimolahoseini, Habib, Lim, Michael, Gosal, Gurpreet, Ahmed, Walid, Liu, Yang, Deng, Gordon
This paper presents our proposed approach that won the first prize at the ICLR competition on Hardware Aware Efficient Training. The challenge is to achieve the highest possible accuracy in an image classification task in less than 10 minutes. The tr
Externí odkaz:
http://arxiv.org/abs/2309.03965
Low Rank Decomposition (LRD) is a model compression technique applied to the weight tensors of deep learning models in order to reduce the number of trainable parameters and computational complexity. However, due to high number of new layers added to
Externí odkaz:
http://arxiv.org/abs/2309.03824
Autor:
Li, Tianda, Mesbahi, Yassir El, Kobyzev, Ivan, Rashid, Ahmad, Mahmud, Atif, Anchuri, Nithin, Hajimolahoseini, Habib, Liu, Yang, Rezagholizadeh, Mehdi
Pre-trained Language Models (PLMs) have been successful for a wide range of natural language processing (NLP) tasks. The state-of-the-art of PLMs, however, are extremely large to be used on edge devices. As a result, the topic of model compression ha
Externí odkaz:
http://arxiv.org/abs/2110.08460
Publikováno v:
In Artificial Intelligence In Medicine April 2018 85:7-15