Výsledky vyhledávání - "Chu, Tianzhe"

Report

White-Box Transformers via Sparse Rate Reduction: Compression Is All There Is?

Autor: Yu, Yaodong, Buchanan, Sam, Pai, Druv, Chu, Tianzhe, Wu, Ziyang, Tong, Shengbang, Bai, Hao, Zhai, Yuexiang, Haeffele, Benjamin D., Ma, Yi

In this paper, we contend that a natural objective of representation learning is to compress and transform the distribution of the data, say sets of tokens, towards a low-dimensional Gaussian mixture supported on incoherent subspaces. The goodness of

Externí odkaz: http://arxiv.org/abs/2311.13110

Zobrazit plný text záznamu

Report

Emergence of Segmentation with Minimalistic White-Box Transformers

Autor: Yu, Yaodong, Chu, Tianzhe, Tong, Shengbang, Wu, Ziyang, Pai, Druv, Buchanan, Sam, Ma, Yi

Transformer-like models for vision tasks have recently proven effective for a wide range of downstream applications such as segmentation and detection. Previous works have shown that segmentation properties emerge in vision transformers (ViTs) traine

Externí odkaz: http://arxiv.org/abs/2308.16271

Zobrazit plný text záznamu

Report

Image Clustering via the Principle of Rate Reduction in the Age of Pretrained Models

Autor: Chu, Tianzhe, Tong, Shengbang, Ding, Tianjiao, Dai, Xili, Haeffele, Benjamin David, Vidal, René, Ma, Yi

The advent of large pre-trained models has brought about a paradigm shift in both visual representation learning and natural language processing. However, clustering unlabeled images, as a fundamental and classic machine learning problem, still lacks

Externí odkaz: http://arxiv.org/abs/2306.05272

Zobrazit plný text záznamu

Report

White-Box Transformers via Sparse Rate Reduction

Autor: Yu, Yaodong, Buchanan, Sam, Pai, Druv, Chu, Tianzhe, Wu, Ziyang, Tong, Shengbang, Haeffele, Benjamin D., Ma, Yi

In this paper, we contend that the objective of representation learning is to compress and transform the distribution of the data, say sets of tokens, towards a mixture of low-dimensional Gaussian distributions supported on incoherent subspaces. The

Externí odkaz: http://arxiv.org/abs/2306.01129

Zobrazit plný text záznamu