Zobrazeno 1 - 10
of 123
pro vyhledávání: '"Wei, Susan"'
The Cold Posterior Effect (CPE) is a phenomenon in Bayesian Deep Learning (BDL), where tempering the posterior to a cold temperature often improves the predictive performance of the posterior predictive distribution (PPD). Although the term `CPE' sug
Externí odkaz:
http://arxiv.org/abs/2410.05757
Autor:
Ng, Kenyon, Wei, Susan
Variational inference in Bayesian deep learning often involves computing the gradient of an expectation that lacks a closed-form solution. In these cases, pathwise and score-function gradient estimators are the most common approaches. The pathwise es
Externí odkaz:
http://arxiv.org/abs/2410.05753
Autor:
Munn, Michael, Wei, Susan
Recent advances in artificial intelligence have been fueled by the development of foundation models such as BERT, GPT, T5, and Vision Transformers. These models are first pretrained on vast and diverse datasets and then adapted to specific downstream
Externí odkaz:
http://arxiv.org/abs/2410.05612
Autor:
Hoogland, Jesse, Wang, George, Farrugia-Roberts, Matthew, Carroll, Liam, Wei, Susan, Murfet, Daniel
We show that in-context learning emerges in transformers in discrete developmental stages, when they are trained on either language modeling or linear regression tasks. We introduce two methods for detecting the milestones that separate these stages,
Externí odkaz:
http://arxiv.org/abs/2402.02364
Fair machine learning aims to prevent discrimination against individuals or sub-populations based on sensitive attributes such as gender and race. In recent years, causal inference methods have been increasingly used in fair machine learning to measu
Externí odkaz:
http://arxiv.org/abs/2401.10632
We investigate phase transitions in a Toy Model of Superposition (TMS) using Singular Learning Theory (SLT). We derive a closed formula for the theoretical loss and, in the case of two hidden dimensions, discover that regular $k$-gons are critical po
Externí odkaz:
http://arxiv.org/abs/2310.06301
The Local Learning Coefficient (LLC) is introduced as a novel complexity measure for deep neural networks (DNNs). Recognizing the limitations of traditional complexity measures, the LLC leverages Singular Learning Theory (SLT), which has long recogni
Externí odkaz:
http://arxiv.org/abs/2308.12108
The ADMANI datasets (annotated digital mammograms and associated non-image datasets) from the Transforming Breast Cancer Screening with AI programme (BRAIx) run by BreastScreen Victoria in Australia are multi-centre, large scale, clinically curated,
Externí odkaz:
http://arxiv.org/abs/2305.12068
Autor:
Wei, Susan, Lau, Edmund
In this work, we advocate for the importance of singular learning theory (SLT) as it pertains to the theory and practice of variational inference in Bayesian neural networks (BNNs). To begin, using SLT, we lay to rest some of the confusion surroundin
Externí odkaz:
http://arxiv.org/abs/2302.06035
Fair machine learning aims to avoid treating individuals or sub-populations unfavourably based on \textit{sensitive attributes}, such as gender and race. Those methods in fair machine learning that are built on causal inference ascertain discriminati
Externí odkaz:
http://arxiv.org/abs/2205.13972