On learning higher-order cumulants in diffusion models
| Autor: | Aarts, Gert, Habibi, Diaa E., Wang, Lingxiao, Zhou, Kai |
|---|---|
| Rok vydání: | 2024 |
| Předmět: | |
| Druh dokumentu: | Working Paper |
| Popis: | To analyse how diffusion models learn correlations beyond Gaussian ones, we study the behaviour of higher-order cumulants, or connected n-point functions, under both the forward and backward process. We derive explicit expressions for the moment- and cumulant-generating functionals, in terms of the distribution of the initial data and properties of forward process. It is shown analytically that during the forward process higher-order cumulants are conserved in models without a drift, such as the variance-expanding scheme, and that therefore the endpoint of the forward process maintains nontrivial correlations. We demonstrate that since these correlations are encoded in the score function, higher-order cumulants are learnt in the backward process, also when starting from a normal prior. We confirm our analytical results in an exactly solvable toy model with nonzero cumulants and in scalar lattice field theory. Comment: 21 pages, many figures. Extended version of contribution accepted in the NeurIPS 2024 workshop "Machine Learning and the Physical Sciences" |
| Databáze: | arXiv |
| Externí odkaz: |
|