A Likelihood Approach to Nonparametric Estimation of a Singular Distribution Using Deep Generative Models
Minwoo Chae, Dongha Kim, Yongdai Kim, Lizhen Lin; 24(77):1−42, 2023.
Abstract
This study explores the statistical properties of a likelihood approach to nonparametric estimation of a singular distribution using deep generative models. The aim is to model high-dimensional data that are concentrated around a low-dimensional structure using a deep generative model. Estimating the distribution supported on this low-dimensional structure, such as a low-dimensional manifold, is challenging due to its singularity with respect to the Lebesgue measure in the ambient space. In this model, the usual likelihood approach may fail to consistently estimate the target distribution due to the singularity. However, we prove that a novel and effective solution can be achieved by perturbing the data with instance noise, leading to consistent estimation of the underlying distribution with desirable convergence rates. Furthermore, we characterize the class of distributions that can be efficiently estimated using deep generative models. This class is broad enough to include various structured distributions such as product distributions, classically smooth distributions, and distributions supported on a low-dimensional manifold. Our analysis provides insights into how deep generative models can overcome the curse of dimensionality in nonparametric distribution estimation. We conduct extensive simulation studies and real data analysis to empirically demonstrate that the proposed data perturbation technique significantly improves estimation performance.
[abs]