On the Relationship Between Distance and Kernel Measures of Conditional Dependence
Tianhong Sheng, Bharath K. Sriperumbudur; 24(7):1−16, 2023.
Abstract
Measuring conditional dependence is a crucial task in statistical inference and plays a fundamental role in various areas such as causal discovery, feature selection, dimensionality reduction, Bayesian network learning, and more. In this study, we investigate the relationship between conditional dependence measures induced by distances on a metric space and reproducing kernels associated with a reproducing kernel Hilbert space (RKHS). We demonstrate that for specific pairs of distance and kernel, the distance-based conditional dependence measures are equivalent to the kernel-based measures. However, we also show that certain popular kernel conditional dependence measures, which are based on the Hilbert-Schmidt norm of a cross-conditional covariance operator, do not have a straightforward representation in terms of distance, except in certain limiting cases.
[abs]