Surrogate to Poincaré inequalities on manifolds for dimension reduction in nonlinear feature spaces

Abstract We aim to approximate a continuously differentiable function $u:\mathbb{R}^{d} \rightarrow \mathbb{R}$ by a composition of functions $f\circ g$ where $g:\mathbb{R}^{d} \rightarrow \mathbb{R}^{m}$, $m\leq d$ and $f: \mathbb{R}^{m} \rightarrow \mathbb{R}$ are built in a two stage procedure. For a fixed $g$, we build $f$ using classical regression methods, involving evaluations of $u$. Recent works proposed to build a nonlinear $g$ by minimizing a loss function $\mathcal{J}(g)$ derived from Poincaré inequalities on manifolds, involving evaluations of the gradient of $u$. A problem is that minimizing $\mathcal{J}$ may be a challenging task. Hence in this work, we introduce new convex surrogates to $\mathcal{J}$. Leveraging concentration inequalities, we provide suboptimality results for a class of functions $g$, including polynomials, and a wide class of input probability measures. We investigate performances on different benchmarks for various training sample sizes. We show that our approach outperforms standard iterative methods for minimizing the training Poincaré inequality-based loss, often resulting in better approximation errors, especially for small training sets and $m=1$.