User:Mim.cis/sandbox/Sobolev-RKHS-CA

Sobolev Smoothness and Reproducing Kernel Hilbert Space
The amount of smoothness was examined by Dupuis et.al. and Trouve using the Sobolev embedding theorem to demonstrate the necessary conditions for constraining the vector fields  $$ v \in V $$ to be in Hilbert space  which is embedded in functions with at least once continuous derivative. The norm of the Hilbert space is defined via a differential operator so as to penalize derivatives in integral-square; proper choice of number of derivatives implies continuous vector fields with flows which are smooth. The Sobolev condition for 1-continuous derivatives for volumes $${\mathbb R}^3$$ is that  $$m \geq 3/2+1$$ square-integral derivatives must exist, requiring each component of the vector field to have finite Sobolev norm with 3 derivatives square-integrable $$\|v_i\|_{H_0^3}^2 < \infty,i=1,2,3.$$ The Hilbert space   norm $$\| v_i \|_V$$ is constructed from a one-to-one differential operator $$L=(\nabla^2-c)^p, p\geq 1.5 $$  to dominate the Sobolev norm

The Sobolev embedding theorem dictates how much differentiation is required so that the space of vector field is continuously embedded in 1-times differentiable vector fields vanishing at infinity $$ V \rightarrow C_0^1({\mathbb R}^3). $$ The Hilbert space of vector field $$ (V,\| \cdot \|_V ) $$ is constructed with inner-product defined via a one-to-one differential operator $$ A : V \rightarrow V^* $$, with $$ V^* $$ the dual space. The dual space contains generalized vector functions or distributions $$ \sigma \in V^* $$, for $$ X \subset {\mathbb R}^3 $$, then $$ \sigma= (\sigma_1,\sigma_2,\sigma_3) \in V^*,$$ with $$ ( \sigma |f ) \doteq \int_X \sum_{i=1}^3 f_i(x) \sigma_i (dx) \, \ for \ f \in V \. $$

We choose our Hilbert space $$ (V,\| \cdot \|_V) $$ with norm so that it dominates the Sobolev norm of proper order, for $$ {\mathbb R}^3 $$ then $$ \| v_t \|_V^2 \doteq (Av_t|v_t) \geq \| v_t \|_{H_0^m({\mathbb R}^3)}^2 ; $$ finiteness of $$ (Av_t | v_t) < \infty $$ implies the Sobolev norm is finite. For d-dimensional backround space $$ X \subset {\mathbb R}^d$$, the Sobolev norm associated to the d-components $$ v_{it},i=1,\dots, d $$, the necessary condition for smooth embedding with k-derivatives, $$V \subset C_0^k({\mathbb R}^d,{\mathbb R}^d)$$ must satisfy $$ m \geq k+d/2 $$

For 1-continuous derivative, the backround space $$ X \subset {\mathbb R}^2 $$, then $$ m\geq 2 $$; for $$ X \subset {\mathbb R}^3 $$, then $$ m \geq 2.5 $$.

In CA, a modelling approach used as in other branches of machine learning is to model the Hilbert space of vector fields as a reproducing kernel Hilbert space (RKHS). The construction begins by defining the squared operator $$ A = L L^\dagger $$, $$L^\dagger$$ the adjoint of $$L$$. The Hilbert space inner-product on $$ \ v,w \in V $$ becomes $$ \langle v, w \rangle_V \doteq (\sigma|w), \sigma \doteq Av \in V^* $$; since$$ A : V \rightarrow V^* $$, $$ V^* $$ the dual space of $$V$$, then $$Av$$ can be a generalized function with the linear form definedas$$(\sigma |v)\doteq \int_X \sum_i v_i(x) \sigma_i(dx)$$. For proper choice of differential operators, then $$ (V,\| \cdot \|_V) $$ is an RKHS with kernel operator $$ K = A^{-1} $$. The kernel smooths$$ K\sigma (x)_i \doteq \sum_j \int_X k_{ij}(x-y) \sigma_j(dy) $$, with kernel $$ k $$.

One operator choice for the norm is the Laplacian; in $${\mathbb R}^3$$ choose, $$A=(\nabla^2-id)^3$$ for which  $$(Av|v) < \infty$$ implies 1 continuous spatial derivative for the kernel
 * $$k(x-y) =(1+\|x-y \|) e^{-\|x-y\|}, \, K(x-y) = k(x-y) Id_3 \, $$,

with $$Id_3 $$ the 3x3 identity matrix. See /Sobolev-RKHS-CA for more details on the reproducing kernel Hilbert space formulation and the conditions for $$ {\mathbb R}^3 $$.

The smoothness required for Equations($$,$$) results from the fact that the kernels  $$ k(x,y) $$ are continuously differentiable in both variables.

Our smoothness condition for smooth flows of the inverse requires control of the first derive $$ v_t \rightarrow Dv_t(x), x \in X $$ which is true for smooth kernel $$ k_{ij}(x,y) $$ in both variables $$ x,y $$.

We require $$ v \in V $$ a Hilbert space which continuously embeds in 1-times differentiable vector fields vanishing at infinity giving the group of diffeomorphisms generated from smooth flows:

Diff_V \doteq \{\varphi=\phi_1: \dot \phi_t = v_t \circ \phi_t, \phi_0 = id, \int_0^1 \|v_t \|_V dt < \infty \}. $$ For proper choice on the operator, then $$ (V,\| \cdot \|_V) $$ is a reproducing kernel Hilbert space with the reproducing kernel $$ K = A^{-1} $$, implying $$ KAv = v $$. Therefore the operator smooths distributions $$ K: V^* \rightarrow V $$ with the kernel  $$ K(x,y) = ( k_{ij}(x,y)) $$   and  $$ K\sigma (x)_i \doteq \sum_j \int_X k_{ij}(x, y) \sigma_j(dy) \. $$ One example, $$X \subset {\mathbb R}^{3}$$, then d=3, p=3, $$A=(\nabla^2 -id)^3$$, Green's operator $$ K(x,y)=k(x-y) Id_3, Id_3 $$  $$k(x-y) =(1+\|x-y\|) e^{-\|x-y\|}$$ .with $$ Id_3 $$diagonal 3x3 identity.