Draft:Category of Markov kernels

In mathematics, the category of Markov kernels, often denoted Stoch, is the category whose objects are measurable spaces and whose morphisms are Markov kernels. It is analogous to the category of sets and functions, but where the arrows can be interpreted as being stochastic.

Several variants of this category are used in the literature. For example, one can use subprobability kernels instead of probability kernels, or more general s-finite kernels.

Definition
Recall that a Markov kernel between measurable spaces $$(X,\mathcal{F})$$ and $$(Y,\mathcal{G})$$ is an assignment $$k:X\times\mathcal{G}\to\mathbb{R}$$ which is measurable as a function on $$X$$ and which is a probability measure on $$\mathcal{G}$$. We denote its values by $$k(B|x)$$ for $$x\in X$$ and $$B\in\mathcal{G}$$, which suggests an interpretation as conditional probability.

The category Stoch has:


 * As objects, measurable spaces;
 * As morphisms, Markov kernels between them;
 * For each measurable space $$(X,\mathcal{F})$$, the identity morphism is given by the kernel

\delta(A|x) = 1_A(x) = \begin{cases} 1 & x\in A ; \\ 0 & x\notin A \end{cases} $$
 * for all $$x\in X$$ and $$A\in\mathcal{F}$$;


 * Given kernels $$k:(X,\mathcal{F})\to(Y,\mathcal{G})$$ and $$h:(Y,\mathcal{G})\to(Z,\mathcal{H})$$, the composite morphism $$h\circ k:(X,\mathcal{F})\to(Z,\mathcal{H})$$ is given by

(h\circ k) (C|x) = \int_Y h(C|y) \, k(dy|x) $$
 * for all $$x\in X$$ and $$C\in\mathcal{H}$$.

This composition formula is sometimes called the Chapman-Kolmogorov equation.

This composition is unital, and associative by the monotone convergence theorem, so that one indeed has a category.

Probability measures
The terminal object of Stoch is the one-point space $$1$$. Morphisms in the form $$1\to X$$ can be equivalently seen as probability measures on $$X$$, since they correspond to functions $$1\to PX$$, i.e. elements of $$PX$$.

Given kernels $$p:1\to X$$ and $$k:X\to Y$$, the composite kernel $$k\circ p:1\to Y$$ gives the probability measure on $$Y$$ with values

(k\circ p) (B) = \int_X k(B|x)\,p(dx) , $$ for every measurable subset $$B$$ of $$Y$$.

Given probability spaces $$(X,\mathcal{F},p)$$ and $$(Y,\mathcal{G},q)$$, a measure-preserving Markov kernel $$(X,\mathcal{F},p)\to(Y,\mathcal{G},q)$$ is a Markov kernel $$k:(X,\mathcal{F})\to(Y,\mathcal{G})$$ such that for every measurable subset $$B\in\mathcal{G}$$,

q(B) = \int_X k(B|x) \, p(dx). $$

Probability spaces and measure-preserving Markov kernels form a category, which can be seen as the slice category $$(\mathrm{Hom}_\mathrm{Stoch}(1,-),\mathrm{Stoch})$$.

Measurable functions
Every measurable function $$f:(X,\mathcal{F})\to(Y,\mathcal{G})$$ defines canonically a Markov kernel $$\delta_f:(X,\mathcal{F})\to(Y,\mathcal{G})$$ as follows,

\delta_f(B|x) = 1_B(f(x)) = \begin{cases} 1 & f(x)\in B ; \\ 0 & f(x)\notin B \end{cases} $$ for every $$x\in X$$ and every $$B\in\mathcal{G}$$. This construction preserves identities and compositions, and is therefore a functor from Meas to Stoch.

Isomorphisms
By functoriality, every isomorphism of measurable spaces (in the category Meas) induces an isomorphism in Stoch. However, in Stoch there are more isomorphisms, and in particular, measurable spaces can be isomorphic in Stoch even when the underlying sets are not in bijection.

Relationship with other categories

 * Stoch is the Kleisli category of the Giry monad. This in particular implies that there is an adjunction

\mathrm{Hom}_\mathrm{Stoch}(X,Y) \cong \mathrm{Hom}_\mathrm{Meas}(X,PY) $$
 * between Stoch and the category of measurable spaces.


 * The left adjoint $$L:\mathrm{Meas}\to\mathrm{Stoch}$$ of the adjunction above is the identity on objects, and on morphisms it gives the canonical Markov kernel induced by a measurable function described above.


 * As mentioned above, one can construct a category of probability spaces and measure-preserving Markov kernels as the slice category $$(\mathrm{Hom}_\mathrm{Stoch}(1,-),\mathrm{Stoch})$$.


 * Similarly, the category of probability spaces can be seen as the comma category $$(\mathrm{Hom}_\mathrm{Stoch}(1,-),L)$$.

Particular limits and colimits
Since the functor $$L:\mathrm{Meas}\to\mathrm{Stoch}$$ is left adjoint, it preserves colimits. Because of this, all colimits in the category of measurable spaces are also colimits in Stoch. For example,


 * The initial object is the empty set, with its trivial measurable structure;
 * The coproduct is given by the disjoint union of measurable spaces, with its canonical sigma-algebra.
 * The sequential colimit of a decreasing filtration is given by the intersection of sigma-algebras.

In general, the functor $$L$$ does not preserve limits. This in particular implies that the product of measurable spaces is not a product in Stoch in general. Since the Giry monad is monoidal, however, the product of measurable spaces still makes Stoch a monoidal category.