Markov kernel

In probability theory, a Markov kernel (also known as a stochastic kernel or probability kernel) is a map that plays the role, in the general theory of Markov processes, that the transition matrix does in the theory of Markov processes with a finite state space.

Formal definition

Let $(X,{\mathcal {A}}),(Y,{\mathcal {B}})$ be measurable spaces. A Markov kernel with source $(X,{\mathcal {A}})$ and target $(Y,{\mathcal {B}})$ is a map $\kappa :X\times {\mathcal {B}}\to [0,1]$ with the following properties:

1. The map $x\mapsto \kappa (x,B)$ is ${\mathcal {A}}$-measurable for every $B\in {\mathcal {B}}.$
2. The map $B\mapsto \kappa (x,B)$ is a probability measure on $(Y,{\mathcal {B}})$ for every $x\in X$.

In other words it associates to each point $x\in X$ a probability measure $\kappa (x,\cdot )$ on $(Y,{\mathcal {B}})$ such that, for every measurable set $B\in {\mathcal {B}}$, the map $x\mapsto \kappa (x,B)$ is measurable with respect to the $\sigma$-algebra ${\mathcal {A}}.$

Examples

Simple random walk

Take $X=Y=\mathbb {Z} ,{\mathcal {A}}={\mathcal {B}}={\mathcal {P}}(\mathbb {Z} )$ (the power set of $\mathbb {Z}$), then the Markov kernel $\kappa$ with

$\kappa (x,B)={\frac {1}{2}}\mathbf {1} _{B}(x-1)+{\frac {1}{2}}\mathbf {1} _{B}(x+1),\quad \forall x\in \mathbb {Z} ,\quad \forall B\in {\mathcal {P}}(\mathbb {Z} ),$

where $\mathbf {1}$ is the indicator function, describes the transition rule for the random walk on $\mathbb {Z} .$

Galton–Watson process

Take $X=Y=\mathbb {N} ,{\mathcal {A}}={\mathcal {B}}={\mathcal {P}}(\mathbb {N} ),$ then

$\kappa (x,B)={\begin{cases}\mathbf {1} _{B}(0)&x=0\\\Pr(\xi _{1}+\cdots +\xi _{x}\in B)&x\neq 0\\\end{cases}}$

with i.i.d. random variables $\xi _{i}$.

General Markov processes with finite state space

Take $X=Y,{\mathcal {A}}={\mathcal {B}}={\mathcal {P}}(X)={\mathcal {P}}(Y)$ and $|X|=|Y|=n,$ then the transition rule can be represented as a stochastic matrix $(K_{ij})_{1\leq i,j\leq n}$ with

$\forall i\in X:\qquad \sum _{j\in Y}K_{ij}=1.$

In the convention of Markov kernels we write

$\kappa (i,B)=\sum _{j\in B}K_{ij},\qquad \forall i\in X,\quad \forall B\in {\mathcal {B}}$.

Construction of a Markov kernel

If $\nu$ is a finite measure on $(Y,{\mathcal {B}})$ and $k:X\times Y\to \mathbb {R} _{+}$ is a measurable function with respect to the product $\sigma$-algebra ${\mathcal {A}}\otimes {\mathcal {B}}$ and has the property

$\forall x\in X\qquad \int _{Y}k(x,y)\nu (\mathrm {d} y)=1,$

then the mapping

${\begin{cases}\kappa :X\times {\mathcal {B}}\to [0,1]\\\kappa (x,B)=\int _{B}k(x,y)\nu (\mathrm {d} y)\end{cases}}$

defines a Markov kernel.

Properties

Semidirect product

Let $(X,{\mathcal {A}},P)$ be a probability space and $\kappa$ a Markov kernel from $(X,{\mathcal {A}})$ to some $(Y,{\mathcal {B}})$. Then there exists a unique measure $Q$ on $(X\times Y,{\mathcal {A}}\otimes {\mathcal {B}})$, such that:

$Q(A\times B)=\int _{A}\kappa (x,B)\,dP(x),\quad \forall A\in {\mathcal {A}},\quad \forall B\in {\mathcal {B}}.$

Regular conditional distribution

Let $(S,Y)$ be a Borel space, $X$ a $(S,Y)$-valued random variable on the measure space $(\Omega ,{\mathcal {F}},P)$ and ${\mathcal {G}}\subseteq {\mathcal {F}}$ a sub-$\sigma$-algebra. Then there exists a Markov kernel $\kappa$ from $(\Omega ,{\mathcal {G}})$ to $(S,Y)$, such that $\kappa (\cdot ,B)$ is a version of the conditional expectation $\mathbb {E} [\mathbf {1} _{\{X\in B\}}\mid {\mathcal {G}}]$ for every $B\in Y$, i.e.

$P(X\in B\mid {\mathcal {G}})=\mathbb {E} \left[\mathbf {1} _{\{X\in B\}}\mid {\mathcal {G}}\right]=\kappa (\omega ,B),\qquad P{\text{-a.s.}}\,\,\forall B\in {\mathcal {G}}.$

It is called regular conditional distribution of $X$ given ${\mathcal {G}}$ and is not uniquely defined.

Generalizations

Transition kernels generalize Markov kernels in the sense that the map

$B\mapsto \kappa (x,B)$

is not necessarily a probability measure but can be any type of measure.

References

• Bauer, Heinz (1996), Probability Theory, de Gruyter, ISBN 3-11-013935-9

§36. Kernels and semigroups of kernels