Part 13 - Brownian Motion and Gaussian Processes I

In a gas, atoms bump into each other and change course randomly. Over time, how does a single atom move, on average? If $f(x, t)$ represents the probability density for the position $x$ of an atom at time $t$ moving along a line, Albert Einstein showd that [^1],

Which has the solution,

This means that the position of the atom at time $t$ is normally distributed with mean $0$ and variance $t$, i.e., $X_t \sim \mathcal{N}(0, t)$.

These random movements are called Brownian motion (or Wiener process).

Brownian motion is a continuous-time stochastic process $\{B_t\}_{t \geq 0}$ with the following properties:

1. $B_0 = 0$.

2. For $t >0$, $B_t \sim \mathcal{N}(0, t)$ (i.e., normally distributed with mean $0$ and variance $t$).

3. For $s, t > 0$, $B_{t + s} - B_s \sim \mathcal{N}(0, t)$ (i.e., the increments are stationary).

4. For $0 \leq q < r \leq s < t$, $B_t - B_s$ is independent of $B_r - B_q$ (i.e., the increments are independent).

5. The function $t \mapsto B_t$ is continuous with probability $1$ (almost surely).

Given time points $0 \eqqcolon t_0 < t_1 < t_2 < \ldots < t_n$, we write for $i > 0$,

where $Z_i \sim \mathcal{N}(0, t_i - t_{i - 1})$.

Thus, we get for independent $Z_1, Z_2, \ldots, Z_n$,

This gives a simple way to simulate Brownian motion at discrete time points. A good way to simulate the path $t \mapsto B_t$ on $t \in [0, a]$ is to set $t_i = \frac{a_i}{n}$, simulate independently,

and compute,

What if we have a Brownian motion path simulated as above, and want to plot it at twice the detail (i.e., at $2n$ points instead of $n$)? We have,

Reformulating using $Z_i = B_{t_{i + 1}} - B_{t_i}$, we have,

Since we have Normal-Normal conjugacy,

So we get $B_{t_{i + \frac{a}{2n}}}$ by simulating $Z_{i0}$ given $Z_i$ as above and adding the result to $B_{t_i}$.

To compute probabilities for Brownian motions, we generally use the properties of the Normal distribution and the independent increments.

A random walk is a discrete-time Markov chain $S_0, S_1, S_2, \ldots$ where $S_0 = 0$ and,

and $Y_1, Y_2, \ldots$ are i.i.d. random variables. Assume $\mathbb{E}[Y_i] = 0$.

Further, if we assume $\mathrm{Var}(Y_i) = 1$, we get $\mathrm{Var}(S_n) = n$.

Interpolating between the values $S_n$ we can make this into a continuous-time process $S_t$ where $\mathrm{Var}(S_t) \approx t$.

We may scale with an $s > 0$ to get processes $S_t^{(s)} = \frac{S_{st}}{\sqrt{s}}$ where we get $\lim_{s \to \infty} \mathrm{Var}(S_t^{(s)}) = t$.

It turns out that the processes $S_t^{(s)}$ when $s \to \infty$ are exactly Brownian motion, no matter what type of $Y_i$ we start with.

This is the Donsker’s invariance principle, which is a generalization of the central limit theorem to stochastic processes [^2].

A set of random variables $X_1, X_2, \ldots, X_k$ has a multivariate normal distribution if, for all real $a_1, a_2, \ldots, a_k$, $a_1 X_1 + a_2 X_2 + \ldots + a_k X_k$ is normally distributed.

It is completely determined by the expectation vector $\mu = (\mathbb{E}[X_1], \mathbb{E}[X_2], \ldots, \mathbb{E}[X_k])$ and the $(k \times k)$ covariance matrix $\Sigma$ where $\Sigma_{ij} = \mathrm{Cov}(X_i, X_j)$.

The joint density function on the vector $x = (x_1, x_2, \ldots, x_k)$ is given by,

where $|2 \pi \Sigma|$ is the determinant of the matrix $2 \pi \Sigma$.

A Gaussian process is a continuous-time stochastic process $\{X_t\}_{t \geq 0}$ with the property that for all $n \geq 1$ and $0 \leq t_1 < t_2 < \ldots < t_n$, $X_{t_1}, X_{t_2}, \ldots, X_{t_n}$ have a multivariate normal distribution.

Thus, a Gaussian process is completely determined by its mean function $\mathbb{E}[X_t]$ and its covariance function $\mathrm{Cov}(X_s, X_t)$.

Brownian motion is a Gaussian process, as we can show that any $a_1 B_{t_1} + a_2 B_{t_2} + \ldots + a_n B_{t_n}$ is normally distributed.

A Gaussian process $\{X_t\}_{t \geq 0}$ is a Brownian motion if and only if,

1. $X_0 = 0$.

2. $\mathbb{E}[X_t] = 0$ for all $t$.

3. $\mathrm{Cov}(X_s, X_t) = \min(s, t)$ for all $s, t$.

4. The function $t \mapsto X_t$ is continuous with probability $1$ (almost surely).

The following transformations of Brownian motion also yield Brownian motion:

• $\{-B_t\}_{t \geq 0}$, negating the process yields another (reflected) Brownian motion.
• $\{B_{t + s} - B_s\}_{t \geq 0}$ for any fixed $s \geq 0$, shifting the time origin yields another Brownian motion.
• $\{\frac{1}{\sqrt{a}} B_{a t}\}_{t \geq 0}$ for any fixed $a > 0$, scaling time and space yields another Brownian motion.
• The process $\{X_t\}_{t \geq 0}$ where $X_0 = 0$ and $X_t = t B_{\frac{1}{t}}$ for $t > 0$, time inversion yields another Brownian motion.

We saw above that, for any fixed $t$ $(B_{t + s} - B_s)_{t \geq 0}$ is a Brownian motion. Does thius phenomenon also hold if we start the chain anew from $T$ when $T$ is random? It depends.

If $T$ is the largest value less than 1 where $B_T = 0$, then $(B_{T + s} - B_T)_{s \geq 0}$ is not a Brownian motion.

If $T$ is the smallest value where $B_T = a$ for some constant $a$, then $(B_{T + s} - B_T)_{s \geq 0}$ is a Brownian motion. The reason is that the event $T = t$ can be determined based on $B_r$ where $0 \leq r \leq t$.

Random $T$’s that have this property are called stopping times. For these $B_{T + s} - B_T$ is a Brownian motion.

Given that $a \neq 0$, what is the distribution of the first hitting time $T_a = \min\{t : B_t = a\}$?

We will prove that,

Assuming that $a > 0$ and using that $T_a$ is a stopping time we get for any $t > 0$ that $P(B_{\frac{1}{t}} > a \mid T_a < \frac{1}{t}) = P(B_{\frac{1}{t} - T_a} > 0) = \frac{1}{2}$.

We also have,

Further, it follows that $P(T_a < \frac{1}{t}) = 2 P(B_{\frac{1}{t}} > a)$ and thus,

Taking the derivative with respect to $t$ we get the Gamma density,

We can define $M_t \coloneqq \max_{0 \leq s \leq t} B_s$.

We may compute for $a > 0$ (using the results above),

Thus, $M_t$ has the same d istribution as $|B_t|$, i.e., the absolute value of $B_t$.