Part 16 - Summary

Given the events $A$ and $B$, the conditional probability of $A$ given $B$ is defined as, $$ P(A \mid B) = \frac{P(A \cap B)}{P(B)} $$ The events $A$ and $B$ are independent if $P(A \cap B) = P(A) P(B)$, or equivalently, if $P(A \mid B) = P(A)$.

The law of total probability states that if $B_1, B_2, \ldots, B_n$ is a sequence of events that partitions S. Then, $$ P(A) = \sum_{i=1}^{n} P(A \cap B_i) = \sum_{i=1}^{n} P(A \mid B_i) P(B_i) $$ Thus, Bayes’ law follow directly from the definition of conditional probability and the law of total probability, $$ P(B \mid A) = \frac{P(A \mid B) P(B)}{P(A)} $$

State $j$ is accessible from state $i$ if $(P^n)_{ij} > 0$ for some $n \geq 0$. States $i$ and $j$ communicate if $i$ is accessible from $j$ and $j$ is accessible from $i$.

“Communication” is transitive, i.e., if $i$ communicates with $j$ and $j$ communicates with $k$, then $i$ communicates with $k$.

Communication is an equivalence relation, subdividing all states into communication classes. Communication classes can be found for example by drawing transition graphs.

Lastly, a Markov chain is irreducible if it has exactly one communication class.

For any discrete random variable $X$ taking values in $\{0, 1, 2, \ldots\}$, define the probability function $G_X(s)$ as, $$ G_X(s) = \mathbb{E}[s^X] = \sum_{k = 0}^{\infty} P(X = k) s^k $$ The series converges absolutely for $\Vert s \Vert \leq 1$. We assume $s$ is a real number in $[0, 1]$.

We get a 1-1 correspondence between probability vectors on $\{0, 1, 2, \ldots\}$ and functions represented by a series where the coefficients sum to 1.

Specifically, if $G_X(s) = G_Y(s)$ for all $s$ for random variables $X$ and $Y$, then $X$ and $Y$ have the same distribution.

For a vertex configuration $\sigma$ and a vertex $v$, let $\sigma_{-v}$ denote the part of $\sigma$ that does not involve $v$. Prpose a new configuration $\sigma^{\star}$ given an old configuration $\sigma$ by first choosing a vertex $v$, then, let $\sigma^{\star}$ be identical to $\sigma$ except possibly at $v$. Decide the spin at $v$ using the conditional distribution given $\sigma_{-v}$, $$ \begin{align*} \pi(\sigma^{\star} \mid \sigma_{-v}) & = \frac{\pi(\sigma_v = 1, \sigma_{-v})}{\pi(\sigma_{-v})} \newline & = \frac{\pi(\sigma_v = 1, \sigma_{-v})}{\pi(\sigma_v = 1, \sigma_{-v}) + \pi(\sigma_v = -1, \sigma_{-v})} \newline & = \frac{1}{1 + \frac{\pi(\sigma_v = -1, \sigma_{-v})}{\pi(\sigma_v = 1, \sigma_{-v})}} \newline & = \frac{1}{1 + \exp(-\beta E(\sigma_v = -1, \sigma_{-v}) + \beta E(\sigma_v = 1, \sigma_{-v}))} \newline & = \frac{1}{1 + \exp(\beta \sum_{v \sim w} \sigma_v \sigma_w \mid \sigma_v = -1 - \beta \sum_{v \sim w} \sigma_v \sigma_w \mid \sigma_v = 1)} \newline & = \frac{1}{1 + \exp(-2 \beta \sum_{v \sim w} \sigma_w)} \newline \end{align*} $$

Given an ergodic Markov chain with finite state space of size $k$ and limiting distribution $\pi$. The idea is that, given $n$ prove that $X_n$ actually has reached the limit distribution $\pi$.

If we can prove that the distribution at $X_n$ is independent of the starting state $X_0$, then we might be able to conclude that $X_n$ has the limiting distribution $\pi$.

Construct $k$ Markov chains that are dependent (“coupled”) but which are marginally Markov chains as above. If they start at the $k$ possible value at $X_0$ but have identical values at $X_n$, then we are done.

Consider the chains $X^{(j)}_{-n}, \ldots, X^{(j)}_0$ for $j = 1, 2, \ldots, k$. Instead of simulating $X^{(j)}_{i + 1}$ based on $X^{(j)}_i$ independently for each j, we define a function $g$ so that $X^{(j)}_{i + 1} = g(X^{(j)}_i, U_i)$ for all $j$ where $U_i \sim \mathrm{Uniform}(0, 1)$.

Thus, if two chains have identical values in $X_i$, they will also be identical in $X_{i + 1}$.

Thus, for a particular $n$, if all chains have not converged at $X_0$, we simulate $k$ chains from $X_{-2n}$ to $X_{-n}$. They might only hit a subset of the $k$ states at $X_{-n}$ and thus might coalesce to one state at $X_0$ using the old simulations. If not, double $n$ again.

Define $T_i$ as the time the continuous-time Markov chain $\{X_t\}_{t \geq 0}$ that always starts in state $i$ stays in $i$ before moving to a different state, so that for any $s > 0$, $$ P(T_i > s) = P(X_u = i, 0 \leq u \leq s) $$ The distribution of $T_i$ is memoryless and thus exponential. We define $q_i$ such that, $$ T_i \sim \mathrm{Exponential}(q_i) $$ Remember that this means that the average time the process stays in $i$ is $\frac{1}{q_i}$. The rate of transition out of the state is $q_i$.

We can define a new stochastic process by listing the states the chain visits. This will be a discrete time Markov chain called the embedded Markov chain with transition matrix $\tilde{P}$.

A way to describe a continuous-time Markov chain is to describe $k \times (k - 1)$ independent “alarm clocks”.

For states $i$ and $j$ so that $i \neq j$, let $q_{ij}$ be the parameter of an Exponentially distributed random variable representing the time until an “alarm clock” rings.

When in state $i$, wait until the first alarm clock $q_i$ rings, then move to the state given by the index $j$ of that alarm clock. This defines a continuous-time Markov chain.

The time until the first alarm clock $q_i$ rings is Exponentially distributed with parameter given by, $$ q_i = q_{i1} + q_{i2} + \ldots + q_{i, i - 1} + q_{i, i + 1} + \ldots + q_{ik}. $$ i.e., the parameter of the holding time distribution at $i$.

The chain is completely described by the rates $q_{ij}$.

Further, we saw also that the chain is completely described by the $p_{ij}$ and the $q_i$. The relationship is described by the equation above, and, $$ p_{ij} = \frac{q_{ij}}{q_{i1} + q_{i2} + \ldots + q_{i, i - 1} + q_{i, i + 1} + \ldots + q_{ik}} = \frac{q_{ij}}{q_i}. $$ for $i \neq j$.

To relate $P(t)$ to the $q_{ij}$‘s, we first relate them to $P^{\prime}(0)$.

Assuming $P(t)$ is differentiable, we can show that, $$ P^{\prime}(0) = \begin{bmatrix} -q_1 & q_{12} & q_{13} & \ldots & q_{1k} \newline q_{21} & -q_2 & q_{23} & \ldots & q_{2k} \newline \vdots & \vdots & \vdots & \ddots & \vdots \newline q_{k1} & q_{k2} & q_{k3} & \ldots & -q_k \newline \end{bmatrix} = Q, $$

Note that the rows of $P^{\prime}(0)$, i.e., $Q$, sum to zero!

The solution to the Kolmogorov Forward and Backward equations is given by the matrix exponential. For any square matrix $A$ we can define the matrix exponential as, $$ e^A \coloneqq \sum_{n = 0}^{\infty} \frac{1}{n!} A^n = I + A + \frac{1}{2} A^2 + \frac{1}{6} A^3 + \ldots $$ The series converges for all square matrices $A$ (but we will not prove this here).

Some important properties of the matrix exponential are:

• $e^0 = I$
• $e^A e^{-A} = I$
• $e^{(s + t) A} = e^{sA} e^{tA}$
• If $AB = BA$, then $e^{A + B} = e^A e^B = e^B e^A$
• $\frac{\partial}{\partial t} e^{tA} = A e^{tA} = e^{tA} A$

Note that $P(t) = e^{tQ}$ is the unique solution to the differential equations given by the Kolmogorov Forward and Backward equations with initial condition $P(0) = I$.

Further, note that $P^\prime(t) = Q P(t)$ for all $t \geq 0$ and $P(0) = I$.

In R one can use the expm package to compute matrix exponentials.

If $p \sim \mathrm{Dirichlet}(\alpha)$ and $x \mid p \sim \mathrm{Multinomial}(n, p)$, then the predictive distribution is given by, $$ \pi(x) = \frac{n!}{x_1! x_2! \ldots x_k!} \cdot \frac{\Gamma(\alpha_1 + x_1)}{\Gamma(\alpha_1)} \cdot \frac{\Gamma(\alpha_2 + x_2)}{\Gamma(\alpha_2)} \cdots \frac{\Gamma(\alpha_k + x_k)}{\Gamma(\alpha_k)} \cdot \frac{\Gamma(\sum_{i = 1}^k \alpha_i)}{\Gamma(n + \sum_{i = 1}^k \alpha_i)}. $$ For example, if $e_i$ is the vector with 1 at position $i$ and zeros elsewhere, then $\pi(x = e_i) = \frac{\alpha_i}{\sum_{j = 1}^k \alpha_j}$. Further, if $x_{\text{new}}$ is a vector of new counts, then, as $p \mid x \sim \mathrm{Multinomial}(x + \alpha)$, we get, $$ \pi(x_{\text{new}} = e_i \mid x) = \frac{\alpha_i + x_i}{\sum_{j = 1}^k \alpha_j + n}. $$ The $\alpha_i$ in the prior are often called pseudocounts, as they can be interpreted as prior counts added to the observed counts.

When the parameters of the HMM are known, we want to know about the values of the hidden variables $X_i$, for example,

• What is the most likely sequence $X_0, X_1, \ldots, X_n$ given the data?
• What is the probability distribution for a single $X_i$ given the data? However, when the parameters of the HMM are unknown, we need to infer these from some data.

If data with all $X_i$ and $Y_i$ known is available, inference for parameters is based on counts of transitions. The inference for the Markov chain is exactly as for the Markov chains we have looked at before. The inference for the emission probabilities, i.e., the parameters of $P(Y_k \mid X_k)$, can be done independently of the inference for the Markov chain.

For a homogeneous Poisson process, we setup up a prior $\pi(\lambda)$ for its parameter $\lambda$, and find the posterior given the observations.

The likelihood for observing $y$ events in an interval of length $t$ is given by, $$ \mathrm{Poisson}(y; \lambda t) = \exp(-\lambda t) \frac{(\lambda t)^y}{y!} \propto_{\lambda} \exp(-\lambda t) \lambda^y. $$ A convenient prior to use is $\lambda \sim \mathrm{Gamma}(\alpha, \beta)$.

In this case, the posterior becomes, $$ \lambda \mid \mathcal{D} \sim \mathrm{Gamma}(\alpha + y, \beta + t), $$ and the predictive distribution for the number of observations $y_n$ in a different interval of length $u$ becomes, $$ \pi(y_n) = \mathrm{Negative-Binomial}\left(y_n; \alpha, \frac{\beta}{\beta + u}\right). $$

Lastyl, recall that a continuous-time Markov chain with finite state space is completely characterized by its holding times parameter vector $q$ and the transition matrix $\tilde{P}$ of its embedded Markov chain.

Parametrizing instead with the “alarm clock” parameters $q_{ij}$ gives an equivalent description of the process.

The two parts of the data can be considered independently are,

• We learn about $\tilde{P}$ from the counts of transitions between states, and
• We learn about $q$ from the observed lengths of stays the process has in each state.

For $\tilde{P}$ the situation is analogous to the one for discrete-time Markov chains, except that the diagonal of $\tilde{P}$ must be zero, so the prior must exclude the possibility of non-zero diagonal elements.

For example, for $\tilde{P}_1$, the first row of $\tilde{P}$, we might use the prior $\mathrm{Dirichlet}(0, 1, \ldots, 1)$, i.e., a Dirichlet prior with a zero pseudocount for the first element.

The holding times in state $i$ are distributed as $\mathrm{Exponential}(q_i)$. If we have observed a total holding time of $h$ over $n$ intervals, that data has likelihood proportional to $e^{-hq_i}q_i^n$. Using $q_i \sim \mathrm{Gamma}(\alpha, \beta)$ as prior results in the posterior $q_i \mid \mathcal{D} \sim \mathrm{Gamma}(\alpha + n, \beta + h)$.

A continuous-time Markov chain is irreducible if for all $i$ and $j$ there exists a $t > 0$ such that $P_{ij}(t) > 0$, i.e., it is possible to get to any state from any state.

However, periodic continuous-time Markov chains do not exist. If $P_{ij}(t) > 0$ for some $t > 0$, then $P_{ij}(t) > 0$ for all $t > 0$.

The embedded chain of a continuous-time Markov chain: The discrete-time Markov chain where holding times are ignored.

The stationary distribution for the embedded chain and for the continuous-time chain are generally not the same!

However, there is a simple relationship: A probability vector $\pi$ is a stationary distribution for a continuous-time Markov chain if and only if $\psi$ is a stationary distribution for the embedded chain, where $\psi_j = C \pi_j q_j$ for a constant $C$ making the entries sum to 1.

A random walk is a discrete-time Markov chain $S_0, S_1, S_2, \ldots$ where $S_0 = 0$ and, $$ S_n = Y_1 + Y_2 + \ldots + Y_n, $$ and $Y_1, Y_2, \ldots$ are i.i.d. random variables. Assume $\mathbb{E}[Y_i] = 0$.

Further, if we assume $\mathrm{Var}(Y_i) = 1$, we get $\mathrm{Var}(S_n) = n$.

Interpolating between the values $S_n$ we can make this into a continuous-time process $S_t$ where $\mathrm{Var}(S_t) \approx t$.

We may scale with an $s > 0$ to get processes $S_t^{(s)} = \frac{S_{st}}{\sqrt{s}}$ where we get $\lim_{s \to \infty} \mathrm{Var}(S_t^{(s)}) = t$.

It turns out that the processes $S_t^{(s)}$ when $s \to \infty$ are exactly Brownian motion, no matter what type of $Y_i$ we start with.

This is the Donsker’s invariance principle, which is a generalization of the central limit theorem to stochastic processes ¹.

Brownian motion is a Gaussian process, as we can show that any $a_1 B_{t_1} + a_2 B_{t_2} + \ldots + a_n B_{t_n}$ is normally distributed.

A Gaussian process $\{X_t\}_{t \geq 0}$ is a Brownian motion if and only if,

1. $X_0 = 0$.

2. $\mathbb{E}[X_t] = 0$ for all $t$.

3. $\mathrm{Cov}(X_s, X_t) = \min(s, t)$ for all $s, t$.

4. The function $t \mapsto X_t$ is continuous with probability $1$ (almost surely).

The following transformations of Brownian motion also yield Brownian motion:

• $\{-B_t\}_{t \geq 0}$, negating the process yields another (reflected) Brownian motion.
• $\{B_{t + s} - B_s\}_{t \geq 0}$ for any fixed $s \geq 0$, shifting the time origin yields another Brownian motion.
• $\{\frac{1}{\sqrt{a}} B_{a t}\}_{t \geq 0}$ for any fixed $a > 0$, scaling time and space yields another Brownian motion.
• The process $\{X_t\}_{t \geq 0}$ where $X_0 = 0$ and $X_t = t B_{\frac{1}{t}}$ for $t > 0$, time inversion yields another Brownian motion.

We saw above that, for any fixed $t$ $(B_{t + s} - B_s)_{t \geq 0}$ is a Brownian motion. Does this phenomenon also hold if we start the chain anew from $T$ when $T$ is random? It depends.

If $T$ is the largest value less than 1 where $B_T = 0$, then $(B_{T + s} - B_T)_{s \geq 0}$ is not a Brownian motion.

If $T$ is the smallest value where $B_T = a$ for some constant $a$, then $(B_{T + s} - B_T)_{s \geq 0}$ is a Brownian motion. The reason is that the event $T = t$ can be determined based on $B_r$ where $0 \leq r \leq t$.

Random $T$‘s that have this property are called stopping times. For these $B_{T + s} - B_T$ is a Brownian motion.

Given that $a \neq 0$, what is the distribution of the first hitting time $T_a = \min\{t : B_t = a\}$?

We will prove that, $$ \frac{1}{T_a} \sim \mathrm{Gamma}\left(\frac{1}{2}, \frac{a^2}{2}\right). $$

Assuming that $a > 0$ and using that $T_a$ is a stopping time we get for any $t > 0$ that $P(B_{\frac{1}{t}} > a \mid T_a < \frac{1}{t}) = P(B_{\frac{1}{t} - T_a} > 0) = \frac{1}{2}$.

We also have, $$ \begin{align*} P(B_{\frac{1}{t}} > a \mid T_a < \frac{1}{t}) & = \frac{P(B_{\frac{1}{t}} > a, T_a < \frac{1}{t})}{P(T_a < \frac{1}{t})} \newline & = \frac{P(B_{\frac{1}{t}} > a)}{P(T_a < \frac{1}{t})}. \end{align*} $$ Further, it follows that $P(T_a < \frac{1}{t}) = 2 P(B_{\frac{1}{t}} > a)$ and thus, $$ \begin{align*} P\left(\frac{1}{T_a} \leq t\right) & = 2 P\left(B_{\frac{1}{t}} > a\right) -1 \newline & = 2 P(B_1 \leq a \sqrt{t}) - 1 \newline \end{align*} $$ Taking the derivative with respect to $t$ we get the Gamma density, $$ \pi_{1/T_a}(t) = 2 \frac{1}{\sqrt{2 \pi}} \exp\left(-\frac{1}{2} \left(a \sqrt{t}\right)^2\right) \frac{a}{2} t^{-1/2}. $$

We can define $M_t \coloneqq \max_{0 \leq s \leq t} B_s$.

We may compute for $a > 0$ (using the results above), $$ \begin{align*} P(M_t > a) & = P(T_a < t) \newline & = 2 P(B_t > a) \newline & = P(|B_t| > a). \end{align*} $$ Thus, $M_t$ has the same d istribution as $|B_t|$, i.e., the absolute value of $B_t$.

Let $L$ be the last zero in $(0, 1)$ of a Brownian motion. (i.e., $L = \max \ \{t : 0 < t < 1, B_t = 0\}$. Then, $$ L \sim \mathrm{Beta}\left(\frac{1}{2}, \frac{1}{2}\right). $$

A (European) stock option is a right (but not an obligation) to buy a stock at a given time $t$ in the future for a given price $K$.

How much can you expect to earn from a stock option at that future time?

We get that (we will derive this), $$ \mathbb{E}[\max(G_t - K, 0)] = G_0 e^{t(\mu + \frac{\sigma^2}{2})} P\left(B_1 > \frac{\beta - \sigma t}{\sqrt{t}}\right) - K P\left(B_1 > \frac{\beta}{\sqrt{t}}\right), $$ where $\beta = \frac{\log(K / G_0) - \mu t}{\sigma}$.

Consider the Markov chain $X_0, X_1, \ldots$ with states $\{0, 1, 2\}$ and with, $$ P = \begin{bmatrix} 0.99 & 0.01 & 0 \newline 0 & 0.9 & 0.1 \newline 0.2 & 0 & 0.8 \end{bmatrix} $$ One can find that the limiting distribution is $v = (\frac{20}{23}, \frac{2}{23}, \frac{1}{23})$.

Consider the function $r(x) = x^5$. If $X$ is a random variable with the limiting distribution, $$ \mathbb{E}[r(X)] = 0^5 \cdot \frac{20}{23} + 1^5 \cdot \frac{2}{23} + 2^5 \cdot \frac{1}{23} = \frac{34}{23} = 1.4783 $$ If $Y_1, \ldots, Y_n$ are all i.i.d variables with the limiting distribution, we can check numerically that, $$ \lim_{n \to \infty} \frac{r(Y_1) + r(Y_2) + \ldots + r(Y_n)}{n} = 1.4783 $$ We also get for $X_0, X_1, \ldots$ that, $$ \lim_{n \to \infty} \frac{r(X_1) + r(X_2) + \ldots + r(X_n)}{n} = 1.4783 $$ but in this case the limit is approached more slowly.

How many zeroes does $B_t$ have in the interval $(0, s)$?

If $L_s$ is the last zero in $(0, s)$, we saw that, $$ \frac{L_s}{s} \sim \mathrm{Beta}\left(\frac{1}{2}, \frac{1}{2}\right), $$ which means that, with probability one, there exists a zero in the interval $(0, s)$ and for the last zero $L_s$ we have $0 < L_s < s$.

Repeating the argument, we have that $0 < L_{L_s} < L_s$ with probability one, and thus there exists another zero in $(0, L_s) \subset (0, s)$.

The conclusion is that, with probability one, there is an infinite number of zeroes in $(0, s)$ for any $s > 0$.

We saw above how to use Gibbs sampling to simulate from the Ising model. We then simulated from the conditional distributions, using, $$ \pi(\sigma_v^{\star} = 1 \mid \sigma_{-v}) = \ldots = \frac{1}{1 + \exp(-2 \beta \sum_{v \sim w} \sigma_w)} $$ Now, we extend this method into a perfect sampling.

Define the partial ordering on configurations $\sigma, \tau$ be defining, $$ \sigma \leq \tau \iff \sigma_v \leq \tau_v \text{ for all vertices } v $$ We have minimal and maximal elements $m = (-1, \ldots, -1)$ and $M = (1, \ldots, 1)$.

Defining $g(\sigma, U)_v = 2 I (U \leq \pi(\sigma_v^{\star} = 1 \mid \sigma_{-v}) ) - 1$, makes $g$ monotone.

Given an Ising model with $\beta > 0$, start with choosing an integer $n > 0$ and setting configurations, $$ \begin{align*} \sigma^{(-n)} & = m = (-1, \ldots, -1) \newline \tau^{(-n)} & = M = (1, \ldots, 1) \newline \end{align*} $$

Assume the offspring process is $\mathrm{Binomial}(N, p)$ for some parameter $p$ and a fixed (known) $N$. By definition, we get the likelihood, $$ \pi(y_1, y_2, \ldots, y_n \mid p) \coloneqq \prod_{i = 1}^n \mathrm{Binomial}(y_i; N, p) $$ A possibility is to use a prior $p \sim \mathrm{Beta}(\alpha, \beta)$. Writing $S = \sum_{i = 1}^n y_i$, we get the posterior, $$ p \mid \mathcal{D} \sim \mathrm{Beta}(\alpha + S, \beta + nN - S). $$ where $\mathcal{D} = {y_1, y_2, \ldots, y_n}$ is the data.

More generally, if $\pi(p) = f(p)$ for any positive function integrating to 1 on $[0, 1]$, we get the posterior, $$ \pi(p \mid \mathcal{D}) \propto_p \mathrm{Beta}(p; S + 1, nN - S + 1) f(p). $$ We can then for example compute numerically the posterior probability that the branching process is supercritical, i.e., that $P(p > \frac{1}{N} \mid \mathcal{D})$, with, $$ \int_{\frac{1}{N}}^1 \pi(p \mid \mathcal{D}) \ dp = \frac{\int_{\frac{1}{N}}^1 \mathrm{Beta}(p; S + 1, nN - S + 1) f(p) \ dp}{\int_0^1 \mathrm{Beta}(p; S + 1, nN - S + 1) f(p) \ dp}. $$

Assume there is a maximum of $N$ offspring and that now $p = (p_0, p_1, \ldots, p_N)$ is an (unknown) probability vector such that $p_i$ is the probability of $i$ offspring. By definition, we get the likelihood, $$ \pi(y_1, y_2, \ldots, y_n \mid p) \coloneqq \mathrm{Multinomial}(c; p), $$ where $c = (c_0, c_1, \ldots, c_N)$ is the vector of counts in the data of cases with $0, \ldots, N$ offspring, respectively.

If we use a prior $p \sim \mathrm{Dirichlet}(\alpha)$, where $\alpha = (\alpha_0, \alpha_1, \ldots, \alpha_N)$ is a vector of pseudocounts, we get the posterior, $$ p \mid \mathcal{D} \sim \mathrm{Dirichlet}(\alpha + c), $$ with expecation, $$ \mathbb{E}[p_i \mid \mathcal{D}] = \frac{\alpha_i + c_i}{\sum_{j = 0}^N (\alpha_j + c_j)}. $$

The simplest example of a birth-and-death process is when $\lambda_i = \lambda$ and $\mu_i = \mu$ for all $i \geq 0$. In this case, the stationary distribution is given by, $$ \begin{align*} v_k & = v_0 \left(\frac{\lambda}{\mu}\right)^k = v_0 \left(\frac{\lambda}{\mu}\right)^k, \newline v_0 & = \left(\sum_{k = 0}^{\infty} \left(\frac{\lambda}{\mu}\right)^k\right)^{-1} = \frac{1}{1 + \frac{\lambda}{\mu} + \left(\frac{\lambda}{\mu}\right)^2 + \ldots} = \frac{1}{\frac{1}{1 - \frac{\lambda}{\mu}}} = 1 - \frac{\lambda}{\mu}. \end{align*} $$ Which is a geometric distribution with parameter $1 - \frac{\lambda}{\mu}$, $\mathrm{Geometric}\left(1 - \frac{\lambda}{\mu}\right)$. Thus, the long-term average value of $X_t$ is, $$ E[X_t] \coloneqq \frac{\frac{\lambda}{\mu}}{1 - \frac{\lambda}{\mu}} = \frac{\lambda}{\mu - \lambda}. $$

Show that $B_1 + B_3 + 2 B_7 \sim \mathcal{N}(0, 50)$.

Show that $P(B_2 > 0 \mid B_1 = 1) = 0.8413$.

Show that $\mathrm{Cov}(B_s, B_t) = \min(s, t)$.

What is the probability that $M_3 > 5$?

Find $t$ such that $P(M_t \leq 4) = 0.9$.

From what we have seen so far, one can model the price of a stock with:

• Use a continuous-time stochastic model.
• Consider the factor with which it changes, not the differences in prices.
• Consider the log-normal distribution for such factors.
• Use a parameter for the trend of the price (drift), and one for the volatility (variance) of the price.
• Make a Markov assumption (although, we have to reflect on this choice).

This leads to using a geometric Brownian motion as a model, $$ G_t = G_0 e^{\mu t + \sigma B_t}. $$ In this context $\sigma$ is called the volatility of the stock.

A stock price is modeled with $G_0 = 67.3$, $\mu = 0.08$, and $\sigma = 0.3$. What is the expected payoff from an option to buy the stock at 100 in 3 years?

Let $Y_t \coloneqq B_t^2 - t$ for $t \geq 0$. Then, $\{Y_t\}_{t \geq 0}$ is a martingale with respect to the Brownian motion $\{B_t\}_{t \geq 0}$.

When making investments, there is always a range of choices, some of which are sometimes called “risk-free”. Such investments may pay a fixed interest.

When interests are compounded frequently, a reasonable model is that an investment of $G_0$ has a value $G_0 e^{rt}$ after time $t$, where $r$ is the “risk-free” investment rate of return.

A common way to take this alternative into account is to instead “discount” all other investments with the factor $e^{-rt}$.

For example, the discounted value of a stock can be modeled as, $$ e^{-rt} G_t = e^{-rt} G_0 e^{\mu t + \sigma B_t} = G_0 e^{(\mu - r) t + \sigma B_t}. $$ A possible assumption about the trend $\mu$ of a stock price is that the discounted value behaves as a martingale with respect to the Brownian motion $B_t$.

Thus, we get, $$ \mu - r + \frac{\sigma^2}{2} = 0, \quad \text{i.e.,} \quad \mu = r - \frac{\sigma^2}{2}. $$

The Black-Scholes formula for option pricing is based on,

• Assuming the discounted stock price is a martingale with respect to the Brownian motion.
• Using discounting when computing the value of the option.

From this, we get, $$ e^{-rt} \mathbb{E}[\max(G_t - K, 0)] = G_0 P\left(B_1 > \frac{\beta - \sigma t}{\sqrt{t}}\right) - K e^{-rt} P\left(B_1 > \frac{\beta}{\sqrt{t}}\right), $$ where $\beta = \frac{\log(K / G_0) - (r - \frac{\sigma^2}{2}) t}{\sigma}$.