This is a brief techical note on Brownian Motion and SDE. For more exhaustive and rigorous material, checkout the references.

Brownian Motion

To define Brownian motion, we first need to know that such disribution exists.

Theorem 1 (existence) $\exist$ a probability distribution over the set of all continuous function $B: \mathbb{R}_{\geq 0}\mapsto \mathbb{R}$ s.t.

1. $P(B(0) = 0) = 1$
2. (Stationary) $\forall 0\leq s \leq t, B(t) - B(s)\sim\mathcal{N}(0, t-s)$
3. (Independent Increment) If intervals $\{[s_i, t_i]\}$ are non-overlapping, then $B(t_i) - B(s_i)$ are independent.

Now we define the Brownian motion:

Definition The distribution given by Theorem 1 is called Brownian motion.

One important property of Brownian motion is that it’s NOT differentiable! We will not prove this. Since Brownian motion can’t be properly computed using normal calculus, we will use Ito’s Calculus to derive all the equations. To see how we can derive Ito’s Lemma, we first check an important theorem:

Theorem 2 (Quadractic Variation) $\forall T>0$, we have:

This is a surprising fact about Brownian motion because for any continuous and differentiable function $f$, we have that

and as $n\rightarrow 0$, $\sum_{i=1}^n (f(t_{i+1}) - f(t_i))^2 \rightarrow 0$. Thus, for normal functions, the quadratic variation is zero. We can denote the quadractic variation of Brownian motion as $(dB)^2 = dt$. We say that square of difference in Brownian motion goes to the difference in time with probability 1.

Intuitively, the quadratic variation simply denote the variance of the Brownian motion between an infinitesimal step, which, by definition, equals to the time step. For deterministic function, the variance is simply zero. This accumulation of the second order term makes the Linear appoximation of Brownian motion much different from ordinary functions.

Ito’s Calculus

Suppose we want to compute $f(B_t)$ for some smooth function $f$. The Taylor Expansion of the $f$ is:

Normally, we would ignore terms after the first order. However, from Theorem 2, we know that $(dB_t)^2 = dt$, and thus we can no longer ignore the second order term. Finally, we can write the (simple) Ito’s Lemma:

Lemma (Simple Ito)

For a stronger Ito’s Lemma, consider the Taylor Expansion of the function $f(t, B_t)$:

Again, we apply the Theorem 2 to get Ito’s Lemma:

Ito Lemma

Now, let’s consider a more general stochastic process $X_t$ defined as

Where $\mu dt$ is called the drift term. We can use Ito Lemma to write the $df(t, X_t)$ as follow:

Stochastic Differential Equation (SDE)

Definition

A SDE is a differential equation of the form

Then, we have the following theorem:

Theorem

SDE has a solution and if given $\mathbf{X}_0$ then the solution is unique as long as $\mu, \sigma$ satisfy certain condition^$\dagger$.

Let’s consider when $\mu$ and $\sigma$ are proportional to $\mathbf{X}_t$:

Using Ito Lemma, we can show that the solution for the given SDE is $X_t = X_0 e^{\sigma \mathbf{X}_t + (\mu - \frac{1}{2}\sigma^2)t}$. Now, let’s consider when the stochastic term $\sigma(t, \mathbf{X}_t)$ is independent of the current state $\mathbf{X}_t$:

This is called the Ornstein-Uhlenbeck (OU) Process. And we have the following solution for the OU process:

For more detailed derivation of the solution, check out the awesome lecture note ^[2] on SDE.

Connection to the Heat Equation

For fun, let’s see the connection between Brownian Motion and the heat equation. This profound connection was discovered by none other than Albert Einstein in 1905 when he was 26 and working on Special Relativity and the Photoelectric Effect :). Let’s first define a Heat Equation:

with a given boundary condition $u(0, x)$ and $u(t,x)$ is the probability density function at time $t$ and position $x$. Due to the linearity property of the heat equation (if $u_1, u_2$ are solutions then $u_1+u_2$ is also a solution), the intergral $\int u_s ds$ is a solution if all $u_s$ are solutions to the heat equation. So, for all initial condition, it’s sufficient to solve for $u(0,x) = \delta (x)$. Let denote the solution to this initial condition as $u_\delta(t,x)$. We can easily see this if we have some complex initial condition $u_0$, we can decompose $u_0$ into delta functions:

Then, the solution given $u_0$ can also be decomposed into solutions of the $u_0 = \delta$:

Finally, the solution $u_\delta(t,x)$ is also known as the heat kernel or gaussian kernel and is given as

To connect this to Brownian Motion, consider the following SDE:

then, the probability density function of $X_t$ is given as

Which is also the solution to the heat equation

References

[0] MIT OCW, Ito Calculus
[1] MIT OCW, Stochastic Differential Equations
[2] Gautam Iyer, Stochastic Calculus for Finance Brief Lecture Notes