Stochastic Differential Equations

Stochastic differential equations (SDEs) describe the continuous-time evolution of systems driven by random noise. They are the continuous-time counterpart of the linear dynamical systems studied in Part IV and underpin a broad range of models in statistics, physics, finance, and machine learning. In this course they arise most prominently in two places: as the principled mathematical framework for denoising diffusion models, and as a lens through which many stationary Gaussian processes can be formulated and computed with efficiently.

Brownian Motion¶

The fundamental source of randomness in continuous-time stochastic systems is Brownian motion (also called the Wiener process).

A $d$-dimensional Brownian motion $\mathbf{w}_t \in \mathbb{R}^d$ is a vector of $d$ independent scalar Brownian motions, so $\mathbf{w}_t - \mathbf{w}_s \sim \mathcal{N}(\mathbf{0},\, (t-s)I)$.

Brownian motion is nowhere differentiable almost surely: sample paths have unbounded variation on every interval and are far too rough for a classical derivative to exist. This is not a pathology — it is essential for the central limit theorem intuition underlying the construction. As we add finer and finer independent noise increments, the accumulated noise scales as $\sqrt{t}$ (not $t$), and this $\sqrt{\Delta t}$ scaling of increments is what forces us to treat Brownian motion differently from smooth driving signals.

The Ito Integral¶

Because $w_t$ is not differentiable, integrals of the form $\int_0^t \sigma(u_s, s)\,dw_s$ must be defined with care. The Ito integral constructs this as an $L^2$ limit using left-endpoint Riemann sums:

The integrand must be adapted to the Brownian filtration $\{\mathcal{F}_t\}$, meaning $\sigma(u_t, t)$ may depend only on the history up to time $t$, not on future values of $w$.

Two key properties of the Ito integral follow directly from the construction.

Zero mean. Each summand $\sigma(u_{t_i}, t_i)[w_{t_{i+1}} - w_{t_i}]$ has zero conditional expectation given $\mathcal{F}_{t_i}$, because the Brownian increment is independent of the past and has mean zero. By the tower property, every partial sum has mean zero, and the limit inherits this:

Ito isometry. Cross-terms in $\bigl(\sum_i \sigma_{t_i}\Delta w_i\bigr)^2$ vanish for $i \neq j$ by the same independence argument. Only the diagonal terms contribute, using $\mathbb{E}[(\Delta w_i)^2] = \Delta t$:

The isometry says that the $L^2$ norm of the integral equals the $L^2$ norm of the integrand in the product space $[0, t] \times \Omega$.

Stochastic Differential Equations¶

Starting from Euler’s method for the ODE $\dot{u} = f(u, t)$,

we augment each step with a Gaussian noise term whose variance scales as $\Delta t$ — matching the variance of a Brownian increment over the same interval:

Recognizing $\varepsilon\sqrt{\Delta t} = w(t + \Delta t) - w(t)$ and taking $\Delta t \to 0$ yields the stochastic differential equation

interpreted rigorously as the integral equation

The function $f : \mathbb{R}^d \times \mathbb{R}_+ \to \mathbb{R}^d$ is the drift coefficient and $\sigma : \mathbb{R}^d \times \mathbb{R}_+ \to \mathbb{R}^{d \times d}$ is the diffusion coefficient. The process $\{u_t\}$ solving the SDE is called an Ito process or diffusion process. Under standard Lipschitz and growth conditions on $f$ and $\sigma$, existence and uniqueness of a strong solution are guaranteed.

Why Gaussian noise? Beyond analytical tractability, the central limit theorem gives physical justification: a large number of small independent perturbations accumulate into something Gaussian. The SDE framework can be generalized to other driving processes (any semimartingale), but Gaussian noise is by far the most common choice.

Ito’s Lemma¶

The ordinary chain rule says: if $u(t)$ is a smooth function and $\phi$ is smooth, then $d\phi(u) = \phi'(u)\,du$. For an Ito process, there is an additional correction term arising from the quadratic variation of Brownian motion.

Derivation sketch. Expand $\phi$ to second order via Taylor’s theorem and substitute $du_t = f_t\,dt + \sigma_t\,dw_t$:

The key step is evaluating $(du_t)^2 = (f_t\,dt + \sigma_t\,dw_t)^2$. Using the multiplication rules $dt^2 \to 0$, $dt\,dw_t \to 0$, and $dw_t^2 \to dt$ — which follow from the $L^2$ behavior of Brownian motion — gives $(du_t)^2 = \sigma_t^2\,dt$. The higher-order terms vanish. The extra $\frac{\sigma_t^2}{2}\frac{\partial^2\phi}{\partial u^2}$ term, absent in ordinary calculus, is called the Ito correction.

Multivariate form. For $\mathbf{u}_t \in \mathbb{R}^d$ satisfying $d\mathbf{u}_t = \mathbf{f}_t\,dt + G_t\,d\mathbf{w}_t$ and a smooth function $\phi : \mathbb{R}^d \times \mathbb{R}_+ \to \mathbb{R}$,

Example: geometric Brownian motion. The SDE $du_t = \mu u_t\,dt + \sigma u_t\,dw_t$ is the classical model for asset prices. Applying Ito’s lemma to $\phi(u) = \log u$ gives

so $\log u_t$ drifts linearly and $u_t = u_0\exp\bigl((\mu - \frac{\sigma^2}{2})t + \sigma w_t\bigr)$ is log-normally distributed. The $-\sigma^2/2$ correction relative to the drift $\mu$ is a direct consequence of the Ito term.

The Fokker-Planck Equation¶

Instead of tracking individual trajectories, one can describe the evolution of the marginal density $p_t(u)$ of the process. For the SDE $du_t = f(u_t, t)\,dt + \sigma(u_t, t)\,dw_t$, the marginal density satisfies the Fokker-Planck equation (also called the forward Kolmogorov equation):

In the scalar case with state-independent diffusion $\sigma(t)$, this simplifies to

The first term describes probability transport due to the drift — it has the form of a continuity equation. The second term is a diffusion equation that spreads probability mass. Together they give a precise PDE governing how the distribution of $u_t$ evolves over time.

The backward Kolmogorov equation is the adjoint of the Fokker-Planck operator and describes how expected functions of the terminal state $\mathbb{E}[\phi(u_T) \mid u_t = u]$ evolve backward in time from $T$ to $t$. It is the continuous-time analogue of the backwards recursions in dynamic programming.

Simulation: Euler–Maruyama¶

Given initial condition $u_0 \sim P_0$ and a time grid $0 = t_0 < t_1 < \cdots < t_n = T$ with step $\Delta t$, the Euler–Maruyama method approximates the SDE solution by

This is the direct stochastic analogue of Euler’s method and has strong convergence rate $O(\sqrt{\Delta t})$. The Milstein scheme adds a correction using the Ito lemma applied to $\sigma$,

improving the strong convergence rate to $O(\Delta t)$ when $\sigma$ depends on the state. When the transition density $p_{t_i, t_{i+1}}(u_{t_{i+1}} \mid u_{t_i})$ is known in closed form — as it is for linear SDEs — exact simulation is possible without any discretization error.

Linear SDEs and the Ornstein–Uhlenbeck Process¶

The most tractable family of SDEs has a drift that is linear in the state:

Because the noise enters linearly, solutions have Gaussian transition densities and therefore define Gaussian processes.

Solution via integrating factor. For the scalar time-invariant case $du_t = a\,u_t\,dt + g\,dw_t$, apply Ito’s lemma to $\phi(u_t, t) = e^{-at}u_t$:

Integrating both sides from 0 to $t$ and rearranging,

The Ito integral is Gaussian (as an $L^2$ limit of Gaussian sums), so the transition density is

For $a < 0$, the mean decays exponentially toward zero and the variance saturates at $g^2/(2|a|)$ as $t \to \infty$.

The Ornstein–Uhlenbeck process. The canonical mean-reverting SDE is

The drift pulls $u_t$ toward the long-run mean $\mu$ at rate $\theta$; $\sigma$ controls the noise level. The transition density is

The marginal distribution converges to the stationary distribution $\mathcal{N}(\mu,\, \sigma^2/(2\theta))$ regardless of the initial condition, with the exponential covariance kernel $k(s, t) = \frac{\sigma^2}{2\theta}e^{-\theta|t-s|}$.

Multivariate case. For $d\mathbf{u}_t = A\mathbf{u}_t\,dt + G\,d\mathbf{w}_t$ with stable $A$, the solution is

with transition covariance $P(t) = \int_0^t e^{As}GG^\top e^{A^\top s}\,ds$. The stationary covariance $P_\infty$ is the unique positive definite solution to the continuous-time Lyapunov equation

Stationary Distributions¶

When does the density $p_t$ converge as $t \to \infty$? A stationary distribution $p^*$ satisfies the time-independent Fokker-Planck equation (right-hand side set to zero). For the scalar constant-diffusion case $\sigma(u, t) = \sigma$, one can integrate the Fokker-Planck equation twice to obtain

This is the Boltzmann distribution with energy $E(u) = -\int^u f(v)\,dv$.

Langevin dynamics. This connection between drift and stationary distribution is the foundation of gradient-based MCMC. To sample from a target distribution $p^*(u) \propto e^{-E(u)}$, one runs the SDE

which has $p^*(u) \propto e^{-E(u)}$ as its unique stationary distribution. In the Bayesian context, $E(\boldsymbol{\theta}) = -\log p(\mathbf{x} \mid \boldsymbol{\theta}) - \log p(\boldsymbol{\theta})$, and Langevin dynamics provides a continuous-time limit of gradient-based MCMC. Discretization via Euler–Maruyama gives the unadjusted Langevin algorithm (ULA); adding a Metropolis–Hastings correction step recovers the Metropolis-adjusted Langevin algorithm (MALA).

Linear SDEs as Gaussian Processes¶

Since solutions to linear SDEs have Gaussian finite-dimensional distributions, they are Gaussian processes. The GP mean and covariance can be computed from the SDE coefficients:

both of which satisfy ODEs derivable from the SDE.

Conversely, many classical stationary covariance kernels correspond exactly to the stationary GPs generated by specific linear SDEs:

This SDE–GP equivalence has an important computational consequence. Standard GP regression with $N$ observations requires $O(N^3)$ time to compute (and invert) the $N \times N$ covariance matrix. When the covariance function corresponds to a linear SDE, the Kalman filter performs the same inference in $O(N)$ time, by exploiting the Markov structure of the SDE solution. This makes SDE-based GP approximation a key technique for large-scale time series Solin & Särkkä, 2020.

The Reverse-Time SDE¶

A foundational result due to Anderson, 1982 shows that the time-reverse of a diffusion process is also a diffusion. If $\{u_t : t \in [0, T]\}$ satisfies the forward SDE

then the reverse-time process $\bar{u}_t = u_{T-t}$ satisfies

where $\bar{w}_t$ is a new Brownian motion (running backward) and $\nabla_u \log p_t(u)$ is the score function of the marginal density at time $t$.

The reverse SDE has the same marginal distributions as the forward SDE, but run in reverse time. Starting from $\bar{u}_0 \sim p_T$ (a simple noise distribution) and simulating the reverse SDE to time $T$ produces a sample from $p_0$ (the data distribution).

This result is the mathematical heart of score-based generative modeling: design a forward SDE that converts data into noise, estimate the score $\nabla_u \log p_t$ using a neural network trained by denoising score matching, then run the reverse SDE to generate new data. The connection between the reverse-time correction and the score function explains why learning to denoise is equivalent to learning to generate — a remarkable duality discussed further in the Denoising Diffusion Models chapter.

Conclusion¶

Stochastic differential equations extend ordinary differential equations by adding Brownian noise, yielding continuous-time models whose sample paths are nowhere differentiable. The Ito integral and Ito’s lemma provide the calculus needed to work with these processes. Linear SDEs are especially tractable: their solutions are Gaussian processes with kernels that correspond to classical covariance functions (exponential, Matérn), and the Kalman filter performs GP inference in linear time by exploiting the Markov structure. Stationary distributions of SDEs are Boltzmann densities, connecting SDEs to Langevin MCMC. The reverse-time SDE of Anderson (1982) is the mathematical foundation of score-based generative modeling, providing a principled framework for understanding denoising diffusion models.