# Phorgy Phynance

## Network Theory and Discrete Calculus – The Discrete Master Equation

This post is a follow up to

Network Theory and Discrete Calculus – Introduction

To give the result first, the master equation can be expressed in terms of discrete calculus simply as

$\partial(\psi P) = 0,$

where $\psi$ is a discrete 0-form representing the states of a Markov chain (at all times), $P$ is a discrete 1-form representing transition probabilities, and $\partial$ is the boundary operator, i.e. a kind of graph divergence.

The rest of this post explains the terms in this discrete master equation and how it works.

### The State-Time Graph

When working with a finite (or countable) number of states, there is nothing new in considering states $\psi_i$ to be associated to nodes and the transition probabilities $P_{i,j}$ to be associated to directed edges of a bi-directed graph. A simple 2-state example is given below

The directed graphs we work with for discrete stochastic calculus are slightly different and could be referred to as “state-time” graphs, which are supposed to make you think of “space-time”. A state $i$ at time $t$ is considered a different node than the state $i$ at time $t+1$. An example 2-state, 2-time directed graph is illustrated below:

There are four directed edges in this state-time graph, which will be labelled

* $\mathbf{e}^{(i,t)(i,t+1)}$
* $\mathbf{e}^{(i,t)(j,t+1)}$
* $\mathbf{e}^{(j,t)(i,t+1)}$
* $\mathbf{e}^{(j,t)(j,t+1)}$

For $N$ states, the state-time graph will look similar but with more states appended horizontally.

### The Discrete Master Equation

A discrete 0-form representing the states at all times can be expressed as

$\psi = \sum_i \sum_t \psi_i^t \mathbf{e}^{(i,t)}$

and a discrete 1-form representing the transition probabilities can be expressed as

$P = \sum_{i,j} \sum_t P_{i,j}^t \mathbf{e}^{(i,t)(j,t+1)}.$

The product of the 0-form $\psi$ and the 1-form $P$ is given by

$\psi P = \sum_{i,j} \sum_t \psi_i^t P_{i,j}^t \mathbf{e}^{(i,t)(j,t+1)}.$

The boundary of a directed edge is given by

$\partial \mathbf{e}^{(i,t)(j,t+1)} = \mathbf{e}^{(j,t+1)} - \mathbf{e}^{(i,t)}.$

Now for some gymnastics, we can compute

\begin{aligned} \partial(\psi P) &= \sum_{i,j} \sum_t \psi_i^t P_{i,j}^t \left[\mathbf{e}^{(j,t+1)} - \mathbf{e}^{(i,t)}\right] \\ &= \sum_{i,j} \sum_t \left[\psi_j^t P_{j,i}^t \mathbf{e}^{(i,t+1)} - \psi_i^t P_{i,j}^t \mathbf{e}^{(i,t)}\right] \\ &= \sum_i \sum_t \left[\sum_j \left(\psi_j^t P_{j,i}^t - \psi_i^{t+1} P_{i,j}^{t+1}\right)\right] \mathbf{e}^{(i,t+1)}. \end{aligned}

This is zero only when the last term in brackets is zero, i.e.

$\sum_j \left(\psi_j^t P_{j,i}^t - \psi_i^{t+1} P_{i,j}^{t+1}\right) = 0$

or

$\psi_i^{t+1} \sum_j P_{i,j}^{t+1} = \sum_j \psi_j^t P_{j,i}^t.$

Since $P$ is right stochastic, we have

$\sum_j P_{i,j}^{t+1} = 1$

so that

$\psi_i^{t+1} = \sum_j \psi_j^t P_{j,i}^t.$

In other words, when $P$ is right stochastic and $\partial(\psi P) = 0$, we get the usual master equation from stochastic mechanics

$\partial(\psi P) = 0\implies \psi_i^{t+1} = \sum_j \psi_j^t P_{j,i}^t.$

### Parting Thoughts

The master equation is a boundary. This makes me wonder about homology, gauge transformations, sources, etc. For example, since

$\partial(\psi P) = 0,$

does this imply

$\psi P = \partial F$

for some discrete 2-form $F$?

If $G$ is a discrete 2-form whose boundary does not vanish, then

$\psi P + \partial G$

gives the same dynamics because $\partial^2 = 0.$ This would be a kind of gauge transformation.

There are several directions to take this from here, but that is about all the energy I have for now. More to come…

Written by Eric

October 29, 2011 at 10:04 pm

## Network Theory and Discrete Calculus – Introduction

I’ve enjoyed applying discrete calculus to various problems since Urs Schreiber and I wrote our paper together back in 2004

Discrete differential geometry on causal graphs

Shortly after that, I wrote an informal paper applying the theory to finance in

Financial modeling using discrete stochastic calculus

From there I wrote up some private notes laying the foundations for applying a higher-dimensional version of discrete calculus to interest rate models. However, life intervened, I went to work on Wall Street followed by various career twists leading me to Hong Kong where I am today. The research has laid fairly dormant since then.

I started picking this up again recently when my friend, John Baez, effectively changed careers and started the Azimuth Project. In particular, I’ve recently developed a discrete Burgers equation with corresponding discrete Cole-Hopf transformation, which is summarized – including numerical simulation results – on the Azimuth Forum here:

Discrete Burgers equation revisited

Motivated by these results, I started looking at a reformulation of the Navier-Stokes equation in

Towards Navier-Stokes from noncommutative geometry

This is still a work-in-progress, but sorting this out is a necessary step to writing down the discrete Navier-Stokes equation.

Even more recently, John began a series of very interesting Azimuth Blog posts on network theory. I knew that network theory and discrete calculus should link up together naturally, but it took a while to see the connection. It finally clicked one night as I laid in bed half asleep in one of those rare “Eureka!” moments. I wrote up the details in

Discrete stochastic mechanics

There is much more to be said about the connection between network theory and discrete calculus. I intend to write a series of subsequent posts in parallel to John’s highlighting how his work with Brendan Fong can be presented in terms of discrete calculus.

Written by Eric

October 28, 2011 at 9:12 am