# Notebook from the-ninth-wave/stat-mech
Path: _notebooks/2021-12-16-simulation.ipynb

# Ising model dynamics

> Notes for a simulation project.

- toc: true
- badges: true
- comments: false
- categories: [jupyter]_____no_output________

[homepage](https://the-ninth-wave.github.io/)

[stat. mech. notes](https://the-ninth-wave.github.io/stat-mech/)

________no_output______project outline______no_output_____$\quad$ These notes supplement an ongoing project with a good friend, [Connor Sempek](https://github.com/connorsempek). Our main objective is to re-create simulations of Ising model dynamics from [this](http://bit-player.org/2021/three-months-in-monte-carlo) blog post by [Brian Hayes](http://bit-player.org/about-the-author). The underlying 'canvas' of the model is two-dimensional, and the simulation generates a sequence of two-color `100x100` pixel images. The two colors represent the two possible _spin_ values of the [Ising model](https://www.unige.ch/math/folks/velenik/smbook/Ising_Model.pdf), $+1$ and $-1$ (though the term spin is used, the model is not quantum-mechanical). The magnetic interpretation is of the model is that each pixel is some atomic site in a crystal lattice, and the color value of each pixel describes one of two opposing magnetic orientations the atom can posess. At a given temperature $T$, a sample from the Ising model (in our context) corresponds to a randomly generated image. The dynamics of the Ising model correspond to a movie formed by a sequence of these images. Each image is obtained from the previous in the sequence by randomly updating the spin $\equiv$ color at a randomly selected pixel, updating one pixel at a time. This movie is analogous to viewing a bar of iron under a microscope, when the updates are rapid enough. _____no_output_____$\quad$ Parameter $T > 0$ controls the degree of structure in the random images generated. At high enough temperatures, any impulse neighboring spins feel to align is nearly washed away by thermal noise. In this regime, the simulation should look very much like static. At low temperatures, the spins exhibit collective alignment, and the simulation should depict large patches of sites all with the same color. The qualitative behavior of the system changes as it crosses a specific _critical temperature_ $T_c$. For the same phenomenon in iron, this $T_c$ is called the Curie temperature, and corresponds to the bar glowing a dull red. A bar of magnetized iron will lose its magnetization when heated above $T_c$. When its temperature is lowered back below $T_c$, the bar is again susceptible to magnetization: immersing the bar in a strong enough magnetic field causes its spins to align with the field. _____no_output_____$\quad$ The above description of the phase transition occuring in iron requires that we observe the behavior of the iron bar as its temperature changes over time. This is natural in reality, where time appears to flow at a constant rate. However, as discussed, a sample from the Ising model at some temperature $T$ is a static picture. In order to simulate in the Ising model what we describe doing to the iron bar, one must first describe how to set the Ising model into motion. There will be two ways of doing this at a fixed temperature $T$, described by two distinct Markovian dynamics defined on the state space of the Ising model. These are so-called _Glauber_ and _Metropolis_ dynamics. One goal of the [motivating blog post](http://bit-player.org/2021/three-months-in-monte-carlo) was to explore how these dynamics differ, and indeed they are visibly different in the simulation. _____no_output_____$\quad$ Hayes' simulation is quite versitile: in real-time, one can adjust the temperature, as well as the 'mode' of the dynamics (Glauber or Metropolis). There is no external magnetic field present, but this is something one could easily insert, so his simulation is robust enough to reproduce the procedure with the iron bar. The first incarnation of our simulation will not be, but this is partly because there are some important questions to address even before scheduling a gradual change in temperature during the simulation, or making it more interactive. _____no_output_____$\quad$ It is written in Python, later to be migrated to javascript. We can currently simulates either type of Ising dynamics for a fixed number of steps, and at a fixed temperature. We do not yet include an external magnetic field. In either type of dynamics, the system (or image) is updated one pixel at a time, and we will perform a large number of updates, let us call this $N$. The output of the simulation is stored in dict format as a `.json` file. The purpose of the notes below (leading up to the simulation) is to develop some of the theory underlying the Monte Carlo method. Then we will be equipped to address the aforementioned questions, which are: how long the simulation must be run so that, for a sparse enough subsequence of the images:

* Each is sampled approximately from the [Gibbs probability measure](https://en.wikipedia.org/wiki/Gibbs_measure) governing the Ising model, and

* these samples are nearly independent?_____no_output_____$\quad$ The latter property is especially important for a longer term goal of the project, which is to use these subsequences of images as the dataset for a computer vision project. Importantly, the Gibbs measure we are sampling from always depends on a global temperature parameter $T$, leading to natural classification and regression tasks._____no_output________

_to explore______no_output_____$\quad$ Here is a starting point for formulating a classification task. Suppose that we can simulate Ising model dynamics well $-$ so that both bullet points above are satisfied $-$ at two _distinct_ temperatures. It feels natural to choose one of these, say $T_1$, above the critical temperature $T_c$ of the model, and the other, $T_2$ below it. We then create two image datasets $\mathcal{D}_1$ and $\mathcal{D}_2$ of equal size by simulating as described by the two points above. Merging these datasets to form what we call $\mathcal{D}$ leads to a natural classification problem: given an element $x \in \mathcal{D}$ chosen uniformly at random, determine whether it came from $\mathcal{D}_1$ or $\mathcal{D}_2$. _____no_output_____$\quad$ It seems the relevance of the Ising model to vision has been explored in depth. A lot of the text below draws from Govind Menon's [pattern theory](https://www.dam.brown.edu/people/menon/publications/pt-2020.pdf) notes, where a chapter is devoted to this: specifically an 'energetic' perspective applied to character recognition tasks. Menon's notes also reference a 1984 [paper](http://www.peterbeerli.com/classes/images/a/a1/Geman_geman_1984.pdf) which develops a way to perform image restoration and enhancement using the Ising model. This makes it a natural model to experiment with, and there are a few natural ML directions to play with longer term:_____no_output_____1. At the lower temperature, $T_2$, simulating the model is akin to an optimization problem: as $T \to 0$, the Gibbs measure concentrates on the set of lowest-energy states, or _ground states_. Any simulation designed to sample from a low-temperature Gibbs measure needs to navigate the energy landscape of the system from some starting point to a low altitude, with the temperature dictating a small range of desirable low altitudes $\equiv$ low energies. This is effectively an optimization problem with cost function corresponding to the energy landscape. There is also an optimization inherent to the process of training any neural network on the classification task suggested. The question to explore, loosely, is whether there are useful connections between these optimization problems. _____no_output_____
2. We will train a simple convolutional architecture on the dataset $\mathcal{D}$. Whether it is through pooling layers, or the local averaging inherent in the convolution operation, these architectures seem to apply a kind of [coarse-graining](https://en.wikipedia.org/wiki/Coarse-grained_modeling) to input signals $x \in \mathcal{D}$. This has prompted investigation into the connections between deep learning and the [renormalization group](https://en.wikipedia.org/wiki/Renormalization_group) of physics. The latter feels like a kind of microscope for theoretical physicists: the renormalization group formalizes a sequence of scaling transformations acting on a physical system, typically zooming out and thereby coarse-graining (lowering the resolution of) the preceding image of the system. This parallels the transformations undergone by an image fed into a "deep" sequence of convolutional layers, as this input signal propagates from one layer to the next. Two references exploring connections between renormalization and ML are a 2013 [paper](https://arxiv.org/abs/1301.3124) of Cédric Bény, and a 2014 [paper](https://arxiv.org/abs/1410.3831v1) of Pankaj Mehta and David Schwab. We imagine there is also more recent work on this to explore. Dataset $\mathcal{D}$ is both an image dataset and a natural input to the renormalization group. It's an opportunity to better understand these connections. In particular one can start by repeating the experiments the latter paper runs on the Ising model._____no_output_____# Markov chains, Monte Carlo methods, pattern theory_____no_output________

_references_

* [Werner Krauth](http://www.lps.ens.fr/~krauth/index.php/Main_Page) ... Statistical Mechanics: Algorithms and Computations

* [Govind Menon](https://www.dam.brown.edu/people/menon/) ...  [Pattern Theory](https://www.dam.brown.edu/people/menon/new_pub.html)

________no_output_____$\quad$ The Monte Carlo method is increasingly becoming part of the discipline it is meant to study. The Monte Carlo method is a statistical (Krauth: _"almost experimental"_) approach to computing integrals using random positions, called __samples__. Some sampling techniques for continuous and discrete variables are discussed below. In probabilistic terms, the integrals we wish to approximate correespond to the expected value of some observable, $\mathcal{O} : \mathcal{S} \to \mathbb{R}$. Here we assume some implicit probability measure $m$ on $\mathcal{S}$, the latter called the __state space__. Formally, the expected value $\mathbb{E}_m \mathcal{O}$ corresponds to the integral
$$
\mathbb{E}_m \mathcal{O} = \int_{ \mathcal{S} } \mathcal{O}(s) \,  m( \textrm{d} s )
$$
When $m$ is understood, we write this expectation either as $\mathbb{E} \mathcal{O}$, or as $\langle \mathcal{O} \rangle$, the latter notation more common in physics.   _____no_output_____$\quad$ We want to simulate most of the objects discussed throughout the notes, so we lose no generality in assuming that $\mathcal{S}$ is finite. $\mathcal{S}$ will be treated as a continuous object when it is more natural or convenient notationally. In the setting where $\mathcal{S}$ is finite, probability measures $m$ on $\mathcal{S}$ correspond to non-negative vectors $(m_i)_{i=1}^{\# \mathcal{S}}$ of length $\#\mathcal{S}$ whose entries sum to one. This vector is identical to the probability mass function (pmf) of $m$. We distinguish between two approaches for sampling from $m$: __direct sampling__ and __Markov chain sampling__. We postpone giving the definition of a Markov chain. _____no_output_____## direct sampling

_(Krauth 1.1.1)______no_output_____$\quad$ Krauth explains the concept of direct sampling through a game played with pebbles. The playing field is a large circle is inscribed into a large square, both drawn in the sand. In direct sampling, one has 'access' to the measure $m$ on $\mathcal{S}$. This means we can draw a collection of i.i.d. random variables $(X_i)_{i=1}^\infty$ taking values in $\mathcal{S}$, each with pmf corresponding to $m$. In the context of the game, we will assume we can directly sample from $m$, which here denotes the uniform measure on the square
$$
\mathcal{S} \equiv \textsf{square} = [-1,1]^2   \,,
$$
a 'birds-eye' view of the playing field. The observable considered in the game is related to the game's objective: to compute an approximation of the number $\pi$. The observable $\mathcal{O} : \mathcal{S} \to \mathbb{R}$ is the indicator function of the unit disc,
$\textsf{disc} = \{ s = (x,y) \in \textsf{square} : x^2 + y^2 \leq 1 \}$
$$
\mathcal{O} \equiv \mathbf{1}_{ \textsf{disc}} =
\begin{cases}
1 & \quad s \in \textsf{disc} \\
0 & \quad \textrm{otherwise}
\end{cases}
$$_____no_output_____$\quad$ A player throws pebbles at the square, successively. Through the direct sampling, these land uniformly at random. The sampling is performed independently across draws from $m$. In particular, no pebble throw ever misses the square. Let $S_1, \dots, S_N$ denote the collection of locations sampled from $\textsf{square}$ through the above procedure: each $S_i$ is a random ordered pair $S_i = (x_i, y_i) \in \textsf{square}$, where the collection of all coordinates, across all pairs, is mutually independent, with eeach coordinate a $\textrm{Unif}([-1,1])$ random variable.

_____no_output_____$\quad$ To describe how the samples $S_i$ are used to approximate the value of $\pi$, we introduce the __empirical measure__ $m_N^{\textrm{direct}}$, a random probability measure on $\mathcal{S}$, defined so as to encode the locations $S_1, \dots, S_N$:
$$
m_N^{\textrm{direct}} := \frac{1}{N} \sum_{i=1}^N \delta_{S_i},
$$
where in general $\delta_S$ denotes the delta mass at some random point $S$.  Measures are in a sense dual to sets, and the __delta mass__ $\delta_X$ is a measure acting on sets in the following way: given measurable $B \subset \textsf{square}$, we have
$$
\delta_S =
\begin{cases}
1 & \quad \textrm{ if } S \in B, \\
0 & \quad \textrm{ otherwise. }
\end{cases}
$$
The empirical measure $m_N^{\textrm{direct}}$, by definition, acts on sets $B \subset \mathcal{S}$ by reporting the fraction of the direct samples $S_i$ landing in $B$. _____no_output_____$\quad$ There is a sense in which the measures $m_N^{\textrm{direct}}$ 'converge' to $m$, and the game (or simulation) computes $\pi$ in a way which leverages this. Let us enumerate the states of $\mathcal{S} = \{s_1, \dots, s_M \}$. In practice, the size of $M$ will be determined by the resolution of the floating points used in sampling. In accordance with the observable $\mathcal{O}$, the set we choose for $B$ is $\textsf{disc}$. This is because
$$
m_N^{\textrm{direct}}( \textsf{disc} ) = \frac{1}{N} \sum_{i=1}^N \delta_{S_i}( \textsf{disc} ) \equiv \frac{1}{N} \sum_{i=1}^M \mathcal{O}(S_i)
$$
The sense in which $m_N^{\textrm{direct}}$ converges to $m$ implies, in particular, that the sequence of numbers $m_N^{\textrm{direct}}(\textsf{disc})$ converges to $m( \textsf{disc} )$ as $N \to \infty$. This convergence can also be seen as a consequence of a law of large numbers. The output of the first algorithm below is
$$
4 \cdot m_N^{\textrm{direct}}(\textsf{disc}) \approx 4 \cdot m( \textsf{disc}) \equiv 4 \cdot \frac{ \pi}{4}
$$
Thus we are approximating $\pi$ by approximating $\pi/4$, which is the probability that a pebble sampled from $m$ lands in $\textsf{disc}$. Python imports:_____no_output_____

#collapse-hide

import numpy as np
import sys
from numpy.random import default_rng

rng = default_rng()_____no_output_____

$\quad$ The next function embodies the 'direct' sampling we supposedly have access to, with the independence assumptions described above. _____no_output________

### Unif_____no_output_____

#collapse-hide
# uniform r.v. over interval [0,1]
def Unif():
    return rng.uniform()_____no_output_____

________no_output_____$\quad$ Algorithm 1.1 is translated from pseudocode (as it is presented in Krauth) to Python._____no_output________

### __Algorithm.__ ( direct_pi )_____no_output_____

#collapse-hide
def direct_pi(N = 4000):

    N_hits = 0

    """
    the for loop is basically about computing the sum in
    \mu_N(disc), stored in N_hits
    """

    for i in range(N):

        # initialize Unif([-1,1]) r.v.'s
        x = 2 * Unif() - 1
        y = 2 * Unif() - 1

        rad_sq = (x ** 2) + (y ** 2)

        if rad_sq <= 1:

            N_hits += 1
    """
    \mu_N(disc) is then computed by dividing by N
    """
    ratio = N_hits / N

    return 4 * ratio_____no_output_____

________no_output_____$\quad$ Running the above:_____no_output_____

#collapse-hide
direct_pi()_____no_output_____

$\quad$ In Krauth, Table 1.1 gives results of five runs of the above with `N = 4000`. We return to this table later to compare with Monte Carlo methods. He remarks that none of the results of this table have fallen in the error bounds known since Archimedes.

| run | `N_hits` | estimate of $\pi$ |
| -------- | -------- | --- |
| 1 | 3156 | 3.156 |
| 2 | 3150 | 3.150 |
| 3 | 3127 | 3.127 |
| 4 | 3171 | 3.171 |
| 5 | 3148 | 3.148 |

 Another remark of Krauth: _"The people adopt a sensible rule: they decide on the total number of throws, before they start the game."_ (This is as in the above python code.) _"They understood that one must not stop a stochastic calculation simply because the current result appears accurate, nor should they continue to play because the answer they get isn't close enough to their idealized target."_

_____no_output_____## Markov-chain sampling

_( Krauth 1.1.2 )______no_output_____$\quad$ Cconsider a new game played on the same field, but with different rules. We again take a birds-eye view of the playing field, so our state space is again $\mathcal{S} = \textsf{square}$. The game starts with a person at the corner of the helipad, corresponding to coordinates $(1,1)$ in $\mathcal{S}$. The walker takes a step (or attempts to) by adding independent, $\textrm{Unif}([-\delta, \delta])$ random variables to each coordinate. Thus it is possible for a next step which takes the person outside the square. If this is the case, instead of taking the step, the walker does nothing. This step still 'counts' in the aggregation to be performed. Krauth describes this procedure in terms of the pebbles: the person takes another pebble from their bag and places it on top of the one pebble (or perhaps pebble pile) marking their current location. After an 'in-bounds' step is taken, the walker places a pebble at their current position. Each walker is equipped with a bag of infinitely many pebbles. These two cases describe the way the new game is played. _____no_output_____$\quad$ In the present setting, the __samples__ $X_1, \dots, X_N$ are not independent, but rather have the structure of a Markov chain, specifically a random walk with independent increments. Let us use $\xi_1, \dots, \xi_N$ denote these independent increments. In Krauth's description, each of these is a $\textrm{Unif}([-\delta, \delta])$ random variable. For reasons that Krauth discusses later, and we will comment on this, we instead assume that each $\xi_j$ is a discrete random variable uniformly distributed on the two-element set $\{ -\delta, \delta \}$. We have distinguished the starting point of the sampling at $X_0 = (1,1)$. Following the above description, one then has
$$
X_1 =
\begin{cases}
X_0 + \xi_1\,, & \quad \textrm{ if } X_0 + \xi_1 \in \textsf{square}\\
X_0 & \quad \textrm{ otherwise }
\end{cases}
$$
and in general,
$$
X_j =
\begin{cases}
X_{j-1} + \xi_j\,, & \quad \textrm{ if } X_{j-1} + \xi_j \in \textsf{square}\\
X_{j-1} & \quad \textrm{ otherwise }
\end{cases}
$$
for all $j = 1, \dots, N$._____no_output_____$\quad$ For this new sequence of random $X_i$, we can define the analogous object to $m_N^{\textrm{direct}}$,
$$
m_N^{\textrm{Markov}} := \frac{1}{N} \sum_{i=1}^N \delta_{X_i}
$$_____no_output_____The algorithm below returns an approximation to $\pi$ via the same kind of ratio
$$
\pi \approx 4 \cdot m_N^{\textrm{Markov}}(\textsf{disc})
$$_____no_output_____$\quad$ To comment on avoiding random initial conditions, specifically those which assign independent $\textrm{Unif}([-1,1])$ random variables to the two coordinates: we avoid this because the Markov chain methods stand out when no direct sampling method exists. It is for exactly this reason that we also avoid directly sampling from $\textrm{Unif}([-\delta, \delta])$ random variables: such objects are a linear transformation away from directly sampling from the $\textrm{Unif}([-1,1])$ distribution, perhaps at lower resolution. So, our Markov-chain sampling simulations start at the corner $(1,1)$ for concreteness. They can also be assumed to start from where a previous simulation left off. Following this convention throughout these notes, we usually focus on going from configuration $i$ to configuration $i+1$. This is the defining property of (discrete-time) Markov chains: the transition probabilities of the process depend only on the current position.

_____no_output________

### __Algorithm.__ ( markov-pi, unmodified )_____no_output_____

#collapse-hide

def markov_pi( delta = .05, N = 40000 ):

    # initial conditions
    x = 1

    y = 1

    N_hits = 0

    num_rej_moves = 0 # number of "rejected" moves

    for i in range(N):

        # get coordinate increments
        del_x_big = 2 * Unif() - 1

        del_y_big = 2 * Unif() - 1

        del_x = delta * del_x_big

        del_y = delta * del_y_big

        #print(del_x, del_y)

        # potential new coordinates
        x_new = x + del_x

        y_new = y + del_y

        # to determine if step leads into disc
        rad_sq_new = ( x_new ** 2 ) + ( y_new ** 2 )

        # ditto for square
        x_in = x_new <= 1 and x_new >= -1

        y_in = y_new <= 1 and y_new >= -1

        in_square = x_in and y_in

        # if move is accepted
        if in_square:

            x = x_new

            y = y_new

        else:

            num_rej_moves += 1

        # if in disc, +1 to number of hits
        if rad_sq_new <= 1:

            N_hits += 1

    # we add in this output to
    # better understand how to pick
    # delta
    print( "fraction of successful moves: ", 1 - ( num_rej_moves / N) )

    return 4 * ( N_hits / N )
_____no_output_____

________no_output_____$\quad$ Running:_____no_output_____

#collapse-hide

markov_pi()fraction of successful moves:  0.976

$\quad$ Let's discuss the choice of 'throwing range', namely $\delta$. It is akin to (largest possible) step size for the walker. Roughly, if $\delta$ is too small, the acceptance rate may be high, but the claim is that the walk will not be able to explore enough to well-approximate the integral. Choosing $\delta$ too large means the walk will have a hard time making successful moves, also hindering exploration. Krauth: _"The time-honored rule of thumb consists in choosing $\delta$ neither too large, nor too small, so that the acceptance rate turns our to be on the order of $\frac{1}{2}$."_ I didn't observe results agreeing with their rule of thumb. I found much more success with $\delta = 0.05$ versus their recommendation of $0.3$. I did however observe the approximation worsen as the step size was taken larger past $\delta = 1$. _____no_output________

### __Algorithm.__  ( markov-pi, modified )_____no_output_____

#collapse-hide

# in this modification, steps are no longer uniform

def markov_pi_modified( delta = .05, N = 40000 ):

    # initial conditions
    x = 1

    y = 1

    N_hits = 0

    num_rej_moves = 0 # number of "rejected" moves

    for i in range(N):
        """
        we use uniform random variables, out of convenience but not to
        determine coordinate steps directly with these. Instead, we
        extract a pair of fair coin tosses from two uniform random
        variables. We use these coin tosses only to determine the sign
        of the increment in each coordinate.
        """

        # generate indep uniforms
        U_1 = Unif()

        U_2 = Unif()

        # use these as coin tosses to determine
        # del_x...

        if U_1 <= 0.5:

            del_x = delta

        else:

            del_x = (-1) * delta

        if U_2 <= 0.5:

            del_y = delta

        else:

            del_y = (-1) * delta

        # potential new coordinates
        x_new = x + del_x

        y_new = y + del_y

        # to determine if step leads into disc
        rad_sq_new = ( x_new ** 2 ) + ( y_new ** 2 )

        # ditto for square
        x_in = x_new <= 1 and x_new >= -1

        y_in = y_new <= 1 and y_new >= -1

        in_square = x_in and y_in

        # if move is accepted
        if in_square:

            x = x_new

            y = y_new

        else:

            num_rej_moves += 1

        # if in disc, +1 to number of hits
        if rad_sq_new <= 1:

            N_hits += 1

    # we add in this output to
    # better understand how to pick
    # delta
    print( "fraction of successful moves: ", 1 - ( num_rej_moves / N) )

    return 4 * ( N_hits / N )
_____no_output_____

________no_output_____$\quad$ Running:_____no_output_____

#collapse-hide

markov_pi_modified()fraction of successful moves:  0.94765

___

__Q__ $\quad$ In both the Glauber and Metropolis dynamics for the evolution of the Ising model, the spin variable to be updated ( i.e. the vertex $v \in \mathbb{T}$ sampled ) is chosen uniformly, through _direct sampling_. What if this is instead done through Markov-chain sampling? Also, instead of rejecting the moves outside the box, we can play with periodic boundary conditions for the walks in both simulations.
________no_output_____## discussion _____no_output_____$\quad$ The Monte Carlo 'games' described above epitomize the two basic approaches to sampling from a probability measure $\mu$ on some space $\mathcal{S}$. Both approaches to the sampling evaluate an observable $\mathcal{O} : \mathcal{S} \to \mathbb{R}$. In our examples, $\mathcal{S} = \textsf{square}$, and the observable $\mathcal{O}$ is the indicator function of the disc within the square:
$$
\mathbf{1}_{\textsf{disc}}(s) = \begin{cases}
1 & \quad s \in \textsf{disc} \\
0 & \quad \textrm{otherwise}
\end{cases} \,, \quad \mathbf{1}_{\textsf{disc}} : \textsf{square} \to \mathbb{R}
$$_____no_output_____In either sampling case, we can define $\mathcal{O}_i := \mathcal{O}(X_i)$, and note that one evaluates
$$
\frac{N_{\textrm{hits}}}{N} = \frac{1}{N} \sum_{i=1}^N \mathcal{O}_i \approx \mathbb{E}_m \mathcal{O},
$$
where $\mathbb{E}_m$ denotes the expected value taken with respect to the probability measure $m$, in this case (as before) uniform on $[-1,1]^2$. _____no_output_____$\quad$ In general, the pmf of $m$ doesn't appear in the approximation $\tfrac{1}{N} \sum_{i=1}^N \mathcal{O}_i$, Krauth: _"...rather than being evaluated, it is sampled. This is what defines the Monte Carlo method."_ He goes on to point out that the multiple integrals also disappear: the expectation is formally an integral in two-variables, but we are approximating it by a path-integral indexed by time. _"This means that the Monte Carlo method allows the evaluation of high-dimensional integrals, such as appear in statistical physics and other domains, if we can only think of how to generate the samples."_

$\quad$ Integration in two variables doesn't feel like a good example a 'high-dimensional' integral to compute. On the other hand, we are already know, through the motivating [blog post](http://bit-player.org/2021/three-months-in-monte-carlo), of a kind of Markov chain on a high-dimensional space, namely the Glauber dynamics for the Ising model on $\mathbb{T}$, the two-dimensional torus $( \mathbb{Z}\, /\, (n\cdot \mathbb{Z}) )^2$, with $n = 100$ for concreteness. All in all, it seems that one is approximating an integral in an arbitrary number of dimensions (number of spins in the case of Ising model) through what is essentially a path integral. The path integral is a sum of observables of the system, as the system evolves. _____no_output_____$\quad$ Krauth: _"Notwithstanding the randomness in the problem, direct sampling, in computation, plays a role similar to exact solutions in analyitcal work, and the two are closely related. In direct sampling, there is no throwing-range issue, no worrying about initial conditions, and there is a straightforward error analysis -- at least if $m$ and $\mathcal{O}$ are well behaved. Many successsful Monte Carlo algorithms contain exact sampling as a key ingredient. Markov-chain sampling, on the other hand, forces us to be much more careful with all aspects of our calculation. The critical issue here is the correlation time, during which the position of the walker retains a memory of the starting configuration. This time can become astronomical. In the usual applications, one is often satisfied with a handful of independent samples, obtained through week-long calculations, but it can require much throught and experience to ensure that even this modest goal is achieved."_
_____no_output_____## Markov chains

_(Menon 1.1, 1.2)______no_output_____$\quad$ The perspective Menon starts his notes with: we model the world as a space of random signals. The world is filled with observers, which record part of these signals and then form inferences about the world. The process of inference is modeled with a Bayesian paradigm. We assume an observer has a stochastic model capable of generating signals similar to the true signal. These assumptions reduce inference to tuning the model parameters to obtain the best match between the generated and observed signals. For instance, if we know we are observing a simulation of the Ising model, inference reduces to estimating the temperature._____no_output_____$\quad$ Menon's notes explore stochastic models of progressively increasing complexity, in tandem with numerical methods for optimization. The simplest class of stochastic models considered are Markov chains. Suppose we are given a finite set $\mathcal{S}$ called the **state space** of the Markov chain. This will sometimes (such as in text-modeling instances) be called the **alphabet**, in the context of language modeling. In the 'linguistic' interpretation of the Ising model dynamics, either Glauber or Metropolis dynamics are run to generate a sequence of spin configurations $s_1, \dots, s_N \in \mathcal{S}$. Each spin configuration (a binary image) is interpreted as a letter in the alphabet of some abstract language. An alphabet of astronomical size can seem useless, linguistically. The notion of a large alphabet, perhaps not astronomically so, seems to occur naturally when viewing language through a _coarse-grained_ perspective. This is how we naturally read: not one letter at a time, but one word. The set of english words can then interpreted as an alphabet, one with around a million symbols. This touches upon the heirarchical structure of language (I'm not talking about Chomsky's heirarchy of languages, only the fact that our alphabet clusters into words, and those into sentences and so on). _____no_output________

### **Definition.** ( stochastic process )

$\quad$ A (discrete-time) **stochastic process** is a sequence of random variables.
________no_output_____$\quad$ We study a stochastic process $(X_t)_{t \in \mathbb{N}}$ taking values in the given finite set $\mathcal{S}$. Thus, the temporal (given by the index $t$ in the stochastic process) and the spatial (given by the state value in $\mathcal{S}$ of the process) aspects of this process are discrete. _____no_output________

### **Definition.** ( Markov chain )

$\quad$ The stochastic process $(X_t)_{t \in \mathbb{N}}$ is a **Markov chain** if for all $t,\,$ $\mathbb{P} ( X_{t +1} | X_1, \dots, X_t ) = \mathbb{P} (X_{t+1} | X_t )
$.

________no_output_____$\quad$ As the above definition indicates, the modeling assumption in Markov process theory is that the future of the process depends on the current state, but not the past. This is a probabilistic analogue of Newtonian determinism, encoded by the _forward equation_ below. Stationary Markov chains are thus characterized by their __transition matrix__
$$
Q(s, \tilde{s}) = \mathbb{P}( X_2 = \tilde{s} | X_1 = s )\,,
$$
with entries indexed by pairs of states $s, \tilde{s} \in \mathcal{S}$. Another interpretation of the Markov property is that the future and past are independent, conditional on the present. For simplicity, we usually assume Markov chains in consideration have the following structure:
_____no_output_____
___

### **Definition.** ( stationary Markov chain )

$\quad$ The Markov chain $(X_t)_{t \in \mathbb{N}}$ is **stationary** if for all $t$, $\mathbb{P}(X_{t+1} | X_t ) = \mathbb{P} (X_2 | X_1 )$.
________no_output_____$\quad$ The most basic statistic associated to the Markov chain is the pmf of the random variable $X_t$. The mass assigned to state $s \in \mathcal{S}$ under this law is denoted
$$
m_t(s) := \mathbb{P}(X_t = s) \,,
$$
and one obtains the following recurrence relation using the so-called law of total probability, along with the definitions of $m_t$ and $Q$:
\begin{align}
m_{t+1}(\tilde{s}) &= \mathbb{P} ( X_{t+1} = \tilde{s}) \\
&= \sum_{s \in \mathcal{S}} \mathbb{P}( X_{t+1} = \tilde{s}, \, X_t = s ) \\
&= \sum_{s \in \mathcal{S}} \mathbb{P} (X_{t+1} = \tilde{s} | X_t = s ) \mathbb{P}(X_t = s)  \\
&= \sum_{s \in \mathcal{S}} Q(s,\tilde{s}) m_t(s)\,,
\end{align}
when we assume the chain is stationary. This calculation yields the **forward equation**
$$
m_{t+1} = m_t Q , \quad t \in \mathbb{N}
$$
using the convention that the pmf $m_t$ of $X_t$ is regarded as a row vector.  The forward equation is linear and explicitly solvable when the chain is stationary. We proceed inductively to find
$$
m_t = m_1 Q^t , \quad t \in \mathbb{N} \,,
$$
and one of the central questions in Markov chain theory is to understand the behavior of $m_t$ as $t\to\infty$. This explains the importance of the eigenvalue spectrum of $Q$ to Markov chain theory: diagonalizing $Q$ is an efficient way to compute $Q^t$ for $t$ large, and the spectrum of $Q$ largely governs the long term behavior of the system.
_____no_output_____## equilibrium measures, ergodicity, random walks

_(Menon 1.2)______no_output________

### **Definition.** ( equilibrium / stationary distribution )

$\quad$ Let $Q$ be the transition matrix of a stationary Markov chain $(X_t)_{t \in \mathbb{N}}$ with state space $\mathcal{S}$, and let $m$ be a probability measure on $\mathcal{S}$, whose pmf we also denote $m$. The measure $m$ is an **equilibrium distribution** or equivalently, a **stationary distribution** for $(X_t)_{t \in \mathbb{N}}$ if
$$
m = m Q,
$$
and if $m$ satisfies the normalization conditions $m(s) \geq 0$ for all $s \in \mathcal{S}$ and $\sum_{s \in \mathcal{S}} m(s) =1$.

________no_output_____$\quad$ The above definition is central to our simulation goals. For Markov chains with a unique stationary distribution $m$, the measures $m_t \equiv \mathbb{P}(X_t = \cdot )$ converge to $m$, in an appropriate sense, as $t \to \infty$. When we simulate a Markov chain, we are sampling from $m_t$, and the idea is to run the simulation long enough (taking $t$ large enough) so that $m_t \approx m$. This is how a Markov chain Monte Carlo (MCMC) simulation can be used to approximate sampling from $m$. The MCMC method then requires _finding_ a stationary ergodic Markov chain in $\mathcal{S}$, whose stationary measure coincides with some $m$ we started with, and wish to sample from. _____no_output_____$\quad$ Krauth discussed the initial conditions of a Markov chain in the context of simulation: in the second pebble game, the person playing always had to start at a prescribed corner of the square. These are _deterministic_ initial conditions. In practice, we assume simulations will start deterministically, or from where a previous simulation left off. In theory, it is useful to have a mathematical object associated to the __initial conditions__ of a Markov chain: a probability measure on $\mathcal{S}$, denoted $m_0$. When initial conditions are deterministic, and start at some distinguished state $s_* \in \mathcal{S}$, the probability measure $m_0$ corresponds to the $\delta$-mass $\delta_{s_*}$. When $m_0$ is general, it models starting from where some simulation left off, whose information we have no (or only partial) access to.

$\quad$ The first equation in the above definition can be understood as describing the evolution of a Markov chain $(X_t)_{t \in \mathbb{N}}$ whose initial conditions are the stationary distribution $m$ associated to $(X_t)_{t \in \mathbb{N}}$. In this case $m_t = m$ for all $t$ if $m_1 = m$. That is, while the values of $X_t$ change with time (it's a stochastic process) its probability mass function does not. Menon: _"the equilibrium distribution of a Markov chain captures the intuitive idea of a dynamic equilibrium."______no_output_____$\quad$ Considering the task of computing $m$ given $Q$: $\,$ observe that $m$ is a left-eigenvector of $Q$ with eigenvalue $1$. Further, $1$ must always be an eigenvalue of $Q$ since it corresponds to the right-eigenvector $(1, \dots, 1)^T$. Thus, if we know how to solve for eigenvectors, we can determine $m$. Menon: _"what is interesting in practice is the converse: for large transition matrices, say when $\# \mathcal{S}$ is $10^6 \times 10^6$, it is more efficient to use Markov chains to approximate $m$, than to use naive linear algebra. This observation plays an important role in web crawlers and search engines."______no_output________

### **Definition.** ( ergodic Markov chain )

$\quad$ A stationary Markov chain $X$ with state space $\mathcal{S}$ and transition matrix $Q$ is **ergodic** if it has a unique equilibrium distribution. A sufficient condition for ergodicity of $X$ is if for every pair $s, \tilde{s} \in \mathcal{S}$ there is a positive integer $j(s,\tilde{s})$ such that $Q^{j(s,\tilde{s})} >0 $.
________no_output_____$\quad$ Many other conditions are known that ensure ergodicity. For example, if all terms of $Q$ are strictly positive, it is possible for any state to get to any other state, and this ensures that the Markov chain is ergodic. This assumption is too restrictive in practice, since we are mainly intersted in Markov chains making 'local' moves such as random walks in a large state space. This behavior is captured by the sufficient condition given in the above definition. Intuitively, it means that every state can visit every other state _given enough time_. We assume this property always holds._____no_output_____$\quad$ Physicsts define the term ergodic ( describing Markov chain $(X_t)_{t\in \mathbb{N}}$ ) to mean the following.
$$
\lim_{T \to \infty} \sum_{t =1}^T \mathcal{O}(X_t) = \mathbb{E}_m\mathcal{O},
$$
where $m = m Q$ and $\mathcal{O}: \mathcal{S} \to \mathbb{R}$ is arbitrary. This formulation has the interpretation that a Markov chain is ergodic when the left-hand side, the _time-average_, is equal to the _spatial-average_ (with respect to the equilibrium measure), which is the right hand side. Menon notes the following bottlenecks and simplifications:

* $\mathcal{S}$ may be very large. For example (in shuffling dynamics we soon describe), $\mathcal{S}$ may be the permutation group.

* $Q$ may be a sparse matrix.

* As discussed previously, we often don't need $m$ explicitly, we instead want to evaluate $\mathbb{E}_m \mathcal{O}$, where $\mathcal{O}: \mathcal{S} \to \mathbb{R}$ is some observable.  _____no_output________

### **Example.** ( simple random walk )

$\quad$ Let $G = (\textrm{V},\textrm{E})$ be a finite graph with vertex set $\textrm{V}$ and edge set $\textrm{E}$. Given $x,y \in \textrm{V}$, write $x \sim y$ if the edge $\{ x, y \}$ lies in $\textrm{E}$. Let $\textrm{deg}(x) \triangleq \sum_{x \sim y } 1$ be the degree of vertex $x$. A **simple random walk** on $G$ is a Markov chain with state space $\textrm{V}$ and transition matrix
$$
Q(x,y)=
\begin{cases}
\frac{1}{\textrm{deg}(x)}, & \quad \text{ if } x \sim y \\
0 & \quad \text{ otherwise. }
\end{cases}
$$
We leave the initial conditions ambiguous in this definition.
________no_output________
### **Example.** ( random walks in the permutation group )

$\quad$ Let $\mathfrak{S}_n$ denote the symmetric group on $n$ symbols, the collection of permutations on the set
$$
[n] := \{ \, 1, \dots, n \, \},
$$ A permutation on $[n]$ is a bijective function from $[n]$ to itself, and the group structure is given by function composition. Below we describe two _distinct_ Markovian dynamics on $\mathfrak{S}_n$ with the same stationary measure. These dynamics arise from putting distinct graph structures (writing $\sim'$ and $\sim''$ in this example) on a set of vertices indexed by $\mathfrak{S}_n$, and considering the associated simple random walk.

1. We declare permutations $\sigma$ and $\tau$ adjacent and write $\sigma \sim' \tau$ if $\sigma$ and $\tau$ are related by a transposition of adjacent elements. The $'$-degree of each permutation in this graph is $n-1$.

2. We declare permutations $\sigma$ and $\tau$ adjacent and write $\sigma \sim'' \tau$ if $\sigma$ and $\tau$ are related by a transposition, not necessarily of adjacent elements. The $''$-degree of each permutation in this graph is ${n \choose 2}$.

These are examples of _distinct_ ergodic random walks on the same state space, namely $\mathfrak{S}_n$, both of which have the uniform measure as their equilibrium measure. The simulation 'purpose' of both random walks is thus to sample from this uniform measure, and the distinctness of the two walks raises the question of whether one method does the sampling better. This is a well-studied question, when $n = 52$ these two walks describe simple procedures for shuffling a deck of cards.
________no_output_____## Shannon's model of text

_( Menon 1.3 )______no_output_____$\quad$ The main idea in Shannon's model (of text / written language) is that text is text is a stationary, ergodic stochastic process $(X_t)$ taking values in an alphabet
$$
\mathcal{A} \triangleq \{\, a, \,b,\, c,\, \dots\,,\, z,\, \_ \,\}.
$$
We use $\mathcal{A}$ for this specific alphabet for clarity in examples which follow. This model of language is easily augmented to include punctuation and case, but we ignore this in the first approximation. We are not assuming that this process is a Markov chain. As discussed, the alphabet is the state space $\mathcal{S}$ from previous sections. Stationarity is distinct from the Markov property $-$ neither property implies the other._____no_output_____
___

### **Definition.** ( stationary stochastic process )

$\quad$ A discrete time stochastic process $X \equiv (X_t)$ is **stationary** if all abitrarily large, but finite, marginal probabilities are invariant under arbitrary shifts in time. More precisely,
$$
\mathbb{P} (\, X_1 = s_1, \dots, X_n = s_n\,)
= \mathbb{P} ( \,X_{1+k} = s_1, \dots, X_{n+k} = s_n \,)
$$
for all positive integers $k$ and $n$.
________no_output_____$\quad$ Stationarity means that if we were to observe strings of length $n$, their statistics are not changed by shifting them forward in time by an integer $k$. For such processes, the notion of ergodicity again means that _time averages_ are equal to _spatial averages_. From the perspective of observables $\mathcal{O} : \mathcal{S}^n \to \mathbb{R}$, which through the above definition requires have an arbitrarily large 'memory' $n$ of the process, one has
$$
\lim_{T \to \infty} \frac{1}{T} \sum_{t =0 }^{T-1} \mathcal{O}(X_{t+1}, \dots, X_{t+n}) =
\mathbb{E}  \mathcal{O}( X_1, \dots, X_n)
$$_____no_output_____$\quad$ Any discussion above of language and heirarchy was probably motivated by Menon, who among other things says:  _"the above assumptions are introduced to conform to the idea that written text contains many heirarchical structures: letters are joined by phonetic rules to form words, words are linked by the rules of grammar into sentences, sentences are organized into paragraphs, and so on. Strange as it may seem at first sight, these rules can be effectively modeled as a stochastic process. The use of stationary ergodic processes provides us with a definition flexible enough to include heirarchical structures (though these may be complex to write down), and simple enough to computationally test."______no_output_____$\quad$ As discussed, written text is modeled by a stationary stochastic process $Y$, whose law is denoted
$$
\mathbb{P}_{\textrm{true}} .
$$
The subscript '$\textrm{true}$' indicates that we view $Y$ not as an approximation to language, but as a stream of 'correct' English text, generated in real-time. This idealization is vague, but the main point is to identify the written language $\mathcal{L}$, in this case English, with a concrete mathematical object: the stationary, stochastic process $Y$. The law $\mathbb{P}_{\textrm{true}}$ of this process is the analogue of $m$ in two previous contexts: the uniform measure on $\mathcal{S} = \textsf{square}$, and the uniform measure on $\mathcal{S} = \mathfrak{S}_{52}$. In either previous case, the purpose of the Markov chains involved was to approximately sample from $m$. Below, we discuss a family of Markov chains for approximately sampling from $m = \mathbb{P}_{\textrm{true}}$. The way in which this sampling happens here seems distinct from before, and is remarked on after we introducing the Markov chains approximating $\mathcal{L}$ below. Menon: "In practice, the probabilities relative to $\mathbb{P}_{\textrm{true}}$ below are approximated by mining a large corpus (e.g. the works of Shakespeare) to approximate $\mathbb{P}_{\textrm{true}}(a_1, \dots, a_n)$ for an arbitrary string $(a_1, \dots, a_n) \in \mathcal{A}^n$."_____no_output________

### **Example.** ( digram model )

_( or 2-gram model )_

$\quad$ This is a Markov chain with state space $\mathcal{A} = \{ \, a,\, b, \, c, \, \dots \, , \,z, \, \_ \}$.
$$
Q^{(2)}(x,y) = \mathbb{P}_{\textrm{true}} (\,X_2 = y | X_1 = x\,),
$$
which is to say, we form the character-to-character transition probabilities of a Markov process using the marginals of the full stationary process being approximated.

________no_output_____$\quad$ We can get better approximations by allowing the Markov chain above to retain more of its history by augmenting the state space to $\mathcal{A}^n$ for some $ n \geq 2$. When $n=2$, we obtain the so-called trigram model. _____no_output_____

___

### **Example.** ( trigram model )

_( or 3-gram model )_

$\quad$ This is a Markov chain with state space $\mathcal{A}^2$, which we style as
$$
\{ aa, \,ab,\,, \dots, az, \dots, \_a, \, \dots, \, \_\,\_ \}
$$
Let $x = a_1a_2$ and $y = b_1b_2$. In order to form a text of three letters from $x$ and $y$, we must ensure that $x$ and $y$ agree on their overlap, that is when $a_2 = b_1$, so that $x$ and $y$ combine to give us the string $a_1a_2b_2$. With this restriction on $x$ and $y$, we obtain the associated transition matrix
$$
Q^{(3)} (x,y)
= \frac{\mathbb{P}_{\textrm{true}} ( a_1 a_2 b_2 ) }{
\mathbb{P}_{\textrm{true}}(a_1 a_2) }
$$
and $Q(x,y) =0$ when $a_2 \neq b_1$.

> Because of the compatability requirement of $a_2 = b_1$, the above Markov chain can still be viewed as a stochastic process taking values in $\mathcal{A}$, it is just no longer a Markov chain from this perspective. This is the sense, however, in which this model is approximating $\mathbb{P}_{\textrm{true}}$.

________no_output_____

___

### **Example.** ( $(n+1$)-gram model )

$\quad$ A Markov chain with state space $\mathcal{A}^{n}$. A state $x$, say $x = (a_1, \dots, a_n)$ can be followed by a string $y = (b_1, \dots, b_n)$ only if $a_2 = b_1, a_3 = b_2, \dots, a_n = b_{n-1}$. This induces transition matrix
$$
Q^{(n+1)} (x,y)
= \frac{\mathbb{P}_{\textrm{true}} ( a_1 a_2 \dots a_n b_n )}{\mathbb{P}_{\textrm{true}} (a_1 a_2 \dots a_n) }
$$
as above.

> Following the previous remark, for similar compatibility reasons, this Markov chain can also be interpreted as a stationary stochastic process taking values in $\mathcal{A}$. As before, this is the sense in which the above process approximates $\mathbb{P}_{\textrm{true}}$.
________no_output_____$\quad$ Menon: _"Shannon proved that as $n \to \infty$, the law of text generated by the above Markov approximations converges to the law of the true language. Note however that a good proof may not correspond to a good algorithm. For example, as $n$ increases, the size of the state space ( of the approximating Markov process ) grows exponentially. Thus the dimensions of the transition matrix are $\#\mathcal{A}^{2n}$, and worse yet, the size of the training data ( to determine $Q^{(n+1)}$ by mining ) also expands exponentially. This follows the rules of everyday language. Once $n$ gets large enough, say $5$ or $6$ in practice, it is much more efficient to make our fundamental unit words, rather than letters, since the number of true words of length $5$ is much lower than the $26^5$ combinations that are possible. This reflects the true nature of the space $\_$ character. In effect, we are still using the digram model, though we have switched to a new 'alphabet' whose fundamental units are words."______no_output_____## information theory_____no_output______references_

* [Ivan Matic](https://mfe.baruch.cuny.edu/maticivan/), _Information Theory and Sanov's Theorem_ ([his notes](https://the-ninth-wave.github.io/matic.pdf) from July 21, 2018)

* Thomas Cover, _Elements of information theory, Ch. 2______no_output_____$\quad$ Before going further, let us return to the object $\mathbb{P}_{\textrm{true}}$, playing the role of $m$ in previous examples. When recalling previous examples, we identified $m$ as the uniform measure on $\mathcal{S}$, in either case. To be formal, $m = \mathbb{P}_{\textsf{true}}$ is a probability measure on $\mathcal{S} \equiv \mathcal{A}^\infty$, where
$$
\mathcal{A}^\infty := \{ a \equiv (a_j)_{j=1}^\infty : a_j \in \mathcal{A} \textrm{ for all } j \}\,,
$$
the space of infinite sequences of elements of $\mathcal{A}$. Any discrete-time $\mathcal{A}$-valued stochastic process, given some initial conditions which we ignore, induces a probability measure on $\mathcal{A}^\infty$. This latter space has the interpretation of a 'path-space', as it is meant to contain all possible trajectories of the process. _____no_output_____$\quad$ The digram model naturally gives rise to a Markov chain with state space $\mathcal{A}$. Let us say that it induces measure $\mathbb{P}_{(2)}$ on $\mathcal{A}^\infty$. The trigram model is defined as a Markov chain with state space $\mathcal{A}^2$, but following the above remark, it can also be interpreted as a stationary stochastic process taking values in $\mathcal{A}$, inducing a measure $\mathbb{P}_{(3)}$ on the path-space $\mathcal{A}^\infty$. In general, the $(n+1)$-gram model induces measure $\mathbb{P}_{(n+1)}$ on $\mathcal{A}^\infty$. Our interpretation of what Menon says at the beginning of the most recent quote, is _"Shannon proved that..."_ $\mathbb{P}_{(n+1)}$ converges to $\mathbb{P}_{\textrm{true}}$, in an appropriate sense.

> The approximation of $\mathbb{P}_{\textrm{true}}$ by $\mathbb{P}_{(n+1)}$ does _not at all_ seeem to be an approximation of the same nature was is described in Krauth. By this I mean that $\mathcal{A}$-valued Markov chains do not play the same role in each case. In earlier cases, they served as a way to approximately sample from a probability measure on $\mathcal{A}$. In the present context, these objects induce a probability measure on the _path-space_ $\mathcal{A}^\infty$, towards approximating $\mathbb{P}_{\textrm{true}}$, the probability measure on $\mathcal{A}^\infty$ encoding the '$\textrm{true}$' language.  _____no_output_____$\quad$ Matic's add to the above discussion, as well as to Menon's upcoming discussion of entropy. Menon: _"the entropy of a random variable describes its uncertainty. In particular, the notion of an optimal code tells us that entropy describes the optimal search procedure to determine the value of $X$ through a series of yes / no questions."_ Matic's notes touch on this optimality, from the perspective of data compression. _____no_output_____$\quad$ To integrate Matic's perspective, he starts by imagining _"...we want to store some big quantity of data, such as a picture, a text document, or a book. We start with a collection of uncompressed data, and we identify the smallest building blocks of each datum as symbols. The set of all symbols is called the alphabet."_ Matic also denotes this alphabet as $\mathcal{A}$.

> It feels worth remarking that languages like Chinese are pictoral, and that Chinese characters can be decomposed into 'brushstrokes'. A space character `_` perhaps corresponds to moving to the next character. This lends itself to the notion that pictoral data can be decomposed into symbols. For images in general, I think 'contours' become the analogue of brushstrokes. Perhaps an efficient way to use a space character `_` is as denoting the start of a new transparent canvas layer, sitting over the previously drawn layers (copying the functionality of applications like photoshop). My impression of $3$-dim spatial data (e.g. virtual environments for games) is that environment information is efficiently stored in surfaces, the higher-dimensional analogue (by one dimension) of contours. _____no_output_____> The above decomposition reminds me of something I heard about the clustering algorithm [t-SNE](https://towardsdatascience.com/t-sne-clearly-explained-d84c537f53a): when applid to MNIST, the number of clusters the algorithm found was more than the number of categories (labeled by digits $0 - 9$). Some digits corresponded to multiple clusters _because_ of there being distinct ways to write this digit, with or without a horizontal slash in the case of $7$, for example. In a sense, the clustering works too well. _____no_output_____$\quad$ Uncompressed data is called a __signal__. Mathematically, we model signals as elements of $\mathcal{A}^M$. Ideally, the signal is sampled from the measure $\mathbb{P}_{\mathcal{L}}$ on $\mathcal{A}^\infty$ corresponding to abstract language $\mathcal{L}$ over alphabet $\mathcal{A}$. This measure corresponds to a stochastic process $(X_t)_{t \in \mathbb{N}}$, which we suppose is stationary. Because of the 'optimal encoding' perspective, it is important to be able to quantify the relationship between the lengths of a signal and its encoding. In particular, we assume some upper bound $M$ on the signal length to begin with, modeling signals as elements of $\mathcal{A}^M$ for some $M \in \mathbb{N}$. When referring to $\mathbb{P}_{\mathcal{L}}$ in this context, it is understood as the projection of this measure onto $\mathcal{A}^M$, corresponding to the 'truncated' process $(X_t)_{t = 1}^{M}$, which is a random signal._____no_output_____$\quad$ Our analysis takes place within an even simpler setting versus the language models discussed. Suppose $\mathbb{P}_{\mathcal{L}}$ corresponds to an i.i.d. sequence of random variables $(X_t)_{t = 1}^M$, where each $X_t$ has the same distribution as some 'model' random variable $X$, corresponding to probability measure $m$ on $\mathcal{A}$. Because $\mathcal{A}$ is finite, so is the space of signals $\mathcal{A}^M$. A lossless data compression is an injective function on this signal space. _____no_output________
### __Definition.__ ( lossless data compression )

$\quad$ A __lossless data compression__ is an injective function
$$
f : \mathcal{A}^M \to \bigcup_{j=1}^N \{ 0 , 1 \} ^j
$$
for some $N \in \mathbb{N}$.
________no_output_____$\quad$ Given some lossless data compression $f$, and for each signal $s \equiv (s_1, \dots, s_M) \in \mathcal{A}^M$, we let $\textsf{length}_f(s)$ denote the length of the signal encoding $f(s)$. Letting $\mathbb{E}$ denote expectation with respect to the 'model' random variable $X$, desirable propereties of the compression $f$ make
$$
\mathbb{E} \, \textsf{length}_f(X)
$$
as small as possible. Heuristically, this means __(1)__ identifying the signals in $\mathcal{A}^M$ which are _typical_ and _atypical_, according to the sampling $(X_t)_{t=1}^M$, and then __(2)__ using shorter binary sequences to encode typical signals, leaving longer binary sequences for atypical signals. This means the expectation is placing as little probablistic mass as possible on the longest encodings. _____no_output_____$\quad$ Let us use $m$ to denote the probability measure on $\mathcal{A}$ induced by the random variable $X$, our template for the i.i.d. sequence. Enumerating $\mathcal{A}$ as $\{ a_1, \dots, a_n\}$, let us write $m_i$ for the number $m( \{ a_i \})$. We identify $m$ with the vector $(m_1, \dots, m_n)$, which is the probability mass function of $X$. It is $m$ that determines how 'typical' a given sequence is._____no_output_____$\quad$ Let $s \equiv (s_1, \dots, s_M) \in \mathcal{A}^M$. Write $\underline{X}$ to denote the random vector $(X_t)_{t=1}^M$. The mutual independence of the $X_t$ implies
$$
\begin{align}
\mathbb{P}( \, \underline{X} = s ) &= \mathbb{P} (X_1 = s_1) \dots \mathbb{P}(X_M = s_M) \\
&= 2^{ \sum_{t=1}^M \log_2\mathbb{P}(X_t = s_t)}\\
&= 2^{ \sum_{i=1}^n n_i \log_2 m_i}
\end{align}
$$
where, in going from the second to third line, we have reindexed the sum over the alphabet, using that $m_i \equiv \mathbb{P}(X = a_i)$, and defining
$$
n_i := \# \{ \, t \in \{ 1, \dots, M \} : X_t = a_i \, \}.
$$
Thus, $n_i$ is the number of times symbol $a_i$ appears in the random signal $\underline{X}$. A typical message is one  in which the frequency of symbol $a_i$ is close to the corresponding probability $m_i$. Thus, for typical messages, we have $n_i \approx m_i M $, and the probability of such a message is approximately
$$
2^{ M \sum_{i=1}^n m_i \log m_i }
$$_____no_output________

### __Definition.__ ( entropy )

$\quad$ Let $m$ be a probability measure on finite state space $\mathcal{A}$ with $\# \mathcal{A} = n$, corresponding to probability mass function $(m_1, \dots, m_n)$. The __entropy__ of $m$ is
$$
\textsf{S}(m) := - \sum_{i=1}^n m_i \log_2 m_i
$$

________no_output_____$\quad$ Let $X$ be a random element of $\mathcal{S}$ corresponding to the probability measure $m$. Cover has the following remark: _"Note that entropy is a functional of the distribution of $X$. It does not depend on the actual values taken by the r.v. $X$, only on the probabilities."______no_output_____$\quad$ Note that $\textsf{S}(m) \geq 0$, and is $=0$ if and only if $m$ corresponds to a deterministic object, namely a $\delta$-mass at some symbol in the alphabet, using the convention $0 \log 0 = 0$. In the above setting, we are imagining the length $M$ of the signal is much larger than the size $n$ of the alphabet $\mathcal{A}$. This is so that the law of large numbers has a chance to kick in for each symbol. In this setting, typical messages appear with probability on the order of $2^{-M \textsf{S}(m)}$. When $m$ is non-trivial, the positivity of $\textsf{S}(m)$ implies that typical signals are _exponentially rare_, in the signal length $M$. _____no_output_____$\quad$ A consquence of the intrinsic high-dimensionality of $\underline{X}$ -- which corresponds to a probability measure on $\mathbb{R}^M$, with $M$ large -- is that signals with symbol frequencies _close_ to the corresponding $m_i$ are very common. The next definition formalizes this._____no_output________

### __Definition.__ ( $\epsilon$-typical signal )

$\quad$ For fixed $\epsilon > 0$, we say $s \in \mathcal{A}^M$ is __$\epsilon$-typical__ if
$$
\mathbb{P}( \, \underline{X} = s ) \in \left( 2^{-M(\textsf{S}(m) +\epsilon) }, \, 2^{-M(\textsf{S}(m) -\epsilon) } \right) \,,
$$
and the set of all $\epsilon$-typical signals is denoted $\mathcal{A}^M_\epsilon$.

________no_output_____$\quad$ As discussed, the goal is to efficiently compress signals $\underline{X}$ drawn from the measure $m$. The relevance of the above defininition to compression is:

1. we first prove that $\underline{X}$ is $\epsilon$-typical with high probability

2. we will see that the set of typical signals is nonetheless small: the number of all possible signals is $\# \mathcal{A}^M \equiv n^M$, and there are only approximately $2^{M\textsf{S}(m)}$

these facts combine to form a compression strategy: when we see a typical message, we begin its binary encoding (towards defining the lossless compression function $f$) with `0`, and we use $M \textsf{S}(m)$ bits to compress it. If the message is atypical, we begin its binary encoding with a `1`, and are more sloppy with how many bits we use to store. The formal statement of the first point is as follows._____no_output________
### __Theorem.__ ( Matic, Theorem 1)

$\quad$ For each $\delta > 0$, there is $M_0 \in \mathbb{N}$ such that for every $M \geq M_0$ there is a lossless data compression $f$ in which the average length of a message satisfies

$$
\mathbb{E} \textsf{length}_f ( \,\underline{X} ) \leq M \textsf{S}(m) + \delta
$$
___

_( we defer recording the proof )______no_output_____$\quad$ Matic: _"The algorithm constructed (in the above proof) creates a lossless compression. It is very effective when the raw data consists of independent components. The `png` format uses a better algorithm."_ To discuss this algorithm, Matic introduces what he calls _types_. These objects were previously introduced as the frequencies $n_i$ above._____no_output________

### __Definition.__ ( the type of a signal )

$\quad$ Given a signal $s \equiv (s_t)_{t=1}^M \in \mathcal{A}^M$, the __type__ of $s$ is the following probability measure on $\mathcal{A}$, constructed through the symbol frequencies of $s$: for $a \in \mathcal{A}$,
$$
m_s( \{ a \} ) := \frac{1}{M} \sum_{t = 1}^M \mathbf{1}\{ s_t = a \}
$$

________no_output_____$\quad$ As a direct consequence of the above definition, two signals $s, \tilde{s} \in \mathcal{A}^M$ have the same type if and only if the coordinates of $\tilde{s}$ form a permutation of the coordinates of $s$. Let $\textsf{types}(\mathcal{A},M)$ denote the collection of all types induced by signals in $\mathcal{A}^M$. The map $\mathcal{A}^M \to \textsf{types}(\mathcal{A},M)$ is effectively a quotient map, and given a probability measure $m \in \textsf{types}( \mathcal{A}, M)$, we let $\textsf{signals}(m)$ denote the collection of signals whose type is $m$:
$$
\textsf{signals}(m) = \{ s \in \mathcal{A}^M : m_s \equiv m \}
$$_____no_output________

### __Theorem.__ ( Matic, Theorem 2 )

Consider the set of types, namely $\textsf{types}( \mathcal{A}, M)$, with $\# \mathcal{A} = n$. One has

1. $\#\, \textsf{types}( \mathcal{A},M) = {M + n -1 \choose M }$

2. $\# \, \textsf{types}( \mathcal{A}, M) \leq (M+1)^n$

________no_output_____$\quad$ Let $\mu$ be a general probability measure on $\mathcal{A}$, corresponding to a random variable $Y$ and let $\underline{Y} \equiv (Y_t)_{t=1}^M$ be a sequence of i.i.d. random variables. Given fixed signal $s \in \mathcal{A}^M$, one has
\begin{align}
\mathbb{P}( \, \underline{Y} = s ) &= \prod_{t=1}^M \mu( Y_t = s_t) \\
&= 2^{ \sum_{t=1}^M \log_2 \mu( Y_t = s_t ) } \\
&= 2^{ \sum_{a \in \mathcal{A} } \sum_{t =1}^M \mathbf{1} \{ X_t = a \} \log_2 \mu(Y_t = s_t)} \\
&= 2^{ \sum_{a \in \mathcal{A} } \sum_{t =1}^M \mathbf{1} \{ X_t = a \} \log_2 \mu(Y_t = a)} \\
&= 2^{ M \left[ \sum_{a \in \mathcal{A} } \left( \frac{1}{M} \sum_{t =1}^M \mathbf{1} \{ X_t = a \} \right) \log_2 \mu(Y_t = a) \right]} \\
&= 2^{ M \left[m_s(a) \log_2 \mu(a) \right] }
\end{align}
The coefficient of $M$ in the exponent directly above can be written in terms of the relative entropy._____no_output________

### __Definition.__ ( relative entropy, or KL divergence )

$\quad$ Let $m$ and $\mu$ be probability measures on some finite state space $\mathcal{S}$.

________no_output______next in section_

* relative entropy or KL divergence_____no_output_____## Gibbs measures

_( Menon 1.4 )______no_output_____$\quad$ Let us abstract away from the above settings, briefly. In particular, we forget everything about the state space $\mathcal{S}$, except that it is a finite set, let us say $\# \mathcal{S} = N$, and we enumerate the states of $\mathcal{S}$ as
$$
s_1, \dots, s_N .
$$

> In the context of the Ising model simulation, $N = 2^{\# \mathbb{T}} \equiv 2^{100^2}$._____no_output_____$\quad$ We suppose there is a distinguished function, $h : \mathcal{S} \to \mathbb{R}$, called the __Hamiltonian__. The Hamiltonian assigns to each state $s_i$ a corresponding __energy__ $h(s_i)$. A Gibbs measure is a probability measure on $\mathcal{S}$, defined from $h$, designed to favor the sampling of low-energy states. In practice, one cares about how the collective behavior of the system changes as a function of the amount of thermal noise. It is traditional to parametrize the amount of thermal noise by an __inverse temperature__ parameter $\beta \in [0, \infty ]$, which is the multiplicative inverse of the system's temperature. Thus $\beta = 0$ corresponds to an infinite temperature limit, and taking $\beta \to \infty$ corresponds to approaching absolute zero.
_____no_output_____$\quad$ To equip $\mathcal{S}$ with a $\beta$-dependent probability measure $m_\beta$, it suffices to specify a collection of functions of $\beta$
$$
p_1(\beta), \dots, p_N(\beta)
$$
corresponding to the probability mass function of $m_\beta$. These depend implicitly on $h$ and are defined as follows._____no_output________

### __Definition.__ ( Gibbs measure )

$\quad$ The __Gibbs measure__ on $\mathcal{S}$ associated to Hamiltonian $h$ and inverse temperature $\beta > 0$, is defined by

$$
p_i( \beta) \propto \exp\left( - \beta h( s_i) \right)
$$

________no_output_____$\quad$ Above, $\propto$ is denotes 'proportional to'. The implicit proportionality constant is non-ambiguous because of the constraint that these numbers sum to one. Being more explicit leads to an object of central importance called the partition function. The __partition function__ is:
$$
Z(h,\beta) : = \sum_{ i = 1 }^N \exp ( - \beta h(s_i) )
$$
_____no_output_____$\quad$ We can thus write the $p_i(\beta)$ in the above definition explicitly as
$$
p_i(\beta) = \frac{ \exp( - \beta h(s_i) ) }{ Z(h,\beta)}
$$
It is useful to consider the endpoint cases $\beta = 0, \infty$. Without much loss of generality, we may assume the $s_i$ are ordered so that

$$
h(s_1) < h(s_2) < \dots < h(s_M) \,,
$$
and one can check that as $\beta \to 0$, namely in the "infinite-temperature" limit, the measure $m_\beta$ becomes uniform over $\mathcal{S}$. The intuition is that thermal noise completely washes away any bias coming from the Hamiltonian.
_____no_output_____$\quad$ On the other hand, in the zero-temperature limit $\beta \to \infty$, The measures $m_\beta$ tend towards concentrating all of their mass on the first state, which has lowest energy. In the limit, we do not even get a random object, provided that the above strict inequalities hold. A state with lowest energy is called a __ground state__. In the above setting, there is a unique ground state. In general, it's possible for the set of ground states to be larger than one, this can arise naturally from symmetries in the model. For instance, the Ising model always has two ground states, corresponding to all $(+1)$ spins and all $(-1)$ spins._____no_output________

_next sections_

* derivation of the Gibbs distribution (continuing Menon 1.4)

* decoding scrambled text (Menon 1.5)

* sampling from a Gibbs distribution (Menon 1.6)

* the Metropolis algorithm (Menon 1.7, Krauth 1.1.4, 1.1.5)

* Markov chains convergence to stationary distribution (Menon 1.8, 1.9)_____no_output_____> The point of building up to this last section is that it lays the theoretical groundwork for how the MCMC method can be used to sample from a Gibbs distribution, abstracting the simulations run in the motivating [blog post](http://bit-player.org/2021/three-months-in-monte-carlo). What's next is to describe the Hamiltonian for the Ising model, from which the Gibbs measures and dynamics are automatically defined. _____no_output_____# simulation_____no_output_____## Ising model_____no_output_____$\quad$ We are simulating the Ising model on the discrete, two-dimensional torus, of side-length 100:

$$
\mathbb{T} := \left( \, \mathbb{Z} \, / \, (100 \cdot  \mathbb{Z}) \, \right)^2
$$

The torus is displayed as a large square grid, with the understanding of "periodic" boundary conditions. We use $\mathbb{T}$ interchangeably to denote the graph and its vertex set. We write $\textrm{E}(\mathbb{T})$ for the edge set of the graph. We write $v \sim w$ if vertices $v$ and $w$ are joined by an edge in $\textrm{E}(\mathbb{T})$. _____no_output_____$\quad$ In the Ising model, where the __spin__ at a given vertex $v \in \mathbb{T}$ is a random variable, $s(v)$, taking values in $\{ \pm 1 \}$. A __state__ of the Ising model is a possible realization of every $s(v)$, for $v \in \mathbb{T}$. The __state space__ of the Ising model is

$$
\begin{align}
\mathcal{S}_{\textrm{Ising}} &= \{ \pm 1 \}^{ \mathbb{T} }\\
&\equiv ( s(v) \in \{ \pm 1\} : v \in \mathbb{T} ).
\end{align}
$$

We do not define the random variables $s(v)$ individually -- instead, we equip the whole state space $\mathcal{S}$ with a family of Gibbs probability measures $( m_\beta : \beta \in [0, \infty] )$. These are completely determined by the Ising Hamiltonian._____no_output________

### __Definition.__ ( Ising Hamiltonian )

The __Ising Hamiltonian__ (in its simplest form) is defined as

$$
H_{\textrm{Ising}}(s) = - \sum_{ v \sim w } s(v) s(w)
$$

for $s \equiv ( s(v) : v \in \mathbb{T} ) \in \mathcal{S}_{\textrm{Ising}}$.

________no_output_____$\quad$ The coefficient of $-1$ in each summand is sometimes written $-J$, for some _coupling constant_ $J > 0$. Here we are setting $J = 1$. The sum is indexed by the edges $e = \{v,w\} \in \textrm{E}(\mathbb{T})$ of $\mathbb{T}$. The negative sign is what makes this a model of _ferromagnetism_, in which neighboring spins are encouraged to align. To see why, note that for the associated Gibbs measure $m_{\,\textrm{Ising},\beta}$, and for $s \in \mathcal{S}_{\textrm{Ising}}$,

$$
m_{\,\textrm{Ising},\beta}(s) \propto \exp \left( \beta \sum_{v \sim w } s(v) s(w) \right),
$$

where the negative sign in the Hamiltonian has cancelled with the negative sign in the definition of Gibbs measure. The double negative allows us to interpret the Hamiltonian as energy: it is physical for low-energy states to be preferred, a kind of nod to the principle of least action, and this is implemented by the exponential biasing the measure towards states whose energy is smallest (perhaps negative).  _____no_output_____$\quad$ Regardless of $\beta$, the exponent in the above display is maximized at two configurations: the first in which $s(v) \equiv 1$ for each $v \in \mathbb{T}$, and the second in which $s(v) \equiv -1$ for each $v \in \mathbb{T}$. We call these states $s_+$ and $s_-$ respectively. That there are two ground states is due to the invariance of the Hamiltonian under 'spin-flip symmetry', i.e. the transformation taking each spin of a configuration to its negative._____no_output_____$\quad$ Moreover, the system is always biased towards these two ground states when $\beta > 0$, it's just that the degree of this bias is mediated by $\beta$. One can interpret this in a 'signal-to-noise' framework: the ground states are like signals, and $\beta$ controls the signal-to-noise ratio. The measures $m_\beta$ exhibit qualitatively different behavior for different ranges of $\beta$, effectively the concept of a phase transition. The Ising model is one of the simplest models that can exhibit this behavior. In particular, there is a __critical inverse temperature__, $\beta_c$, such that the Ising model exhibits distinct behavior for $\beta > \beta_c$ and $\beta < \beta_c$. This value implicitly depends on the fact that we are working with the Ising Hamiltonian on the specific graph $\mathbb{T}$. The case $\beta = \beta_c$ is special, and for now we only concern ourselves with the two regimes away from the critical point. _(If it's the case that simulation becomes more difficult near the critical point, how close to $\beta_c$ does one start to become seriously impeded by these difficulties?)_

| ..... $\beta$ range ..... | temperature range | name of regime |
| -------- | -------- | --- |
| $[0, \beta_c)$ | high-temperature | subcritical |
| $(\beta_c, \infty]$ | low-temperature | supercritical |_____no_output_____## dynamics_____no_output_____$\quad$ We revisit the description of the dynamics. We always need some initial conditions in the form of spin configuration in $\mathcal{S}_{\textrm{Ising}}$. A spin configuration $s \in \mathcal{S}_{\textrm{Ising}}$ is a function $\mathbb{T} \to \{ \pm 1\}$, and we choose to store this data in the computer as an $n \times n$ matrix. We also call this matrix $s$. The periodicity will be encoded in the Hamiltonian, as well as in the dynamics (the neighboring vertices of each site are defined according to this periodicity). _____no_output_____$\quad$ Let us first suppose that updates to the configuration happen at discrete, integer times
$$
1, 2, \dots, \mathfrak{t},
$$
at each such time a (possibly) new spin configuration is created in a way we describe later. This leads to the sequence of spin configurations
$$
s_0, s_1, \dots, s_{\mathfrak{t}} \in \mathcal{S}_{\textrm{Ising}}.
$$
We avoid using $T$, because it already has the meaning of temperature, $T \equiv \beta^{-1}$._____no_output_____$\quad$ In the case that it is later easier to think of updates as happening in continuous time (according to some collection of $\textrm{Exponential}$ clocks, for instance), we will write

$$
t_1 < t_2 < \dots < \mathfrak{t}
$$

for the sequence of update times. The final time $\mathfrak{t}$ may no longer be integer, but we can still use the same notation for the resulting sequence of spin configurations, in either case. _____no_output_____$\quad$ From an earlier [blog post](http://bit-player.org/2019/glaubers-dynamics) of Brian Hayes: _"There’s no evidence Glauber was thinking of his method as an algorithm suitable for computer implementation. The subject of simulation doesn’t come up in his 1963 paper, where his primary aim is to find analytic expressions for the distribution of up and down spins as a function of time. (He did this only for the one-dimensional model.) Nevertheless, Glauber dynamics offers an elegant approach to programming an interactive version of the Ising model. "_ What Hayes calls 'Glauber' dynamics in the first post is actually 'Metropolis-Hastings' dynamics $-$ this was pointed out in the comments. Hayes wanted to understand the distinction between the two better, and the more recent blog post linked to at the top is the really nice result. _____no_output________
_important simulation ingredients and notation______no_output_____* $\quad$ Write $\# \mathbb{T}$ for the number of sites ('pixels') in the torus $\mathbb{T}$.

* $\quad$ For $v \in \mathbb{T}$, let $\textsf{neigh}(v)$ denote the neighbors of $v$ within $\mathbb{T}$. On a torus, every vertex has four neighbors. In contrast to the blog (which uses $\sigma$ to denote spin configurations, where we use $s$), we use $\sigma$ to denote the random field obtained from $s$ by, at each $v \in \mathbb{T}$, assigning the local aggregation
$$
\sigma(v) := \sum_{ w \, \in \, \textsf{neigh}(v) } s(w)\,,
$$
so that each $\sigma(v) \in \{ -4, -2, 0 , 2, 4\}$. _____no_output_____
* $\quad$ The dynamics rely on comparing the current energy of the lattice with the energy resulting from flipping the spin at $v$. We wish to write down the energy difference concisely. To this end, let $s^{[v]}$ denote the spin configuration obtained from $s$ via
$$
s^{[v]} = \begin{cases}
s(w) & \quad \text{ if } w \neq v\\
-s(w) & \quad \text{ if} w = v
\end{cases}
$$

* $\quad$ Let $\delta_v H_{\textrm{Ising}}$ denote the change in energy in going from $H_{\textrm{Ising}}(s)$ to $H_{\textrm{Ising}}(s^{[v]})$:
$$
\delta_v H_{\textrm{Ising}}(s) = H_{\textrm{Ising}}(s^{[v]}) - H_{\textrm{Ising}}(s)
$$
_____no_output________
_on computing $\delta_vH(s)$______no_output_____$\quad$ Let us see why we have the following equality, which is used in the pseudocode below.
$$
\delta_vH_{\textrm{Ising}}(s) = 2 s(v) \sigma(v)
$$
In taking the above difference, we can expand each of $H_{\textrm{Ising}}(s)$ and $H_{\textrm{Ising}}(s^{[v]})$. The sum in the Ising Hamiltonian is always indexed by the edges of the torus, $e \in \textrm{E}( \mathbb{T})$. We can partition this edge set according to $v$. Let $\textrm{E}_1(v)$ denote all edges having $v$ as an endpoint, and let $\textrm{E}_2(v)$ denote the complementary set in $\textrm{E}(\mathbb{T})$. Then,
$$
\begin{align}
H_{\textrm{Ising}}(s) &=  - \sum_{ \, \{ u, w \} \,  \equiv \, e \, \in \, \textrm{E}(\mathbb{T}) } s(u)s(w) \\
&= - \left( \sum_{ \, \{ u, w \} \,  \equiv \, e \, \in \, \textrm{E}_1(v) } s(u) s(w) \right) - \left( \sum_{ \, \{ u, w \} \,  \equiv \, e \, \in \, \textrm{E}_2(v) } s(u) s(w) \right) \\
&= - s(v) \sigma(v) - \left( \sum_{ \, \{ u, w \} \,  \equiv \, e \, \in \, \textrm{E}_2(v) } s(u) s(w) \right)
\end{align}
$$
and when we compute the analogous quantity for $H_{\textrm{Ising}}(s^{[v]})$, the second term remains the same, while the first term has its sign flipped. Subtracting the "flipped Hamiltonian" from the original, the second terms cancel. _____no_output________

### __Algorithm.__ ( Metropolis-Hastings )

Repeat $\# \mathbb{T}$ times:

1. Choose a site $v \in \mathbb{T}$ uniformly at random.

2. Compute $\sigma(v)$ by summing the spins of all vertices neighboring $v$.

3. Compute $\delta_v H_{\textrm{Ising}}(s) = 2 s(v) \sigma(v)$,

4. Spin flip protocol:

    a. If $\delta_v H_{\textrm{Ising}}(s) < 0$, set $s(v) = -s(v)$. This ensures that we visit the configuration $s^{[v]}$ whenever flipping the spin at $v$ lowers the energy of the spin configuration

    b. If $\delta_v H_{\textrm{Ising}}(s) \geq 0$, flip the spin at $v$ with probability
    $$
    \exp ( - \beta \delta_v H_{\textrm{Ising}}(s)  )
    $$

________no_output________

### __Algorithm.__ ( Glauber )

Repeat $\#\mathbb{T}$ times:

1. Choose a site $v \in \mathbb{T}$ uniformly at random.

2. Compute $\sigma(v)$

3. Compute $\delta_v H_{\textrm{Ising}}(s) = 2s(v) \sigma(v)$

4. Flip the spin at $v$ with probability
$$
\frac{ \exp(\, - \beta \delta_v H_{\textrm{Ising}}(s) \,) }{1 + \exp( \, -\beta \delta_vH_{\textrm{Ising}}(s)\,)}
$$

___
_____no_output_____## implementation_____no_output______some resources_

* https://numpy.org/doc/stable/reference/random/generated/numpy.random.uniform.html

* https://numpy.org/doc/stable/reference/random/index.html#random-quick-start_____no_output______imports______no_output_____

#collapse-hide
import numpy as np
import sys
from numpy.random import default_rng

rng = default_rng()_____no_output_____

___

### coin_toss_____no_output_____

#collapse-hide

def coin_toss(p = .5):

    if Unif() <= p:

        return True

    else:

        return False_____no_output_____

________no_output________

### grid_____no_output_____

#collapse-hide

class grid():

    ##
    #   initialize
    #
    def __init__(self,
                 temperature,
                 n = 100,
                 starting_spin_config = 'inf_temp'):

        self.n = n
        #   ^ side-length of torus

        self.d = 2
        #   ^ a two-dimensional torus, by default

        self.N = self.n ** self.d

        self.shape = (n, n)

        self.spins = np.zeros(shape = self.shape,
                              dtype = int)

        ##
        # initial conditions
        #
        # starting_spin_config =
        #     * 'inf_temp'
        #     * 'plus'
        #     * 'minus'
        self.starting = starting_spin_config

        ##
        #   'inf_temp'
        if self.starting == 'inf_temp':

            for x in range(n):

                for y in range(n):

                    coin = coin_toss()

                    if coin:

                        self.spins[x][y] = 1

                    else:

                        self.spins[x][y] = -1

        ##
        #    'plus'
        elif self.starting == 'plus':

            for x in range(n):

                for y in range(n):

                    self.spins[x][y] = 1

        ##
        #    'minus'
        elif self.starting == 'minus':

            for x in range(n):

                for y in range(n):

                    self.spins[x][y] = -1

        """
        later we implement some kind of loading
        from the end of a previous simulation
        """

        # regardless of the initial conditions,
        # we need a starting temperature, because
        # it governs each step of dynamics
        self.temperature = temperature

        self.beta = 1 / self.temperature

        # we need this to implement code related to
        # github author's "note on boltzmann weight"
        self.max_size = sys.maxsize

        """
        on storing dynamics history.
        """

        # we will track how many microsteps
        # the dynamics take. Number of microsteps
        # initialized at 0.
        self.current_time_step = 0

        self.history = {}

        # this records initial conditions in history

        first_key = str( self.current_time_step )

        first_spin_list = self.spins.tolist()

        self.history[first_key] = first_spin_list

    ##
    #   given vertex (x,y) return a list of its
    #   four neighbors
    #
    def neighbors(self, x, y):

        # coordinates
        y_north = (y + 1) % self.n

        y_south = (y - 1) % self.n

        x_east = (x + 1) % self.n

        x_west = (x - 1) % self.n

        # tuples
        north = ( x, y_north )

        south = ( x, y_south )

        east = ( x_east, y )

        west = ( x_west, y )

        return [ north, south, east, west ]

    ##
    #   using the neighbors method, we compute
    #   $\delta_v H(s)$ given v = (x,y)
    #
    def delta_v_H(self, x, y):

        # s(v)
        site_spin = self.spins[x][y]

        # get neighbors
        nbs = self.neighbors(x,y)

        # get their spins
        north_spin = self.spins[ nbs[0][0] ][ nbs[0][1] ]

        south_spin = self.spins[ nbs[1][0] ][ nbs[1][1] ]

        east_spin = self.spins[ nbs[2][0] ][ nbs[2][1] ]

        west_spin = self.spins[ nbs[3][0] ][ nbs[3][1] ]

        # compute sigma(v)
        sigma_v = north_spin + south_spin + east_spin + west_spin

        # return
        return 2 * site_spin * sigma_v

    """

    ( from the first github program, why we
      introduce self.max_size )

    NOTE ON BOLTZMANN WEIGHT: After calculating the weight as
    exp(dE/T), why the Math.min and
    MAX_VALUE? If dE has a large negative value,
    and T is very small, exp(dE/T) can cause
    floating-point overflow. For example, exp(8/0.01)
    is > 10^347. JavaScript barfs on anything greater
    than 10^308.

    """

    ##
    #   this function returns the boltzmann weight
    #   used to determine whether spin is flipped
    #   ( it is used in both types of dynamics )
    #
    def boltzmann_weight( self, x, y ):

        energy_change = self.delta_v_H(x,y)

        exponent = (-1) * self.beta * energy_change

        raw_weight = np.exp( exponent )

        # to_return = np.min( raw_weight, self.max_size )

        return raw_weight # to_return

    ##
    #   the next method handles the uniform selection
    #   of site v from the torus.
    #
    def site_select( self ):

        # first define two random arrays
        # they are Gaussian random variables,
        # but this is going to be incidental.

        # to select a coordinate uniformly at random,
        # we pick the one with the largest value.

        array_1 = rng.standard_normal( self.n )

        array_2 = rng.standard_normal( self.n )

        first_coord = np.argmax( array_1 )

        second_coord = np.argmax( array_2 )

        return first_coord, second_coord

    ##
    #   we perform one step ( a "microstep" )
    #   of the metropolis dynamics. This will
    #   later be repeated N times to form a
    #   "macrostep"
    #
    def metropolis_step(self):

        # new time step
        self.current_time_step += 1

        ##
        # step 1: select site 𝑣 ∈ 𝕋 uniformly
        x, y = self.site_select()

        ##
        # step 2: compute 𝜎(𝑣)
        # step 3: compute 𝛿_𝑣 𝐻(𝑠) = 2 𝑠(𝑣) 𝜎(𝑣)
        #
        # ( both steps built into the function called )
        delta_H = self.delta_v_H(x,y)

        ##
        # step 4: spin flip protocol
        #
        #
        #    a.   If 𝛿_𝑣 𝐻(𝑠) < 0, set 𝑠(𝑣) = −𝑠(𝑣)
        if delta_H < 0:

            self.spins[x][y] = (-1) * self.spins[x][y]
        #
        #
        #    b.   If 𝛿_𝑣 𝐻(𝑠) ≥ 0,
        #   flip the spin at 𝑣 with probability
        #      p := exp( − 𝛽 𝛿_𝑣 𝐻(𝑠) )
        else:

            p = self.boltzmann_weight(x,y)

            boo = coin_toss( p )

            if boo:

                self.spins[x][y] = (-1) * self.spins[x][y]

            else:

                pass

        self.write_state()

    ##
    # Now we do the analogous thing for Glauber,
    # the following is one "microstep" of Glauber
    # dynamics
    #
    def glauber_step(self):

        # new time step
        self.current_time_step += 1

        ##
        # step 1: select site 𝑣 ∈ 𝕋 uniformly
        x, y = self.site_select()

        ##
        # step 2: compute 𝜎(𝑣)
        # step 3: compute 𝛿_𝑣 𝐻(𝑠) = 2 𝑠(𝑣) 𝜎(𝑣)
        delta_H = self.delta_v_H(x,y)

        ##
        # step 4: spin flip protocol:
        #
        #     flip the spin at 𝑣 with probability
        #
        #       exp( −𝛽 𝛿_𝑣 𝐻(𝑠) ) /
        #                         / ( 1 + exp( −𝛽 𝛿_𝑣 𝐻(𝑠) ) )
        #
        weight = self.boltzmann_weight(x,y)

        p = weight / ( 1 + weight )

        boo = coin_toss( p )

        if boo:

            self.spins[x][y] = (-1) * self.spins[x][y]

        else:

            pass

        self.write_state()

    """
    before introducing methods that run a macrostep
    of each type of dynamics, we first provide a method
    that stores the current state of the system.

    In examining how to do this, it makes sense to
    provide `grid` with another attribute:
    `current_time_step` .. these numbers will serve
    as keys for the dictionary that stores the
    history of the dynamics
    """
    def write_state(self):

        dict_key = str( self.current_time_step )

        spin_list = self.spins.tolist()

        self.history[dict_key] = spin_list

    # the above stores spin state to dict
    # we also need a method that writes
    # the dict to a file whose name is provided
    # we store as .json

    def write_history(self, fp):
      """dump the results of the simulation stored in `self.history` as json
      to the output file path specfied by the parameter `fp`
      """
      import json
      with open(fp, 'w') as f:
        json.dump(self.history, f)
        print(f'Wrote results of simulation as json to {fp}.')_____no_output_____

________no_output_____$\quad$ We run the simulation for 10 microsteps of Metropolis dynamics, to test.
_____no_output_____

#collapse-hide
g = grid( temperature = 100 )

for i in range(10):
  g.metropolis_step()

g.write_history( fp = "./test.json" )Wrote results of simulation as json to ./test.json.
#collapse-hide
import json

fil = open( "./test.json", 'r' )

history_dict = json.load( fil )

print( type(history_dict) )

"""
for key, value in history_dict.items():
    print(f"\n\nKey: {key}")
    print(f"\nValue: {value}\n")
"""

fil.close()<class 'dict'>