Markov Chain Monte Carlo: Simulating the Normal Distribution

Markov Chain Monte Carlo (MCMC) is a strategy for simulating some target distribution when sampling directly from the distribution is difficult. For ergodic (irreducible and aperiodic) Markov Chains, the distribution of realizations from the Markov Chain will converge to the stationary distribution as the number of realizations increases. MCMC tries to construct a Markov Chain with a stationary distribution that is the same as the target distribution. In this toy example, we use the Metropolis Hastings algorithm backed by a random walk to simulate the standard normal distribution.