Publication Details
Edna Milgo, Edna Milgo



Markov chain Monte Carlo (MCMC) methods are used to generate samples from complex probability distributions using a proposal distribution assumed to be Gaussian in our research. Adaptive MCMC learns the covariance of the proposal during sampling. We expound on three different principles of adaptation that can be used to improve the efficiency of MCMC samplers. The first principle is used in Adaptive Metropolis (AM), which is the benchmark adaptive MCMC. It estimates the covariance of the proposal using past samples of the chain. The second is the maximum entropy principle that adapts the covariance such that the entropy of the proposal is maximized given some constraints depending on whether the proposed sample was rejected or accepted. Third is the principle that adapts the proposal distribution such that the likelihood of generating better search points is increased. It uses a predefined target acceptance rate. The last two principles are respectively used in Gaussian Adaptation (GaA) and Covariance Matrix Adaptation Evolution Strategy (CMAES), both of which are stochastic optimization algorithms. GaA and the (1+1)-variant of CMAES are hill climbers that can be transformed in a straightforward way into MCMC samplers herein referred to as MGaA and MCMA respectively. Adapting the proposal using past samples annuls the Markov property of the chain and the guarantee that the chain converges to the target. Therefore, we further sought to find the impact of diminishing adaptation in the effectiveness of the samplers. AM by design has diminishing adaption, while MGaA and MCMA do not. We experiment with adaptation stopped halfway and varying rates of diminishing adaptation to find the best variant. Using five performance measure, we compare AM and the best variants of MGaA and MCMA on a test suite of seven target distributions with dimensions ranging from 2 to 50.