ICMS

Advances in Markov Chain Monte Carlo

Apr 23, 2012 - Apr 25, 2012

ICMS, 15 South College Street Edinburgh

Organisers

Name	Institution
Girolami, Mark	University College London
Mira, Antonietta	University of Lugano
Robert, Christian P	University of Paris-Dauphine

The popularisation and entry into mainstream statistical practice of Markov chain Monte Carlo (MCMC) methods and associated simulation algorithms over the last twenty years has been due, largely, to their enabling practical use of Bayesian statistical inference. MCMC methods for statistical inference are routinely being deployed in the basic sciences such as Genetics, Molecular and Cellular Biology, Ecology, Clinical Medicine, High Energy Physics, Cosmology, as well as the engineering, social, political, and computing sciences.

The recent methodological advances in MCMC and theory now presents an opportunity to gather leading experts in the field to present latest developments, update researchers, provide a comprehensive perspective on where the research is heading, and define the next set of advances.  

This workshop also aims to publicise the research area to younger researchers within the UK and Europe.

 

Arrangements

Conference Co-ordinator

Catherine McDonald is the conference co-ordinator for this workshop. email: Catherine.McDonald@icms.org.uk

Information about different aspects of the workshop arrangements can be accessed via the following links:

General information | Information for invited speakers | Information for all other participants

---

INFORMATION FOR INVITED SPEAKERS

Registration and Financial Matters

ALL INVITED SPEAKERS SHOULD HAVE RECEIVED AN INVITATION EMAIL FROM ICMS AND SHOULD REGISTER USING THE UNIQUE LINK CONTAINED IN THAT MESSAGE. 

A registration fee of 50GBP is payable by all invited speakers.

The registration fees can be paid by completing the details on this credit/debit card payment form.   As email is not secure, please print the form and return it by fax using the fax number at the top of the form.  ICMS will take your payment and a receipt will be given to you at Registration.  Alternatively, we can accept sterling cheques, made payable to Heriot-Watt University but sent to Catherine McDonald, ICMS, 15 South College Street, Edinburgh, EH8 9AA.  If you have any difficulty in paying in advance, please email Catherine McDonald.

Accommodation for speakers

ICMS has reserved rooms for all invited speakers at Masson House, Pollock Halls, Holyrood Park Road, Edinburgh. These are double rooms with en suite bathrooms. All accommodation is no more than a twenty minute walk from ICMS. Please indicate on your registration form if you wish to take up the offer of these rooms or make your own arrangements.

Lecture theatre facilities

The lecture theatre is fitted with data projection, visualiser (the new generation of over-head), smart-board and blackboard.  More information about the facilities are available from Catherine McDonald.

General information | Information for invited speakers | Information for all other participants

---

 

INFORMATION FOR ALL OTHER PARTICIPANTS

Registration and Financial Matters

If you would like to attend the workshop you apply using this form. Please make sure you have submitted this registration form by Thursday 5 April 2012. There is no automatic acknowledgement of your application.If your application is successful your name will appear on the workshop web page. 

The registration fee for other participants is 100 GBP. The registration fee covers the cost of lunch each day, coffee and tea and a workshop dinner on Tuesday evening. 

The registration fees can be paid by completing the details on this credit/debit card payment form.   As email is not secure, please print the form and return it by fax using the fax number at the top of the form.  ICMS will take your payment and a receipt will be given to you at Registration.  Alternatively, we can accept sterling cheques, made payable to Heriot-Watt University but sent to Catherine McDonald, ICMS, 15 South College Street, Edinburgh, EH8 9AA.  If you have any difficulty in paying in advance, please email Catherine McDonald: Catherine.McDonald@icms.org.uk .

Accommodation

ICMS cannot reserve accommodation for you. Suggestions for accommodation close to ICMS can be found here.

General information | Information for invited speakers | Information for all other participants

---

 

GENERAL INFORMATION

Workshop venue
The workshop will take place at ICMS 15 South College Street Edinburgh EH8 9AA. A map of the venue and the location of accommodation booked for invited speakers can be found here.

Visas

ICMS asks attendees to check if they require a visa for entry to the United Kingdom. This government website gives more information. If you do require a visa ICMS will provide you with a letter in support of your application.

Meals and Refreshments

Morning and afternoon refreshments will be provided on each day of the workshop. A buffet lunch will be provided at ICMS in the Chapterhouse on the ground floor of 15 South College Street on Monday, Tuesday and Wednesday. A workshop dinner will be held on Tuesday.

Travel within the UK and Edinburgh

Information about travel to the UK and Edinburgh is available here. Please note that it is your responsibility to have adequate travel insurance to cover medical and other emergencies that may occur on your trip.

A taxi directly from the airport will cost approximately 18.00 to 20.00 GBP to the city centre for a one-way journey. From the airport, you can take the airport bus Airlink to the city centre. It will cost you 3.50 GBP for a single ticket, or 6.00 GBP return. More details on the airport bus Airlink is available here. The bus will take you to Waverley Bridge (next to Waverley Railway Station). If travelling by train, please note that Edinburgh has two railway stations - Waverley Railway Station being the main station and closest to the workshop venue at ICMS and George Square. If you alight at Edinburgh Waverley, the workshop venue is an easy 10 minute walk over North and South Bridge map. The second railway station is called Haymarket and is at the West End of the city centre.

Lothian buses, Edinburgh's local bus service, charge 1.40 GBP for a single journey and 3.50 GBP for a day ticket. Please note that the exact fare is required and no change is given.

General information | Information for invited speakers | Information for all other participants

---

 

Programme

Provisonal Programme

Monday 23 April

08.45 - 09.40	Registration and coffee in the Chapterhouse, Level 1(ground floor), 15 South College Street
09.40 - 09.50	Introduction and welcome
	Theory Session (Chair Christian Robert)
09.50 - 10.40	Gareth Roberts (University of Warwick) Why does the Gibbs sampler work on hierarchical models?
10.40 - 10.50	Floor Discussion
10.50 - 11.20	Coffee and Tea in the Chapterhouse, Level 1(ground floor)
11.20 - 11.55	Christophe Andrieu (University of Bristol) Stability and stabilisation of controlled Markov chains and their applications in statistics
11.55 - 12.30	Jim Hobert (University of Florida) Convergence rate results for two Gibbs samplers
12.30 - 13.00	Yves Atchade (University of Michigan) Computing Bayes factors with confidence
13.00 - 14.00	Buffett Lunch in the Chapterhouse, Level 1 (ground floor)
	Methodology Session (Chair Xiao-Li Meng)
14.00 - 14.35	Nicolas Chopin (ENSAE-CREST) My memory is long, my patience is not: how not to use MCMC for Bayesian spectral density estimation for long-memory processes
14.35 - 15.05	Nial Friel (Univeristy College Dublin) Estimating the evidence for doubly intractable distributions
15.05 - 15.30	Omiros Papaspiliopoulos (Universitat Pompeu Fabra) Path augmentation
15.30 - 16.00	Coffee and Tea in the Chapterhouse, Level 1(ground floor)
	Applications Session
16.00 - 16.35	Darren Wilkinson (Newcastle University) Bayesian inference for Markov processes with application to biochemical network dynamics
16.35 - 17.10	Andrew Golightly (Newcastle University) Exact inference for stochastic kinetic models via a linear noise approximation
17.10 - 17.45	Maria de Iorio (University College London) A Bayesian model of NMR spectra for the deconvolution andquantification of metabolites in complex biological mixtures
17.45 - 18.20	Dan Lawson (University of Bristol) The Dirichlet process in genetics
18.30 - 20.00	Poster Session and Wine Reception in The Atrium, Level 3

 

 

Tuesday 24 April

	Methodology Session (Chair Mark Girolami)
09.15 - 10.05	Xiao Li Meng (Harvard University) Thank God that regressing Y or X is not the same as regressing X on Y: interweaving residual augmentations
10.05 - 10.15	Floor Discussion
10.15 - 10.45	Coffee and Tea in the Chapterhouse, Level 1(ground floor)
10.45 - 11.20	Daniele Imparato (Università dell'Insubria) Density estimators through zero variance Markov Chain Monte Carlo
11.20 - 11.55	Faming Liang (Texas A&M University) Bayesian subset modeling for high dimensional generalized linear models and its asymptotic properties
11.55 - 12.30	Arnaud Doucet (University of Oxford) Derivative Free Estimates of the Score Vector and Observed Information Matrix
12.30 - 13.05	Jim Griffin (University of Kent) Adaptive Monte Carlo methods for variable selection
13.05 - 14.00	Lunch in the Chapterhouse, Level 1(ground floor)
	Theory Session (Chair Heikki Haario)
14.00 - 14.35	Natesh Pillai (Harvard University) Recent advances in high dimensional covariance matrix estimation
14.35 - 15.10	Alex Beskos (Univeristy College London) Hamiltonian dynamics in high dimensions
15.10 - 15.45	Simon Byrne (Univeristy College London) & Frank Critchley (The Open University) Orthogonal foliations for graphical models
15.45 - 16.15	Coffee and Tea in the Chapterhouse, Level 1(ground floor)
	Applications Session
16.15 - 16.50	David van Dyke (Imperial College London) Metropolis Hastings within partially collapsed Gibbs samplers, with application in high-energy astrophysics
16.50 - 17.25	Iain Murray (University of Edinburgh) Sampling hierarchical latent Gaussian models
17.25 - 18.00	Simo Sarkka (Aalto University) Posterior inference on parameters of stochastic differential equations via Gaussian process approximations
19.30 - 22.00	Workshop Dinner (The Magnum Restaurant, 1 Albany Street, Edinburgh EH1 3PY)

 

 

Wednesday 25 April

	Application Session (Chair Antonietta Mira)
09.15 - 10.05	Heiki Haario (Lappeenranta University of Technology) State and parameter estimation for large scale models
10.05 - 10.15	Floor Discussion
10.15 - 10.50	Marko Laine (Finnish Meteorological Institute) Efficient adaptive MCMC for complex models
10.50 - 11.20	Coffee and Tea in the Chapterhouse, Level 1(ground floor)
	Theory Session
11.20 - 11.55	Gersende Fort (LTCI, CNRS - TELECOM ParisTech) Stochastic approximation for adaptive interacting MCMC samplers
11.55 - 12.30	Andrew Stuart (Univeristy of Warwick) Random walk metropolis algorithms in high dimensions
12.30 - 13.05	Krysztof Latuszynski (University of Warwick) Robustness of manifold MALA and related algorithms
13.05 - 13.35	Nick Whiteley (University of Bristol) Stability properties of particle filters
13.35 - 14.35	Lunch in the Chapterhouse, Level 1(ground floor)
	Methodology Session (Chair Gareth Roberts)
14.35 - 15.10	Jean-Michel Marin (Université Montpellier 2) Bayesian inference on a mixture model with spatial dependence
15.10 - 15.45	Gareth Tribello (ETH Zurich and Universita della Svizzera Italiana) Methods for surveying complex probability distributions
15.45 - 16.00	Closing Remarks
16.00	Workshop closes

 

 

Presentations:

Presentation Details
Andrieu, Christophe
Stability and stabilisation of controlled Markov chains and their applications in statistics
View Abstract Hide Abstract
The initial motivation for the work presented was to understand the behaviour of popular algorithms used for statistical inference, such as Monte Carlo approximations of the EM (expectation maximisation) algorithm or adaptive Markov chain Monte Carlo methods which fall in the category of controlled Markov chain processes. Informally, given a family of Markov transition probabilities indexed by a parameter, such processes correspond, at a given time instant, to updating the state of the process with one of those transition probabilities whose indexing parameter is chosen according to a rule which may depend on the whole past of the process. Such processes can be particularly unstable when the indexing parameter approaches a set of critical values. The aim of the presentation is to provide some insights into the observed stability of some controlled Markov chains of interest and present some stabilisation techniques which are designed to ensure the stability of the process. As we shall see some of the aforementioned algorithms used in statistics possess natural "strong" stability properties, while others may require the use of stabilisation techniques.
Atchade, Yves
Computing Bayes factors with confidence
View Abstract Hide Abstract
The talk will present a flexible implementation of the path sampling identity using non-homogeneous Markov chains. This allows for rigorous confidence interval estimation for ratio of normalizing constants.
Beskos, Alexandros
Hamiltonian dynamics in high dimensions
View Abstract Hide Abstract
We look at the behavior of Hybrid Monte-Carlo methods in high dimensions, and illustrate results designating an asymptotically optimal acceptance probability of 0.651. We also present an "advanced" version of Hybrid Monte-Carlo on Hilbert spaces, for target distributions defined as change of measures from Gaussian laws arising in applications. Analytical and experimental results will illustrate the superiority of advanced Hybrid Monte-Carlo over alternative "standard" MCMC methods.
Byrne, Simon
Orthogonal foliations for graphical models (Joint presentation with Frank Critchley, Open University)
View Abstract Hide Abstract
Bayesian hierarchical models are one of the most common applications of MCMC, but can be still quite difficult to implement effectively. Conditional independence means that simple methods such as Gibbs sampling often work quite well for the lower-level parameters, however the high correlation between levels means that it can be difficult to construct efficient proposal schemes for the higher-level parameters. The geometry of such models suggests that these updates should be done in a manner that is geometrically orthogonal to the lower-level updates. This defines a "foliation": a partition of the model into locally parallel submanifolds. In order to efficiently sample from this submanifold, we can adapt the Riemannian manifold Hamiltonian Monte Carlo (RMHMC) algorithm to utilise constrained Hamiltonian dynamics.
Chopin, Nicolas
My memory is long, my patience is not: how not to use MCMC for Bayesian spectral density estimation for long-memory processes
View Abstract Hide Abstract
Long-memory or short-memory Gaussian time-series models are often specified through their spectral density. Such models pose several computational challenges, such as the non-sparseness of the covariance matrix, which makes the likelihood expensive to compute, and the absence of any structure in the likelihood, which prevents a Gibbs sampling implementation. The first part of this work develops a fast approximation of the likelihood, which can be computed in O(n) time. It is shown that the variance of the weights of an importance sampling step, from the approximate to the true likelihood, goes to zero as the sample size goes to infinity. The second part of this work uses a tempering version of Sequential Monte Carlo to sample from the approximate posterior (i.e. the prior times the approximate likelihood). We show that this SMC sampler is faster, and, more importantly, more robust to multimodality than plain MCMC.
De Iorio, Maria
A Bayesian model of NMR spectra for the deconvolution andquantification of metabolites in complex biological mixtures
View Abstract Hide Abstract
Nuclear Magnetic Resonance (NMR) spectra are widely used in metabolomics to obtain profiles of metabolites dissolved in biofluids such as cell supernatants. Methods for estimating metabolite concentrations from these spectra are presently confined to manual peak fitting and to binning procedures for integrating resonance peaks. Extensive information on the patterns of spectral resonance generated by human metabolites is now available in online databases. By incorporating this information into a Bayesian model we can deconvolve resonance peaks from a spectrum and obtain explicit concentration estimates for the corresponding metabolites. Spectral resonances that cannot be deconvolved in this way may also be of scientific interest so we model them using wavelets. We describe a Markov chain Monte Carlo algorithm which allows us to sample from the joint posterior distribution of the model parameters, using specifically designed block updates to improve mixing. We assess our method for peak alignment and concentration estimation and find that alignment is unbiased and precise. Moreover, our concentration estimates significantly reduce mean estimation error when compared to conventional numerical integration methods. Finally, we apply our method to a spectral dataset taken from an investigation of the metabolic response of yeast to recombinant protein expression.
Fort, Gersende
Stochastic approximation for adaptive interacting MCMC samplers
View Abstract Hide Abstract
Interacting MCMC samplers are based on the construction of K parallel processes targeting different distributions. For example, the K limiting distributions can be chosen as tempered version of the target density. These parallel processes are defined so that interaction with the whole past of a neighboring process is allowed thus improving the convergence properties of the sampler. In this context, the parallel processes are no more a Markov chain. Among Interacting MCMC samplers, the Equi-Energy sampler (Kou et al 2006) is a powerful algorithm for sampling multimodal distributions. In this algorithm, interactions only involve points with the same energy level. This algorithm depends on many design parameters such as the number of parallel processes K and number of rings, the probability of interaction, the choice of the energy rings. In this talk, I will first propose and Adaptive Equi-Energy sampler, in which the energy rings are adapted on the fly based on the history of the parallel processes. When deriving convergence results for this adaptive Equi-Energy sampler, we are faced to challenging problems about stability and convergence of stochastic approximation schemes with discontinuous dynamics, when the mean field is approximated by Interacting MCMC samples. I will discuss recent results on these problems, and will show how they are used to prove ergodicity and law of large numbers for the Adaptive Equi-Energy sampler. This is a joint work with A. Garivier, E. Moulines, A. Schreck - from LTCI, France; and M. Vihola - from Jyvaskyla, Finland
Friel, Nial
Estimating the evidence for doubly intractable distributions
View Abstract Hide Abstract
There has been considerable interest in the literature in parameter estimation for doubly intractable distributions. Posterior distributions arising from Markov random fields, such as those found in spatial statistics and social network analysis, are examples of this intractability. This talk will illustrate an approach, extending this methodology, to allow one estimate the evidence for this class of statistical models, thereby facilitating model choice.
Golightly, Andrew
Exact inference for stochastic kinetic models via a linear noise approximation
View Abstract Hide Abstract
Recently proposed particle MCMC methods provide a flexible way of performing Bayesian inference for parameters governing stochastic kinetic models defined as Markov (jump) processes (MJPs). Each iteration of the scheme requires an estimate of marginal likelihood calculated from the output of a sequential Monte Carlo scheme (also known as a particle filter). Consequently, the method can be extremely computationally intensive. We therefore aim to negate the expensive likelihood calculation through use of a fast approximation, or `emulator'. We use the tractable marginal likelihood under a linear noise approximation of the MJP to calculate a first step acceptance probability. If a proposal is accepted under the approximation, we then run a sequential Monte Carlo scheme to compute an estimate of the marginal likelihood under the MJP and construct a second stage acceptance probability that permits exact (simulation based) inference for the MJP. We therefore avoid expensive calculations for proposals that are likely to be rejected. We illustrate the method by considering inference for parameters governing a Lotka-Volterra system.
Griffin, Jim
Adaptive Monte Carlo methods for variable selection
View Abstract Hide Abstract
The Bayesian approach to variable selection in regression models is well-established with a prior placed on the space of all possible regression models and their parameters. The posterior distribution is often hard to simulate using Metroplis-Hastings methods since it can have multiple, well-separated modes leading to low acceptance rates for between model moves. If our data include many potential regressors but few observations, the posterior distribution often becomes flatter and the acceptance rate for between model moves can become large. The high acceptance rates suggests that more efficient algorithms can be obtained by proposing ``bigger'' between model moves. This talk will discuss how such moves can be constructed and how the ideas of adaptive Monte Carlo can be used to tune them to give efficient methods. The methods will be illustrated on examples in both linear and binary regression problems.
Haario, Heikki
State and parameter estimation for large scale models
View Abstract Hide Abstract
The Kalman Filter and nonlinear extensions of it provide a natural approach for data assimilation problems. But they are infeasible for large enough models. Here we discuss low-memory approximations based on optimization algorithms. While MCMC can be technically performed even for such large systems, and an additional challenge is due to chaoticity: small parameter changes may lead to large 'random' variations of the likelihood. As a remedy, we propose the use of filtering-based likelihoods. Finally, we discuss an approach where an operational ensemble weather prediction system is slighly modified for parameter estimation purposes, with practically no additional CPU cost.
Hobert, Jim
Convergence rate results for two Gibbs samplers
View Abstract Hide Abstract
After a brief review of Markov chain CLTs, geometric ergodicity and geometric drift conditions, I will describe new convergence rate results for two block Gibbs samplers. The first is a recently developed algorithm for exploring the intractable posterior density associated with a Bayesian quantile regression model. The second is a well-known Gibbs sampler for the Bayesian version of the general linear mixed model with improper priors. (This is joint work with K Khare and J Roman)
Imparato, Daniele
Density estimators through zero variance Markov Chain Monte Carlo
View Abstract Hide Abstract
A Markov Chain Monte Carlo method is proposed for the point-evaluation of a density whose normalization constant is not known. This method was introduced in the physics literature by Assaraf et al. (2007). Conditions for unbiasedness of the estimator and for central limit theorem are derived in this context. The new idea is tested on some toy-examples.
Laine, Marko
Efficient adaptive MCMC for complex models
View Abstract Hide Abstract
In multi scale dynamical process models, such as those used for weather prediction and climate studies, sub-grid scale processes must be replaced by physical parameterization schemes. Tuning and uncertainty analysis of closure parameters related to these schemes is complicated by the high complexity and computational burden of the models. We review some efficient MCMC techniques for performing statistical inference for computationally highly complex models: adaptation, parallel chains and early rejection. The examples describe our analysis of ECHAM5 climate model. We discuss our MCMC experiments, specification of cost function, parameter identification, and some alternative Monte Carlo techniques based on ensemble model runs. References: A. Solonen, P. Ollinaho, M. Laine, H. Haario, J. Tamminen, H. Järvinen: Efficient MCMC for climate model parameter estimation: parallel adaptive chains and early rejection, Bayesian Analysis, accepted, 2012. J. Hakkarainen, A. Ilin, A. Solonen, M. Laine, H.Haario, J. Tamminen, E. Oja, H. Järvinen: On closure parameter estimation in chaotic systems, Nonlinear Processes in Geophysics, accepted, 2012. H. Järvinen, P. Räisänen, M. Laine, J. Tamminen, A. Ilin, E. Oja, A. Solonen, H. Haario: Estimation of ECHAM5 climate model closure parameters with adaptive MCMC, Atmospheric Chemistry and Physics, 10(1), pages 9993--10002, 2010. doi:10.5194/acp-10-9993-2010
Latuszynski, Krzysztof
Robustness of manifold MALA and related algorithms
View Abstract Hide Abstract
The standard MALA algorithms generally have improved convergence properties over the Random Walk Metropolis, however they cannot be applied for target distributions with too heavy or too light tails. Motivated by this imperfection and inspired by the recent work on Manifold Monte Carlo Methods (Girolami and Calderhead 2011), we investigate properties of a more general version of MALA algorithms based on discretising a family of Langevin diffusions with local diffusion coefficient targeting the stationary measure. We analyse it for a benchmark family of targets and extend towards more general distributions. This is joint work with Gareth O. Roberts and Katarzyna Wolny.
Lawson, Dan
The Dirichlet process in genetics
View Abstract Hide Abstract
The Dirichlet Process is an extremely important and popular tool for a wide range of clustering problems. This computationally cheap Bayesian prior allows numerical integration over the assignment of samples to clusters, whilst also determining the number of clusters from the data. A mechanistic model for the data can still be placed within the Dirichlet Process Prior, making this a very flexible modelling framework. In this talk I'll give an introduction to the Dirichlet Process and discuss the MCMC moves needed for efficient mixing. I'll also give an overview of some interesting extensions used in the (genetics) literature. Finally, I'll conclude with my own research which maps an inherently complex problem down to a Dirichlet Process MCMC problem by careful exploitation of the approximate behaviour of the underlying "correct" model. This allows for very efficient and very precise population assignment of individuals using their genetics.
Liang, Faming
Bayesian subset modeling for high dimensional generalized linear models and its asymptotic properties
View Abstract Hide Abstract
We propose a new prior setting for high dimensional generalized linear models (GLMs), which leads to a Bayesian subset regression (BSR) with the maximum a posteriori model coinciding with the minimum extended BIC model. We establish consistency of the posterior under mild conditions and sample from the posterior using the stochastic approximation Monte Carlo algorithm. Further, we propose a variable screening procedure based on the marginal inclusion probability and show that this procedure shares the same properties of sure screening and consistency with the existing sure independence screening (SIS) procedure. However, since the proposed procedure makes use of joint information of all predictors, it generally outperforms SIS and its improved version in real applications. For subject classification, we propose a selected Bayesian classifier based on the goodness-of-fit test for the models sampled from the posterior. The new classifier is consistent and robust to the choice of priors. The numerical results indicate that the new classifier can generally outperform the Bayesian model averaging classifier and the classifier based on the high posterior probability models. We also made extensive comparisons of BSR with the popular penalized likelihood methods, including Lasso, elastic net, SIS and ISIS. The numerical results indicate that BSR can generally outperform the penalized likelihood methods: The models elected by BSR tend to be sparser and, more importantly, of higher generalization ability. In addition, we find that the performance of the penalized likelihood methods tend to deteriorate as the number of predictors increases, while this is not significant for BSR.
Marin, Jean-Michel
Bayesian inference on a mixture model with spatial dependence
View Abstract Hide Abstract
We introduce a new technique in order to select the number of components of a mixture model with spatial dependences. Typically, we consider an image field where the grey-scale values are Gaussian random variables depending on the component of the associated pixel and the components are distributed according to a Potts model. Our approach is deduced from an approximation of the integrated completed likelihood and from the use of specific auxilliary variable MCMC algorithms which avoid the difficulty associated to the fact the normalizing constant of the Potts model is not available.
Meng, Xiao-Li
Thank God that regressing Y on X is not the same as regressing X on Y: interweaving residual augmentations
View Abstract Hide Abstract
What does regressing Y on X versus regressing X on Y have anything to do with MCMC? It turns out that many strategies for speeding up data augmentation type of algorithms intrinsically share the same principle as regression in the sense of fostering independence between a regression function and the corresponding residual, thereby reducing or even eliminating dependence among MCMC iterates. There are two general classes of algorithms, those corresponding to regressing parameters on augmented data/auxiliary variables and those the other way around. The interweaving strategy (Yu and Meng, 2011, JCGS) amounts to taking advantage of both, and it is the existence of two different types of residuals that makes the interweaving strategy seemingly magical in some cases and promising in general. This talk reports on the latest findings in the search for effective residual augmentations, as well as an intriguing phase transition type phenomenon that might provide general insight into constructing interweaving algorithms that meet the 3-S criterion: simple, speedy, and stable. (This is joint work with Xiaojin Xu and Yaming Yu.)
Murray, Iain
Sampling hierarchical latent Gaussian models
View Abstract Hide Abstract
Latent Gaussian models are a standard workhorse for statistical modelling. Applications are found in diverse areas such as astro-physics, and modelling sports outcomes. They are also a useful test-bed for probabilistic inference methods, as several difficulties manifest themselves that are common to many inference problems. I'll outline some of my work on Markov chain Monte Carlo methods for simulating the posteriors of hierarchical latent Gaussian models.
Papaspiliopoulos, Omiros
Path augmentation
View Abstract Hide Abstract
In this talk I will present a novel framework for data augmentation which considerably extends the ideas of non-centering and it is particularly attractive for inference for hierarchical models with latent stochastic processes. The framework, together with additional results which link continuous and discrete time approaches to inference for diffusions, provides a unification and theoretical justification of popular algorithms for inference for diffusions.
Pillai, Natesh
Recent advances in high dimensional covariance matrix estimation
View Abstract Hide Abstract
We will present a brief overview of our recent work on high dimensional covariance estimation. Firstly,we will present a flexible class of models based on scale mixture of uniform distributions to construct shrinkage priors for covariance matrix estimation. This new class of priors enjoys a number of advantages over the traditional scale mixture of normal priors, including its simplicity and flexibility in characterizing the prior density. We also exhibit a simple, easy to implement Gibbs sampler for posterior simulation which leads to efficient estimation in high dimensional problems. Next we will present some recent theoretical results on factor models.
Roberts, Gareth
Why does the Gibbs sampler work on hierarchical models?
View Abstract Hide Abstract
Contrary to common perception, the Gibbs sampler does not intrinsically break down the curse of dimensionality. However, as we know, it (and its generalisations) have been highly successful in Bayesian inference with high-dimensional parameter spaces. So what is special about "statistical" high-dimensional distributions that has facilitated this? This presentation will explore the links between hierarchical models and stability properties of the Gibbs sampler such as geometric ergodicity. Some general results will be given which have simple and practical implications for MCMC algorithm design. This is joint work with Omiros Papaspiliopoulos, Kryszof Latuszynski, and Natesh Pillai.
Särkkä, Simo
Posterior inference on parameters of stochastic differential equations via Gaussian process approximations
View Abstract Hide Abstract
This talk is concerned with Bayesian estimation of parameters in stochastic differential equation (SDE) models occurring in physics, engineering and financial applications. In particular, we consider implementation of MCMC based sampling methods for partially observed irreducible Brownian-motion-driven non-linear multivariate SDEs. For these kind of SDEs computation of the transition densities is not possible in closed form and thus the implementation of MCMC methods is not possible without additional approximations to them. In this talk we show how Taylor series and Gaussian cubature methods can be used for forming Gaussian approximations to the transition densities and how non-linear Kalman filters can be used in efficient implementation of MCMC sampling. In addition to MCMC, we also discuss implementation of Laplace approximations and other types of methods for the posterior parameter inference by using similar approximations.
Stuart, Andrew
Random walk metropolis algorithms in high dimensions
View Abstract Hide Abstract
I will describe recent theoretical developments which explain how to develop and analyze random walk type Metropolis algorithms in high dimensions. The setting is based on studying probability measures which have density with respect to a Gaussian random field, and their finite dimensional approximations. Diffusion limits to stochastic PDEs, and spectral gaps using the Wasserstein-1 distance, are shown to provide effective and robust tools which quantify computational complexity in this scenario. Joint work with Martin Hairer, Jonathan Mattingly, Natesh Pillai, Alex Thiery and Sebastian Vollmer will be described.
Tribello, Gareth
Methods for surveying complex probability distributions
View Abstract Hide Abstract
The probability of adopting a particular molecular configuration is given by the Boltzmann distribution and in physics we often use molecular dynamics (MD) to probe this distribution. However, when we discretize Newton's equations of motion in our MD simulation we are forced to use a very short timestep so we often do not see interesting, long-time-scale chemical processes. For simple chemical processes it is straightforward to resolve this second problem using the well-established metadynamics [1] method. This method generates a history dependent bias that forces the system to move away from previously visited configurations and into unexplored parts of phase space. The problem with this method is that it is reliant on user defined collective variables (CVs). Suggesting such CVs is simple when the underlying physics is straightforward but is considerably more difficult for processes such as protein folding that have complex and less well understood mechanisms. Hence, the work I will present is concerned with finding CVs that can be used to accelerate sampling when the reaction mechanism is complicated and our understanding is limited. The first method I will present is the well tempered ensemble [2], which uses the potential energy as a CV. It is possible to show that biasing the energy in this way enhances the fluctuations in the energy and thus increases sampling. However, problems occur for this method when there are multiple structures with similar energies as there is no bias present to force the system to travel between the different chemical states. Hence, the other methods I will show work by assuming that there are a small number of low-energy configurations and that there are a small number of connections (transition pathways) between these low energy states. The interesting low energy parts of phase space will be highly populated in short MD trajectories so we can recognize energetic basins using a Gaussian Mixture model [3]. The fit obtained from this model can be used to generate a bespoke bias that can in turn be used to force a more rapid sampling of CV space. A separate but related problem to the one of sampling is how to understand the data that we obtain from a thorough sampling of phase space. I will thus also show how we can use dimensionality reduction to understand MD trajectories [4] and how the low-dimensionality coordinates so obtained can be used to enhance the sampling of phase space [5]. [1] A. Laio and M. Parrinello Escaping free energy minima Proc. Natl. Acad. Sci. U.S.A. 99 12562-12566 (2002) [2] M. Bonomi and M. Parrinello Enhanced sampling in the well-tempered ensemble Phys. Rev. Lett 104 190601 (2010) [3] G. A. Tribello, M. Ceriotti and M. Parrinello A self-learning algorithm for biased molecular dynamics Proc. Natl. Acad. Sci. U.S.A 107 17509-17514 (2010) [4] M. Ceriotti, G. A. Tribello and M. Parrinello Simplifying the representation of complex free energy landscapes using sketch-map Proc. Natl. Acad. Sci. U.S.A. 108 13023-13028 (2011) [5] G. A. Tribello, M. Ceriotti and M. Parrinello Using sketch-map coordinates to analyze and bias molecular dynamics simulations Proc. Natl. Acad. Sci. U.S.A. 109 5196-5201 (2012)
van Dyk, David
Metropolis Hastings within partially collapsed Gibbs samplers, with application in high-energy astrophysics
View Abstract Hide Abstract
The Partially Collapsed Gibbs (PCG) sampler offers a new strategy for improving the convergence of a Gibbs sampler. PCG achieves faster convergence by reducing the conditioning in some of the draws of its parent Gibbs sampler. Although this can significantly improve convergence, care must be taken to ensure the stationary distribution is preserved. The conditional distributions sampled in a PCG sampler may be functionally incompatible and permuting them may upset the station- ary distribution. Extra care must be taken when Metropolis steps are used. Reducing the conditioning in an MH within Gibbs sampler can change the stationary distribution, even when the PCG sampler would work perfectly if all the conditional updates were available. We illustrate the challenges that may arise when using an MH within PCG sampler and develop a general strategy for using such updates while maintaining the stationary distribution.
Whiteley, Nick
Stability properties of particle filters
View Abstract Hide Abstract
Particle filtering has become a very popular method for inferential computation in state space models. What arguably makes particle filters useful is that they exhibit some form of stability; it is commonly observed that the stochastic errors associated with the particle approximation do not explode over time, so that these algorithms can be applied successfully at a computational cost which scales slowly with the length of the data record under analysis. Whilst there is now a rich literature on various theoretical properties of particle filters, most existing results which explain their stability properties rely on strong mixing assumptions which do not hold in typical applications. One of the main obstacles is dealing with a non-compact state space. This talk will describe recent developments which allow some stability properties of particle filters to be verified when the state space is non-compact.
Wilkinson, Darren
Bayesian inference for Markov processes with application to biochemical network dynamics
View Abstract Hide Abstract
A number of interesting statistical applications require the estimation of parameters underlying a nonlinear multivariate continuous time Markov process model, using partial and noisy discrete time observations of the system state. Bayesian inference for this problem is difficult due to the fact that the discrete time transition density of the Markov process is typically intractable and computationally intensive to approximate. It turns out to be possible to develop particle MCMC algorithms which are exact, provided that one can simulate exact realisations of the process forwards in time. Such algorithms, often termed "likelihood free" or "plug-and-play" are very attractive, as they allow separation of the problem of model development and simulation implementation from the development of inferential algorithms. Such techniques break down in the case of perfect observation or high-dimensional data, but more efficient algorithms can be developed if one is prepared to deviate from the likelihood free paradigm, at least in the case of diffusion processes. The methods will be illustrated using examples from population dynamics and stochastic biochemical network dynamics.

Participants

Name	Institution
Andrieu, Christophe	University of Bristol
Atchade, Yves	University of Michigan
Beskos, Alexandros	University College London
Byrne, Simon	University College London
Chopin, Nicolas	ENSAE-CREST
Critchley, Frank	The Open University
De Iorio, Maria	University College London
Doucet, Arnaud	University of Oxford
Fort, Gersende	LTCI, CNRS - TELECOM ParisTech
Friel, Nial	University College Dublin
Gamado, Kokouvi Mawuli	BioSS and Heriot-Watt University
Girolami, Mark	University College London
Golightly, Andrew	Newcastle University
Griffin, Jim	University of Kent
Haario, Heikki	Lappeenranta University of Technology
Hamid, Bazargan	Heriot Watt University
Hobert, Jim	University of Florida
Imparato, Daniele	Università dell'Insubria
Laine, Marko	Finnish Meteorological Institute
Latuszynski, Krzysztof	University of Warwick
Lawson, Dan	University of Bristol
Liang, Faming	Texas A & M University
Lord, Gabriel	Heriot-Watt University
Marin, Jean-Michel	University Montpellier 2
Marion, Glenn	Biomathematics & Statistics Scotland,
Mavros, George	Heriot Watt University
Meng, Xiao-Li	Harvard University
Miasojedow, Blazej	LTCI, TELECOM ParisTech
Mira, Antonietta	University of Lugano
Murray, Iain	University of Edinburgh
Papaspiliopoulos, Omiros	Universitat Pompeu Fabra
Peluso, Stefano	Universita della Svizzera Italiana
Pereyra, Marcelo	University of Toulouse
Pillai, Natesh	Harvard University
Pollock, Murray	University of Warwick
Robert, Christian P	University of Paris-Dauphine
Roberts, Gareth	University of Warwick
Särkkä, Simo	Aalto University
Siu Yin, LAU	Heriot-Watt University, Edinburgh
Solgi, Reza	Swiss Finance Institute at University of Lugano
Stuart, Andrew	University of Warwick
Thiery, Alexandre	Warwick University
Tribello, Gareth	ETH Zurich + Università della Svizzera Italiana
van Dyk, David	Imperial College London
Vollmer, Sebastian	University of Warwick
Whiteley, Nick	University of Bristol
Wilkinson, Darren	Newcastle University
Wolny, Kasia	University of Warwick