David Rossell (Universitat Pompeu Fabra, Barcelona, Spain) (webinar)

Abstract: Bayesian model selection requires an integration exercise in order to assign posterior model probabilities to each candidate model. The computation becomes cumbersome when the integral has no closed-form, particularly when the sample size is large, or the number of models is large. We present a simple yet powerful idea based on the Laplace approximation (LA) to an integral. LA uses a quadratic Taylor expansion at the mode of the integrand and is typically quite accurate, but requires cumbersome likelihood evaluations (for large n) an optimization (for large p). We propose the approximate Laplace approximation (ALA), which uses an Taylor expansion at the null parameter value. ALA brings very significant speed-ups by avoiding optimizations altogether, and evaluating likelihoods via sufficient statistics. ALA is an approximate inference method equipped with strong model selection properties in the family of non-linear GLMs, attaining comparable rates to exact computation. When (inevitably) the model is misspecified the ALA rates can actually be faster than for exact computation, depending on the type of misspecification. We show examples in non-linear Gaussian regression with non-local priors, for which no closed-form integral exists, as well as non-linear logistic, Poisson and survival regression.