Loading Events

Luigi Malagò (Shinshu University, Japan)

13 March 2015 @ 11:00

 

  • Past event

Details

Date:
13 March 2015
Time:
11:00
Event Category:

Information geometry of the Gaussian distribution in view of stochastic optimization: first and second order geometry

We study the optimization of a continuous function by its stochastic relaxation, i.e., the optimization of the expected value of the function itself with respect to a density in a statistical model. In the first part of the talk we focus on gradient descent techniques applied to models from the exponential family and in particular on the multivariate Gaussian distribution. From the theory of the exponential family, we reparametrize the Gaussian distribution using natural and expectation parameters, and we derive formulas for natural gradients in both parameterizations. We discuss some advantages of the natural parameterization for the identification of sub-models in the Gaussian distribution based on conditional independence assumptions among variables. In the second part of the talk we introduce second-order geometry for exponential families. Second-order optimization methods have been widely used in optimization on manifolds, e.g., matrix manifolds, but appear to be relatively new in statistical manifolds. We show how to compute the Reimannian hessian in a statistical manifold and we apply the formalism to the discussion of the Newton method in the context of the optimization of the relaxed function. Gaussian distributions are widely used in stochastic optimization and in particular in model-based Evolutionary Computation, such as in Estimation of Distribution Algorithms and Evolutionary Strategies.