Fast Model Selection for Interpretable Gaussian Process Models using Laplace Approximation

Model selection aims to find the best model in terms of ac-
curacy, interpretability or simplicity. Here, we focus on evaluating model
performance of Gaussian process models, which can be used for data
analysis of uncertain data. While previous work uses methods like the
Marginal log likelihood, AIC or nested sampling to perform model se-
lection, they either lack performance or have significant runtime issues,
limiting their applicability. We address these challenges by introducing
multiplemetricsbasedontheLaplaceapproximation,whereweovercome
aninconsistencyoccurringduringitsnaiveapplicationwhichcausesover-
fittingintermsofmodelselection.WeshowhowAICcanfailtorecognize
the more appropriate model and perform large scale experiments to con-
clude that our Laplace-based metrics outperform the state of the art for
fast model selection. Our model selection metrics allow fast and high
quality model selection of Gaussian processes.