Model selection criteria based on computationally intensive estimators of the expected optimism

Abstract

A model selection criterion based on a divergence or discrepancy measure is generally comprised of a goodness-of-fit term and a penalty term. The penalty term, which reflects model complexity, serves as an estimate of a quantity known as the expected optimism. Classical approaches to approximating the expected optimism often lead to simplistic penalizations. However, such approaches usually involve stringent assumptions that may fail to hold in practical applications. Modern computational statistical methods facilitate the development of improved estimators of the expected optimism. Selection criteria based on such penalty terms often provide more realistic measures of predictive efficacy than their classical counterparts, thereby resulting in superior model determinations. To survey this methodology, we outline the general framework for discrepancy-based model selection criteria, and review computationally intensive approaches for evaluating complexity penalizations.