Optuna Constrained Tree-Structured Parzen Estimator Is a Joint Density Generalization of c-TPE
Shuhei Watanabe, Kaichi Irie
cs.LG
Abstract
Constrained hyperparameter optimization (HPO) is common in practice, yet Optuna's widely used constrained TPE lacks algorithmic analysis. While c-TPE proposes an expected constrained improvement (ECI) approach assuming independence between the objective and constraints, Optuna uses a single joint density over both. We show that Optuna's constrained TPE is joint c-TPE -- the same ECI acquisition function using a joint likelihood. We demonstrate joint c-TPE is invariant to constraint duplication whereas independent c-TPE degrades as the product accumulates duplicated factors. We outline practical tradeoffs between the formulations and directions for future study.
Topics
Classified with taxonomy v2 on Sat, 5 Sept 2026.