Hyperparameter optimization

From Rice Wiki
Revision as of 21:06, 18 May 2024 by Rice (talk | contribs) (→‎Other techniques)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)


Aspects of an ML model is controlled by hyperparameters. Hyperparameter optimization is an iterative process that attempts to find the best hyperparameters for a given model.

Grid search

Grid search (aka. grid search cross validation) computes all possible combinations of hyperparameters based on your input (possible values for each hyperparameter). The model is then trained and validated, and the best performance result is taken.

Other techniques

Other techniques include

  • Random search: faster but no guarantee
  • Bayesian optimizaiton
  • Hyperband