AutoML supports six Hyperparameter Optimization (HPO) search algorithms. This page lists the supported regions and describes each algorithm to help you choose the right one for your use case.
Supported regions
AutoML is available in the following regions: China (Hangzhou), China (Shanghai), China (Beijing), China (Shenzhen), and China (Hong Kong).
Search algorithms
The table below summarizes all six algorithms. Detailed descriptions follow.
| Algorithm | Best for | Limitation |
|---|---|---|
| TPE | Complex, nonlinear, high-dimensional problems | Cannot model connections between parameters |
| GridSearch | Small, well-bounded search spaces | Only practical for a small number of possible combinations |
| Random | Nonlinear and high-dimensional problems when exhaustive search is not feasible | Randomly selects combinations without exhaustive evaluation |
| Evolution | Iterative, generation-based search; easy to extend | Requires multiple trials to converge |
| GP | Sequential trial budgets where a surrogate model can improve over time | Posterior distribution accuracy depends on amount of data obtained |
| PBT | Fixed compute budgets with model and hyperparameter optimization | Optimizes a fixed number of models |
TPE (Tree-structured Parzen Estimator)
Use TPE for complex, nonlinear, high-dimensional problems that require significant computation. TPE is the default algorithm in HPO.
TPE requires no additional dependencies and supports all search space types. Note that it does not model connections between different parameters.
For the underlying method, see Algorithms for Hyper-Parameter Optimization.
GridSearch
Use GridSearch when the number of possible combinations is small enough to evaluate exhaustively.
GridSearch evenly divides the search space into a grid and evaluates every combination to find the optimal one. This algorithm works well for a small number of possible combinations.
Random
Use Random Search for nonlinear and high-dimensional problems when exhaustive search is not feasible.
Random Search divides the search space into a grid and randomly selects one hyperparameter combination per trial. Unlike GridSearch, it does not evaluate all combinations, so it scales better to large search spaces.
Evolution
Use Evolution when you need an iterative, generation-based search that is easy to extend to new features, and when you can afford multiple trials.
Evolution initializes the search space and then selects the better-performing hyperparameters in each generation to produce offspring combinations. The algorithm requires multiple trials to converge.
This algorithm is based on Large-Scale Evolution of Image Classifiers.
GP (Gaussian Process)
Use GP when you can run enough sequential trials to build an accurate surrogate model. The algorithm becomes more effective as trial data accumulates.
GP is a Bayesian Optimization (BO) method that uses a Gaussian Process to estimate loss. As more trial data accumulates, the posterior distribution becomes more accurate and optimization improves.
PBT (Population Based Training)
Use PBT when you have a fixed compute budget and want to optimize models and their hyperparameters.
PBT is an asynchronous optimization algorithm designed for fixed computing resources. It optimizes a fixed number of models and their hyperparameters, continually iterating a single set of hyperparameters to find the optimal combination.
This algorithm is based on Population Based Training of Neural Networks.