Exploration Strategies for Multi-trial NAS

Usage of Exploration Strategy

To use an exploration strategy, users simply instantiate an exploration strategy and pass the instantiated object to RetiariiExperiment. Below is a simple example.

import nni.retiarii.strategy as strategy

exploration_strategy = strategy.Random(dedup=True)  # dedup=False if deduplication is not wanted

Supported Exploration Strategies

NNI provides the following exploration strategies for multi-trial NAS. Users could also customize new exploration strategies.

Name

Brief Introduction of Algorithm

Random Strategy

Randomly sampling new model(s) from user defined model space. (nni.retiarii.strategy.Random)

Grid Search

Sampling new model(s) from user defined model space using grid search algorithm. (nni.retiarii.strategy.GridSearch)

Regularized Evolution

Generating new model(s) from generated models using regularized evolution algorithm . (nni.retiarii.strategy.RegularizedEvolution)

TPE Strategy

Sampling new model(s) from user defined model space using TPE algorithm . (nni.retiarii.strategy.TPEStrategy)

RL Strategy

It uses PPO algorithm to sample new model(s) from user defined model space. (nni.retiarii.strategy.PolicyBasedRL)