Construct Model SpaceΒΆ
NNI provides powerful (and multi-level) APIs for users to easily express model space (or search space).
Mutation Primitives: high-level APIs (e.g., ValueChoice, LayerChoice) that are utilities to build blocks in search space. In most cases, mutation pritimives should be straightforward yet expressive enough. We strongly recommend users to try them first, and report issues if those APIs are not satisfying.
Hyper-module Library: plug-and-play modules that are proved useful. They are usually well studied in research, and comes with pre-searched results. (For example, the optimal activation function in AutoActivation is reported to be Swish).
Mutator: for advanced users only. NNI provides interface to customize new mutators for expressing more complicated model spaces.
The following table summarizes all the APIs we have provided for constructing search space.
Name |
Category |
Brief Description |
---|---|---|
|
Select from some PyTorch modules |
|
|
Select from some inputs (tensors) |
|
|
Select from some candidate values |
|
|
Repeat a block by a variable number of times |
|
|
Cell structure popularly used in literature |
|
|
Cell structure (variant) proposed by NAS-Bench-101 |
|
|
Cell structure (variant) proposed by NAS-Bench-201 |
|
|
Searching for activation functions |
|
|
Flexible mutations on graphs. See tutorial here |