Construct Model SpaceΒΆ
NNI provides powerful (and multi-level) APIs for users to easily express model space (or search space).
Mutation Primitives: high-level APIs (e.g., LayerChoice) that are utilities to build blocks in search space. In most cases, mutation pritimives should be straightforward yet expressive enough. We strongly recommend users to try them first, and report issues if those APIs are not satisfying.
Hyper-module Library: plug-and-play modules that are proved useful. They are usually well studied in research, and comes with pre-searched results. (For example, the optimal activation function in AutoActivation is reported to be Swish).
Mutator: for advanced users only. NNI provides interface to customize new mutators for expressing more complicated model spaces.
The following table summarizes all the APIs we have provided for constructing search space.
Name |
Category |
Brief Description |
---|---|---|
|
Mutation Primitives |
All model spaces should inherit this class |
|
Mutation Primitives |
Modules with mutable parameters should inherit this class |
|
Mutation Primitives |
Select from some PyTorch modules |
|
Mutation Primitives |
Select from some inputs (tensors) |
|
Mutation Primitives |
Repeat a block by a variable number of times |
|
Mutation Primitives |
Cell structure popularly used in literature |
|
Mutation Primitives |
Cell structure (variant) proposed by NAS-Bench-101 |
|
Mutation Primitives |
Cell structure (variant) proposed by NAS-Bench-201 |
|
Hyper-modules library |
Searching for activation functions |
|
Flexible mutations on graphs. See tutorial here |