Mutable

Base Classes

class nni.mutable.Mutable

Mutable is the base class for every class representing a search space.

To make a smooth experience of writing new search spaces, we provide multiple kinds of mutable subclasses. There are basically two types of needs:

1. Express one variable (aka. one dimension / parameter) of the search space. We provide the expressiveness to describe the domain on this dimension.

2. Composition of multiple dimensions. For example, A new variable that is the sum of two existing variables. Or a PyTorch module (in the NAS scenario) that relies on one or several variables.

In most cases, spaces are type 2, because it’s relatively straightforward to programmers, and easy to be put into a evaluation process. For example, when a model is to search, directly programming on the deep learning model would be the most straightforward way to define the space.

On the other hand, most algorithms only care about the underlying variables that constitutes the space, rather than the complex compositions. That is, the basic dimensions of categorical / continuous values in the space. Note that, this is only algorithm-friendly, but not friendly to those who writes the space.

We provide two methods to achieve the best both worlds. simplify() is the method to get the basic dimensions (type 1). Algorithms then use the simplified space to run the search, generate the samples (which are also in the format of the simplified space), and then, freeze() is the method to get the frozen version of the space with the sample.

For example:

>>> from nni.mutable import Categorical
>>> mutable = Categorical([2, 3]) + Categorical([5, 7])
>>> mutable
Categorical([2, 3], label='global_1') + Categorical([5, 7], label='global_2')
>>> mutable.simplify()
{'global_1': Categorical([2, 3], label='global_1'), 'global_2': Categorical([5, 7], label='global_2')}
>>> sample = {'global_1': 2, 'global_2': 7}
>>> mutable.freeze(sample)
9

In the example above, we create a new mutable that is the sum of two existing variables (with MutableExpression), and then simplify it to get the basic dimensions. The sample here is a dictionary of parameters. It should have the exactly same keys as the simplified space, and values are replaced with the sampled values. The sample can be used in both contains() and freeze().

• Use if mutable.contains(sample) to check whether a sample is valid.

• Use mutable.freeze(sample) to create a fixed version of the mutable.

Subclasses of mutables must implement leaf_mutables() (which is the implementation of simplify()), check_contains(), and freeze(). Subclasses of LabeledMutable must also implement default(), random() and grid().

One final note, Mutable is designed to be framework agnostic. It doesn’t have any dependency on deep learning frameworks like PyTorch.

as_legacy_dict()

Convert the mutable into the legacy dict representation.

For example, {"_type": "choice", "_value": [1, 2, 3]} is the legacy dict representation of nni.mutable.Categorical([1, 2, 3]).

check_contains(sample)

Check whether sample is validly sampled from the mutable space. Return an exception if the sample is invalid, otherwise return None. Subclass is recommended to override this rather than contains().

Parameters:

sample (Sample) – See freeze().

Return type:

Optionally a SampleValidationError if the sample is invalid.

contains(sample)

Check whether sample is validly sampled from the mutable space.

Parameters:

sample (Dict[str, Any]) – See freeze().

Return type:

Whether the sample is valid.

default(memo=None)

Return the default value of the mutable. Useful for debugging and sanity check. The returned value should be one of the possible results of freeze().

The default implementation of default() is to call default() on each of the simplified values and then freeze the result.

Parameters:

memo (Sample | None) – A dict of mutable labels and their default values. Use this to share the sampled value among mutables with the same label.

equals(other)

Compare two mutables.

Please use equals() to compare two mutables, instead of ==, because == will generate mutable expressions.

extra_repr()

Return a string representation of the extra information.

freeze(sample)

Create a frozen (i.e., fixed) version of this mutable, based on sample in the format of simplify().

For example, the frozen version of an integer variable is a constant. The frozen version of a mathematical expression is an evaluated value. The frozen version of a layer choice is a fixed layer.

Parameters:

sample (Dict[str, Any]) – The sample should be a dict, having the same keys as simplify(). The values of the dict are the choice of the corresponding mutable, whose format varies depending on the specific mutable format.

Return type:

The frozen version of this mutable.

See also

LabeledMutable

grid(memo=None, granularity=None)

Return a grid of sample points that can be possibly sampled from the mutable. Used in grid search strategy. It should return all the possible results of freeze().

The default implementation of grid() is to call iterate over the product of all the simplified grid values. Specifically, the grid will be iterated over in a depth-first-search order.

The deduplication of grid() (even with a certain granularity) is not guaranteed. But it will be done at a best-effort level. In most cases, results from grid() with a lower granularity will be a subset of results from grid() with a higher granularity. The caller should handle the deduplication.

Parameters:

• memo (Sample | None) – A dict of mutable labels and their values in the current grid point. Use this to share the sampled value among mutables with the same label.

• granularity (int | None) – Optional integer to specify the level of granularity of the grid. This only affects the cases where the grid is not a finite set. See Numerical for details.

leaf_mutables(is_leaf)

Return all the leaf mutables.

The mutables could contain duplicates (duplicate instances / duplicate labels). All leaf mutables should be labeled for the purpose of deduplication in simplify().

Subclass override this (and possibly call leaf_mutables() of sub-mutables). When they are implemented, they could use is_leaf to check whether a mutable should be expanded, and use yield to return the leaf mutables.

Parameters:

is_leaf (Callable[[Mutable], bool]) – A function that takes a mutable and returns whether it’s a leaf mutable. See simplify().

Return type:

An iterable of leaf mutables.

random(memo=None, random_state=None)

Randomly sample a value of the mutable. Used in random strategy. The returned value should be one of the possible results of freeze().

The default implementation of random() is to call random() on each of the simplified values and then freeze the result.

It’s possible that random() raises SampleValidationError, e.g., in cases when constraints are violated.

Parameters:

memo (Sample | None) – A dict of mutable labels and their random values. Use this to share the sampled value among mutables with the same label.

robust_default(memo=None, retries=1000)

Return the default value of the mutable. Will retry with random() in case of failure.

It’s equivalent to the following pseudo-code:

for attempt in range(retries + 1):
    try:
        if attempt == 0:
            return self.default()
        else:
            return self.random()
    except SampleValidationError:
        pass

Parameters:

• memo (Sample | None) – A dict of mutable labels and their default values. Use this to share the sampled value among mutables with the same label.

• retries (int) – If the default sample is not valid, we will retry to invoke random() for retries times, until a valid sample is found. Otherwise, an exception will be raised, complaining that no valid sample is found.

simplify(is_leaf=None)

Summarize all underlying uncertainties in a schema, useful for search algorithms.

The default behavior of simplify() is to call leaf_mutables() to retrieve a list of mutables, and deduplicate them based on labels. Thus, subclasses only need to override leaf_mutables().

Parameters:

is_leaf (Callable[[Mutable], bool] | None) – A function to check whether a mutable is a leaf mutable. If not specified, MutableSymbol instances will be treated as leaf mutables. is_leaf is useful for algorithms to decide whether to, (i) expand some mutables so that less mutable types need to be worried about, or (ii) collapse some mutables so that more information could be kept.

Return type:

The keys are labels, and values are corresponding labeled mutables.

Notes

Ideally simplify() should be idempotent. That being said, you can wrap the simplified results with a MutableDict and call simplify again, it will get you the same results. However, in practice, the order of dict keys might not be guaranteed.

There is also no guarantee that all mutables returned by simplify() are leaf mutables that will pass the check of is_leaf. There are certain mutables that are not leaf by default, but can’t be expanded any more (e.g., MutableAnnotation). As long as they are labeled, they are still valid return values. The caller can decide whether to raise an exception or simply ignore them.

See also

LabeledMutable

validate(sample)

Validate a sample. Calls check_contains() and raises an exception if the sample is invalid.

Parameters:

sample (Dict[str, Any]) – See freeze().

Raises:

nni.mutable.exception.SampleValidationError – If the sample is invalid.

Return type:

None

class nni.mutable.LabeledMutable

Mutable with a label. This should be the super-class of most mutables. The labels are widely used in simplified result, as well as samples. Usually a mutable must be firstly converted into one or several LabeledMutable, before strategy can recognize and process it.

When two mutables have the same label, they semantically share the same choice. That means, the choices of the two mutables will be shared. The labels can be either auto-generated, or provided by the user.

Being a LabeledMutable doesn’t necessarily mean that it is a leaf mutable. Some LabeledMutable can be further simplified into multiple leaf mutables. In the current implementation, there are basically two kinds of LabeledMutable:

1. MutableSymbol. This is usually referred to as a “parameter”. They produce a key-value in the sample.

2. MutableAnnotation. They function as some kind of hint, and do not generate a key-value in the sample. Sometimes they can also be simplified and the MutableSymbol they depend on would appear in the simplified result.

class nni.mutable.MutableSymbol(label)

MutableSymbol corresponds to the concept of a variable / hyper-parameter / dimension.

For example, a learning rate with a uniform distribution between 0.1 and 1, or a convolution filter that is either 32 or 64.

MutableSymbol is a subclass of Symbol. Therefore they support arithmetic operations. The operation results will be a MutableExpression object.

See also

nni.mutable.symbol.Symbol

float()

Cast the mutable to a float.

int()

Cast the mutable to an integer.

class nni.mutable.MutableExpression(function, repr_template, arguments)

Expression of mutables. Common use cases include: summation of several mutables, binary comparison between two mutables.

The expression is defined by a operator and a list of operands, which must be one or several MutableSymbol or MutableExpression.

The expression can be simplified into a dict of LabeledMutable. It can also be evaluated to be a concrete value (via freeze()), when the values it depends on have been given.

See also

nni.mutable.symbol.SymbolicExpression

Categorical

class nni.mutable.Categorical(values, *, weights=None, default='__missing__', label=None)

Choosing one from a list of categorical values.

Parameters:

• values (Iterable[Choice]) – The list of values to choose from. There are no restrictions on value types. They can be integers, strings, and even dicts and lists. There is no intrinsic ordering of the values, meaning that the order in which the values appear in the list doesn’t matter. The values can also be an iterable, which will be expanded into a list.

• weights (list[float] | None) – The probability distribution of the values. Should be an array with the same length as values. The sum of the distribution should be 1. If not specified, the values will be chosen uniformly.

• default (Choice | str) – Default value of the mutable. If not specified, the first value will be used.

• label (str) – The label of the mutable. If not specified, a label will be auto-generated.

Examples

>>> x = Categorical([2, 3, 5], label='x1')
>>> x.simplify()
{'x1': Categorical([2, 3, 5], label='x1')}
>>> x.freeze({'x1': 3})
3

default(memo=None)

The default() of Categorical is the first value unless default value is set.

See also

Mutable.default

grid(memo=None, granularity=None)

Return also values as a grid. Sorted by distribution from most likely to least likely.

See also

Mutable.grid

random(memo=None, random_state=None)

Randomly sample a value from choices. Distribution is respected if provided.

See also

Mutable.random

CategoricalMultiple

class nni.mutable.CategoricalMultiple(values, *, n_chosen=None, weights=None, default='__missing__', label=None)

Choosing multiple from a list of values without replacement.

It’s implemented with a different class because for most algorithms, it’s very different from Categorical.

CategoricalMultiple can be either treated as a atomic LabeledMutable (i.e., simple format), or be further simplified into a series of more fine-grained mutables (i.e., categorical format).

In categorical format, class:CategoricalMultiple can be broken down to a list of Categorical of true and false, each indicating whether the choice on the corresponding position should be chosen. A constraint will be added if n_chosen is not None. This is useful for some algorithms that only support categorical mutables. Note that the prior distribution will be lost in this process.

Parameters:

• values (Iterable[Choice]) – The list of values to choose from. See Categorical.

• n_chosen (int | None) – The number of values to choose. If not specified, any number of values can be chosen.

• weights (list[float] | None) – The probability distribution of the values. Should be an array with the same length as values. When n_chosen is None, it’s the probability that each candidate is chosen. When n_chosen is not None, the distribution should sum to 1.

• default (list[Choice] | str) – Default value of the mutable. If not specified, the first n_chosen value will be used.

• label (str) – The label of the mutable. If not specified, a label will be auto-generated.

Examples

>>> x = CategoricalMultiple([2, 3, 5, 7], n_chosen=2, label='x2')
>>> x.random()
[2, 7]
>>> x.simplify()
{'x2': CategoricalMultiple([2, 3, 5, 7], n_chosen=2, label='x2')}
>>> x.simplify(lambda t: not isinstance(t, CategoricalMultiple))
{
    'x2/0': Categorical([True, False], label='x2/0'),
    'x2/1': Categorical([True, False], label='x2/1'),
    'x2/2': Categorical([True, False], label='x2/2'),
    'x2/3': Categorical([True, False], label='x2/3'),
    'x2/n': ExpressionConstraint(...)
}
>>> x.freeze({'x2': [3, 5]})
[3, 5]
>>> x.freeze({'x2/0': True, 'x2/1': False, 'x2/2': True, 'x2/3': False})
[2, 5]

default(memo=None)

The first n_chosen values. If n_chosen is None, return all values.

See also

Mutable.default

grid(memo=None, granularity=None)

Iterate over all possible values.

If n_chosen is None, iterate over all possible subsets, in the order of increasing length. Otherwise, iterate over all possible combinations of n_chosen length, using the implementation of itertools.combinations().

See also

Mutable.grid

leaf_mutables(is_leaf)

If invoking is_leaf returns true, return self. Otherwise, further break it down to several Categorical and Constraint.

See also

Mutable.leaf_mutables

random(memo=None, random_state=None)

Randomly sample n_chosen values. If n_chosen is None, return an arbitrary subset.

The random here takes distribution into account.

See also

Mutable.random

Numerical

class nni.mutable.Numerical(low=-inf, high=inf, *, mu=None, sigma=None, log_distributed=False, quantize=None, distribution=None, default='__missing__', label=None)

One variable from a univariate distribution.

It supports most commonly used distributions including uniform, loguniform, normal, lognormal, as well as the quantized version. It also supports using arbitrary distribution from scipy.stats.

Parameters:

• low (float) – The lower bound of the domain. Used for uniform and loguniform. It will also be used to clip the value if it is outside the domain.

• high (float) – The upper bound of the domain. Used for uniform and loguniform. It will also be used to clip the value if it is outside the domain.

• mu (float | None) – The mean of the domain. Used for normal and lognormal.

• sigma (float | None) – The standard deviation of the domain. Used for normal and lognormal.

• log_distributed (bool) – Whether the domain is log distributed.

• quantize (float | None) – If specified, the final value will be postprocessed with clip(round(uniform(low, high) / q) * q, low, high), where the clip operation is used to constrain the generated value within the bounds. For example, when quantize is 2.5, all the values will be rounded to the nearest multiple of 2.5. Note that, if low or high is not a multiple of quantize, it will be clipped to low or high after rounding.

• distribution (_distn_infrastructure.rv_frozen | None) – The distribution to use. It should be a rv_frozen instance, which can be obtained by calling scipy.stats.distribution_name(...). If specified, low, high, mu, sigma, log_distributed will be ignored.

• default (float | str) – The default value. If not specified, the default value will be the median of distribution.

• label (str) – The label of the variable.

Examples

To create a variable uniformly sampled from 0 to 1:

Numerical(low=0, high=1)

To create a variable normally sampled with mean 2 and std 3:

Numerical(mu=2, sigma=3)

To create a normally sampled variable with mean 0 and std 1, but always in the range of [-1, 1] (note that it’s not truncated normal though):

Numerical(mu=0, sigma=1, low=-1, high=1)

To create a variable uniformly sampled from 0 to 100, but always multiple of 2:

Numerical(low=0, high=100, quantize=2)

To create a reciprocal continuous random variable in the range of [2, 6]:

Numerical(low=2, high=6, log_distributed=True)

To create a variable sampled from a custom distribution:

from scipy.stats import beta Numerical(distribution=beta(2, 5))

default(memo=None)

If default value is not specified, Numerical.default() returns median.

See also

Mutable.default

equals(other)

Checks whether two distributions are equal by examining the parameters.

See also

Mutable.equals

grid(memo=None, granularity=None)

Yield a list of samples within the distribution.

Since the grid of continuous space is infinite, we use granularity to specify the number of samples to yield. If granularity = 1, grid only explores median point of the distribution. If granularity = 2, the quartile points of the distribution will also be generated. Granularity = 3 explores the 1/8th points of the distribution, and so on. If not specified, granularity defaults to 1.

Grid will eliminate duplicates within the same granularity. Duplicates across different granularity will be ignored.

Examples

>>> list(Numerical(0, 1).grid(granularity=2))
[0.25, 0.5, 0.75]
>>> list(Numerical(0, 1).grid(granularity=3))
[0.125, 0.25, 0.375, 0.5, 0.625, 0.75, 0.875]
>>> list(Numerical(mu=0, sigma=1).grid(granularity=2))
[-0.6744897501960817, 0.0, 0.6744897501960817]
>>> list(Numerical(mu=0, sigma=1, quantize=0.5).grid(granularity=3))
[-1.0, -0.5, 0.0, 0.5, 1.0]

See also

Mutable.grid

qclip(x)

Quantize and clip the value, to satisfy the Q-constraint and low-high bounds.

random(memo=None, random_state=None)

Directly sample from the distribution.

See also

Mutable.random

Containers

nni.mutable.container

alias of <module ‘nni.mutable.container’ from ‘/home/docs/checkouts/readthedocs.org/user_builds/nni/checkouts/latest/nni/mutable/container.py’>

Annotation

class nni.mutable.annotation.Constraint

Constraints put extra requirements to make one sample valid.

For example, a constraint can be used to express that a variable should be larger than another variable, or certain combinations of variables should be strictly avoided.

Constraint is a subclass of MutableAnnotation, and thus can be used as a normal mutable. It has a special contains() method, which is used to check whether a sample satisfies the constraint. A constraint is satisfied if and only if contains() returns None.

In general, users should inherit from Constraint to implement customized constraints. ExpressionConstraint is a special constraint that can be used to express constraints in a more concise way.

See also

ExpressionConstraint

check_contains(sample)

Override this to implement customized constraint. It should return None if the sample satisfies the constraint. Otherwise return a ConstraintViolation exception.

See also

nni.mutable.Mutable.check_contains

freeze(sample)

Validate the sample (via validate()) and returns None.

See also

nni.mutable.Mutable.freeze, nni.mutable.Mutable.validate

grid(memo=None, granularity=None)

Yield all samples that satisfy the constraint.

If some samples the constraint relies on have not been frozen yet, it will be sampled here and put into the memo. After that, it checks whether the sample satisfies the constraint after sampling (via contains()). If the sample doesn’t satisfy the constraint, it will be discarded.

Each yielded sample of the Constraint.grid() itself is None, because Constraint.freeze() also returns None.

leaf_mutables(is_leaf)

Override this to implement customized constraint. It should return a list of leaf mutables that are used in the constraint.

See also

nni.mutable.Mutable.leaf_mutables

class nni.mutable.annotation.ExpressionConstraint(expression, *, label=None)

A constraint that is expressed as MutableExpression.

The expression must evaluate to be true to satisfy the constraint.

Parameters:

• expression (MutableExpression) – A MutableExpression that evaluates to a boolean value.

• label (str) – The semantic name of the constraint.

Examples

>>> a = Categorical([1, 3])
>>> b = Categorical([2, 4])
>>> list(MutableList([a, b, ExpressionConstraint(a + b == 5)]).grid())
[[1, 4, None], [3, 2, None]]

class nni.mutable.annotation.MutableAnnotation

Provide extra annotation / hints for a search space.

Sometimes, people who authored search strategies might want to recognize some hints from the search space. For example,

• Adding some extra constraints between two parameters in the space.

• Marking some choices as “nodes” in a cell.

• Some parameter-combinations should be avoided or explored at the very first.

MutableAnnotation is defined to be transparent, i.e., it doesn’t generate any new dimension by itself, and thus typically doesn’t introduce a new key in the sample received by nni.mutable.Mutable.freeze(), but it affects the sampling the freezing process implicitly.

This class is useful for isinstance check.

Frozen

nni.mutable.frozen.ensure_frozen(mutable, *, strict=True, sample=None, retries=1000)

Ensure a mutable is frozen. Used when passing the mutable to a function which doesn’t accept a mutable.

If the argument is not a mutable, nothing happens. Otherwise, freeze() will be called if sample is given. If sample is None, ensure_frozen() will also try to fill the sample with the content in frozen_context. Or else robust_default() will be called on the mutable.

Parameters:

• mutable (nni.mutable.Mutable or any) – The mutable to freeze.

• strict (bool) – Whether to raise an error if sample context is not provided and not found.

• sample (Sample | None) – The context to freeze the mutable with.

• retries (int) – Control the number of retries in case robust_default() is called.

Examples

>>> with frozen_context({'a': 2}):
...     ensure_frozen(Categorical([1, 2, 3], label='a'))
2
>>> ensure_frozen(Categorical([1, 2, 3]), strict=False)
1
>>> ensure_frozen(Categorical([1, 2, 3], label='a'), sample={'a': 2}, strict=False)
2
>>> ensure_frozen('anything', strict=False)
'anything'

class nni.mutable.frozen.frozen_context(sample=None)

Context manager to set a sample into context. Then the sample will be retrievable from an arbitrary level of function calls via current_frozen_context().

There are two use cases:

1. Setting a global sample so that some modules can directly create the frozen version, rather than first-create-and-freeze.

2. Sharing default / dry-run samples when the search space is dynamically created.

The implementation is basically adding another layer of empty dict on top of a global stack. When retrieved, all dicts in the stack will be merged, from the bottom to the top. When updated, only the dict on the top will be updated.

Parameters:

sample (Sample | None) – The sample to be set into context.

Return type:

Context manager that provides a frozen context.

Examples

::

def some_func():

print(frozen_context.current()[‘learning_rate’]) # 0.1

with frozen_context({‘learning_rate’: 0.1}):

some_func()

static bypass()

Ignore the most recent frozen_context.

This is useful in creating a search space within a frozen_context() context. Under the hood, it only disables the most recent one frozen context, which means, if it’s currently in a nested with-frozen-arch context, multiple bypass() contexts is required.

Examples

>>> with frozen_context(arch_dict):
...     with frozen_context.bypass():
...         model_space = ModelSpace()

static current()

Retrieve the current frozen context. If multiple layers have been found, they would be merged from bottom to top.

Returns:

• The sample in frozen context.

• If no sample is found, return none.

Return type:

dict | None

static update(sample)

Update the current dry run context. Only the topmost context will be updated.

Parameters:

sample (Dict[str, Any]) – The sample to be updated into context.

class nni.mutable.frozen.frozen_factory(callable, sample)

Create a factory object that invokes a function with a frozen context.

Parameters:

• callable (Callable[..., Any]) – The function to be invoked.

• sample (Sample | frozen_context) – The sample to be used as the frozen context.

Examples

>>> factory = frozen_factory(ModelSpaceClass, {"choice1": 3})
>>> model = factory(channels=16, classes=10)

Exception

exception nni.mutable.exception.ConstraintViolation(msg, paths=None)

Exception raised when constraint is violated.

exception nni.mutable.exception.SampleMissingError(label_or_msg: str, keys: list[str])

exception nni.mutable.exception.SampleMissingError(label_or_msg: str)

Raised when a required sample with a particular label is missing.

exception nni.mutable.exception.SampleValidationError(msg, paths=None)

Exception raised when a sample is invalid.

Symbol

Infrastructure for symbolic execution.

Symbolic execution is a technique for executing programs on symbolic inputs. It supports arithmetic operations and comparisons on abstract symbols, and can be used to represent symbolic expressions for potential evaluation and optimization.

The symbolic execution is implemented by overriding the operators of the Symbol class. The operators are implemented in a way that they can be chained together to form a SymbolicExpression. The symbolic execution is lazy, which means that the expression will not be evaluated until the final value is substituted.

Examples

>>> from nni.mutable.symbol import Symbol
>>> x, y = Symbol('x'), Symbol('y')
>>> expr = x * x + y * 2
>>> print(expr)
(x * x) + (y * 2)
>>> list(expr.leaf_symbols())
[Symbol('x'), Symbol('x'), Symbol('y')]
>>> expr.evaluate({'x': 2, 'y': 3})
10

class nni.mutable.symbol.Symbol(label)

The leaf node of a symbolic expression. Each Symbol represents one variable in the expression.

Variable with the same label share the same value.

Operations on symbols (e.g., a + b) will result in a new SymbolicExpression.

Parameters:

label (str) – Each symbol is bound with a label, i.e., the variable name.

class nni.mutable.symbol.SymbolicExpression(function, repr_template, arguments)

Implementation of symbolic execution.

Each instance of SymbolicExpression is a node on the expression tree, with a function and a list of children (i.e., function arguments).

The expression is designed to be compatible with native Python expressions. That means, the static methods (as well as operators) can be also applied on plain Python values.

static case(pred_expr_pairs)

Return the first expression with predicate that is true.

For example:

if (x < y) return 17;
else if (x > z) return 23;
else (y > z) return 31;

Equivalent to:

SymbolicExpression.case([(x < y, 17), (x > z, 23), (y > z, 31)])

Notes

This function performs lazy evaluation. Only the expression will be recorded when the function is called. The real evaluation happens when the inner value choice has determined its final decision. If no value choice is contained in the parameter list, the evaluation will be intermediate.

static condition(pred, true, false)

Return true if the predicate pred is true else false.

Examples

>>> SymbolicExpression.condition(Symbol('x') > Symbol('y'), 2, 1)

Notes

This function performs lazy evaluation. Only the expression will be recorded when the function is called. The real evaluation happens when the inner value choice has determined its final decision. If no value choice is contained in the parameter list, the evaluation will be intermediate.

property expr_cls: Type[SymbolicExpression]

The created expression will be using this class.

leaf_symbols()

Return a generator of all leaf symbols.

Useful for when you want to inspect when the symbols come from. No deduplication even if the symbols has duplicates.

static max(arg0, *args)

Returns the maximum value from a list of symbols. The usage should be similar to Python’s built-in symbols, where the parameters could be an iterable, or at least two arguments.

Notes

This function performs lazy evaluation. Only the expression will be recorded when the function is called. The real evaluation happens when the inner value choice has determined its final decision. If no value choice is contained in the parameter list, the evaluation will be intermediate.

static min(arg0, *args)

Returns the minimum value from a list of symbols. The usage should be similar to Python’s built-in symbols, where the parameters could be an iterable, or at least two arguments.

Notes

This function performs lazy evaluation. Only the expression will be recorded when the function is called. The real evaluation happens when the inner value choice has determined its final decision. If no value choice is contained in the parameter list, the evaluation will be intermediate.

static switch_case(branch, expressions)

Select the expression that matches the branch.

C-style switch:

switch (branch) {  // c-style switch
    case 0: return 17;
    case 1: return 31;
}

Equivalent to:

SymbolicExpression.switch_case(branch, {0: 17, 1: 31})

Notes

This function performs lazy evaluation. Only the expression will be recorded when the function is called. The real evaluation happens when the inner value choice has determined its final decision. If no value choice is contained in the parameter list, the evaluation will be intermediate.

static to_float(obj)

Convert the current value to a float. .. rubric:: Notes

This function performs lazy evaluation. Only the expression will be recorded when the function is called. The real evaluation happens when the inner value choice has determined its final decision. If no value choice is contained in the parameter list, the evaluation will be intermediate.

static to_int(obj)

Convert the current value to an integer. .. rubric:: Notes

This function performs lazy evaluation. Only the expression will be recorded when the function is called. The real evaluation happens when the inner value choice has determined its final decision. If no value choice is contained in the parameter list, the evaluation will be intermediate.

Utilities

class nni.mutable.utils.ContextStack(key, value)

This is to maintain a globally-accessible context environment that is visible to everywhere.

To initiate:

with ContextStack(namespace, value):
    ...

Inside the context, you can access the nearest value put into with:

get_current_context(namespace)

Notes

ContextStack is not multi-processing safe. Also, the values will get cleared for a new process.

exception nni.mutable.utils.NoContextError

Exception raised when context is missing.

nni.mutable.utils.auto_label(name=None, scope=None)

Automatically generate a formatted and reproducible label.

In case name is not set, the label name will use the uid sequence of scope. If scope is not set, it will fetch the nearest scope. If no scope is found, it will use the global scope (label_scope.global_()).

If the scope is found and is not global scope, the scope’s full name will be prepended to the label, i.e., the label will then be formatted as {scope}/{label}. Otherwise, there are two cases. Firstly, if label is manually specified, it will be returned directly. Secondly, we rely on the global scope to generate the label name, and the scope name will still be prepended. The rules can be better explained with the examples below.

Notes

We always recommend specifying the label manually.

Parameters:

• name (str | label | None) – The label name to use. If not specified, it will be generated by the scope.

• scope (label_scope | None) – The scope to use. If not specified, the nearest scope will be used.

Returns:

• A string that represents the label.

• For advanced users, please be advised that

• the returned object would be a label object, rather than a string.

• This is to make sure auto_label() is idempotent.

• We expect some side effects, but it should work seamlessly with most of the code.

Return type:

str

Examples

>>> label1 = auto_label('bar')          # bar, because the scope is global
>>> label2 = auto_label()               # global/1, because label is not provided
>>> with label_scope('foo'):
...     label3 = auto_label()           # foo/1, because in the scope "foo"
>>> with label_scope():                 # scope is global/2
...     label4 = auto_label()           # global/2/1
>>> with label_scope('another'):
...     label5 = auto_label()           # another/1
...     label6 = auto_label('thing')    # another/thing
...     label7 = auto_label()           # another/2

class nni.mutable.utils.label_scope(basename=None, *, _path=None)

To support automatic labeling of mutables.

Labels are named like a file-system. The analogy here is that: scope is like a directory, and label is like a file. The label name is like a file name. It can’t contain slash (/) or underscore (_). The scope name is like a directory name. It also can’t contain / or _. When we refer to a “label”, we will usually use the full name, which is like an absolute file path.

label_scope is usually jointly used with auto_label(), where label_scope is used to generate the “scope” (directory) part, and auto_label() is used to generate the “name” (file) part. A label_scope can be entered, and then auto_label() can be called inside. The labels as well as scopes generated inside can be automatically named with natural integers starting from 1 (see examples below), and we guarantee the generation of labels to be reproducible. It can also be naturally nested.

label_scope is NOT thread-safe. The behavior is undefined if multiple threads are trying to enter the scope at the same time.

label_scope is implemented based on ContextStack.

Parameters:

basename (str | label | label_scope | None) – The last part of current scope name. If not specified, it will be generated by the parent scope. If the parent scope is not found, the scope name will be param by default. label_scope is idempotent, so basename also accepts label_scope and label, though it will return a new instance.

Examples

>>> with label_scope('model'):
...     label1 = auto_label()       # model/1
...     label2 = auto_label()       # model/2
...     label3 = auto_label('foo')  # model/foo
...     with label_scope():
...         label4 = auto_label()   # model/3/1
...         label5 = auto_label()   # model/3/2
...     with label_scope('another'):
...         label6 = auto_label()   # model/another/1
...     with label_scope('model'):
...         label7 = auto_label()   # model/model/1
>>> with label_scope('model'):
...     label8 = auto_label()       # model/1, because the counter is reset
>>> with label_scope():
...     label9 = auto_label()       # global/1/1

property absolute_scope: str

Alias of name.

check_entered()

Raise error if the scope is not entered.

static current()

Fetch the nearest label scope activated by with.

If label scope is never used, or we are currently within no with-block, return none.

Examples

>>> with label_scope() as scope1:
...     # somewhere in the middle of the code.
...     label_scope.current()     # Return scope1

static global_()

Fetch the global label scope.

This label scope can be created on-the-fly and can live without the with-blocks.

property name: str

The full name of current namespace.

For example, model/cell/2.

next_label()

Generate the “name” part.

nni.mutable.utils.reset_uid(namespace='default')

Reset counter for a specific namespace.

nni.mutable.utils.uid(namespace='default')

Global counter for unique id. Not thread-safe.