Mutable¶
Base Classes¶
- class nni.mutable.Mutable[source]¶
Mutable is the base class for every class representing a search space.
To make a smooth experience of writing new search spaces, we provide multiple kinds of mutable subclasses. There are basically two types of needs:
Express one variable (aka. one dimension / parameter) of the search space. We provide the expressiveness to describe the domain on this dimension.
Composition of multiple dimensions. For example, A new variable that is the sum of two existing variables. Or a PyTorch module (in the NAS scenario) that relies on one or several variables.
In most cases, spaces are type 2, because it’s relatively straightforward to programmers, and easy to be put into a evaluation process. For example, when a model is to search, directly programming on the deep learning model would be the most straightforward way to define the space.
On the other hand, most algorithms only care about the underlying variables that constitutes the space, rather than the complex compositions. That is, the basic dimensions of categorical / continuous values in the space. Note that, this is only algorithm-friendly, but not friendly to those who writes the space.
We provide two methods to achieve the best both worlds.
simplify()
is the method to get the basic dimensions (type 1). Algorithms then use the simplified space to run the search, generate the samples (which are also in the format of the simplified space), and then,freeze()
is the method to get the frozen version of the space with the sample.For example:
>>> from nni.mutable import Categorical >>> mutable = Categorical([2, 3]) + Categorical([5, 7]) >>> mutable Categorical([2, 3], label='global_1') + Categorical([5, 7], label='global_2') >>> mutable.simplify() {'global_1': Categorical([2, 3], label='global_1'), 'global_2': Categorical([5, 7], label='global_2')} >>> sample = {'global_1': 2, 'global_2': 7} >>> mutable.freeze(sample) 9
In the example above, we create a new mutable that is the sum of two existing variables (with
MutableExpression
), and then simplify it to get the basic dimensions. The sample here is a dictionary of parameters. It should have the exactly same keys as the simplified space, and values are replaced with the sampled values. The sample can be used in bothcontains()
andfreeze()
.Use
if mutable.contains(sample)
to check whether a sample is valid.Use
mutable.freeze(sample)
to create a fixed version of the mutable.
Subclasses of mutables must implement
leaf_mutables()
(which is the implementation ofsimplify()
),check_contains()
, andfreeze()
. Subclasses ofLabeledMutable
must also implementdefault()
,random()
andgrid()
.One final note,
Mutable
is designed to be framework agnostic. It doesn’t have any dependency on deep learning frameworks like PyTorch.- as_legacy_dict()[source]¶
Convert the mutable into the legacy dict representation.
For example,
{"_type": "choice", "_value": [1, 2, 3]}
is the legacy dict representation ofnni.mutable.Categorical([1, 2, 3])
.
- check_contains(sample)[source]¶
Check whether sample is validly sampled from the mutable space. Return an exception if the sample is invalid, otherwise return
None
. Subclass is recommended to override this rather thancontains()
.- Parameters:
sample (Sample) – See
freeze()
.- Return type:
Optionally a
SampleValidationError
if the sample is invalid.
- contains(sample)[source]¶
Check whether sample is validly sampled from the mutable space.
- Parameters:
sample (Dict[str, Any]) – See
freeze()
.- Return type:
Whether the sample is valid.
- default(memo=None)[source]¶
Return the default value of the mutable. Useful for debugging and sanity check. The returned value should be one of the possible results of
freeze()
.The default implementation of
default()
is to calldefault()
on each of the simplified values and then freeze the result.- Parameters:
memo (Sample | None) – A dict of mutable labels and their default values. Use this to share the sampled value among mutables with the same label.
- equals(other)[source]¶
Compare two mutables.
Please use
equals()
to compare two mutables, instead of==
, because==
will generate mutable expressions.
- freeze(sample)[source]¶
Create a frozen (i.e., fixed) version of this mutable, based on sample in the format of
simplify()
.For example, the frozen version of an integer variable is a constant. The frozen version of a mathematical expression is an evaluated value. The frozen version of a layer choice is a fixed layer.
- Parameters:
sample (Dict[str, Any]) – The sample should be a dict, having the same keys as
simplify()
. The values of the dict are the choice of the corresponding mutable, whose format varies depending on the specific mutable format.- Return type:
The frozen version of this mutable.
See also
- grid(memo=None, granularity=None)[source]¶
Return a grid of sample points that can be possibly sampled from the mutable. Used in grid search strategy. It should return all the possible results of
freeze()
.The default implementation of
grid()
is to call iterate over the product of all the simplified grid values. Specifically, the grid will be iterated over in a depth-first-search order.The deduplication of
grid()
(even with a certain granularity) is not guaranteed. But it will be done at a best-effort level. In most cases, results fromgrid()
with a lower granularity will be a subset of results fromgrid()
with a higher granularity. The caller should handle the deduplication.- Parameters:
memo (Sample | None) – A dict of mutable labels and their values in the current grid point. Use this to share the sampled value among mutables with the same label.
granularity (int | None) – Optional integer to specify the level of granularity of the grid. This only affects the cases where the grid is not a finite set. See
Numerical
for details.
- leaf_mutables(is_leaf)[source]¶
Return all the leaf mutables.
The mutables could contain duplicates (duplicate instances / duplicate labels). All leaf mutables should be labeled for the purpose of deduplication in
simplify()
.Subclass override this (and possibly call
leaf_mutables()
of sub-mutables). When they are implemented, they could useis_leaf
to check whether a mutable should be expanded, and useyield
to return the leaf mutables.- Parameters:
is_leaf (Callable[[Mutable], bool]) – A function that takes a mutable and returns whether it’s a leaf mutable. See
simplify()
.- Return type:
An iterable of leaf mutables.
- random(memo=None, random_state=None)[source]¶
Randomly sample a value of the mutable. Used in random strategy. The returned value should be one of the possible results of
freeze()
.The default implementation of
random()
is to callrandom()
on each of the simplified values and then freeze the result.It’s possible that
random()
raisesSampleValidationError
, e.g., in cases when constraints are violated.- Parameters:
memo (Sample | None) – A dict of mutable labels and their random values. Use this to share the sampled value among mutables with the same label.
- robust_default(memo=None, retries=1000)[source]¶
Return the default value of the mutable. Will retry with
random()
in case of failure.It’s equivalent to the following pseudo-code:
for attempt in range(retries + 1): try: if attempt == 0: return self.default() else: return self.random() except SampleValidationError: pass
- Parameters:
memo (Sample | None) – A dict of mutable labels and their default values. Use this to share the sampled value among mutables with the same label.
retries (int) – If the default sample is not valid, we will retry to invoke
random()
forretries
times, until a valid sample is found. Otherwise, an exception will be raised, complaining that no valid sample is found.
- simplify(is_leaf=None)[source]¶
Summarize all underlying uncertainties in a schema, useful for search algorithms.
The default behavior of
simplify()
is to callleaf_mutables()
to retrieve a list of mutables, and deduplicate them based on labels. Thus, subclasses only need to overrideleaf_mutables()
.- Parameters:
is_leaf (Callable[[Mutable], bool] | None) – A function to check whether a mutable is a leaf mutable. If not specified,
MutableSymbol
instances will be treated as leaf mutables.is_leaf
is useful for algorithms to decide whether to, (i) expand some mutables so that less mutable types need to be worried about, or (ii) collapse some mutables so that more information could be kept.- Return type:
The keys are labels, and values are corresponding labeled mutables.
Notes
Ideally
simplify()
should be idempotent. That being said, you can wrap the simplified results with a MutableDict and call simplify again, it will get you the same results. However, in practice, the order of dict keys might not be guaranteed.There is also no guarantee that all mutables returned by
simplify()
are leaf mutables that will pass the check ofis_leaf
. There are certain mutables that are not leaf by default, but can’t be expanded any more (e.g.,MutableAnnotation
). As long as they are labeled, they are still valid return values. The caller can decide whether to raise an exception or simply ignore them.See also
- validate(sample)[source]¶
Validate a sample. Calls
check_contains()
and raises an exception if the sample is invalid.- Parameters:
sample (Dict[str, Any]) – See
freeze()
.- Raises:
nni.mutable.exception.SampleValidationError – If the sample is invalid.
- Return type:
None
- class nni.mutable.LabeledMutable[source]¶
Mutable
with a label. This should be the super-class of most mutables. The labels are widely used in simplified result, as well as samples. Usually a mutable must be firstly converted into one or severalLabeledMutable
, before strategy can recognize and process it.When two mutables have the same label, they semantically share the same choice. That means, the choices of the two mutables will be shared. The labels can be either auto-generated, or provided by the user.
Being a
LabeledMutable
doesn’t necessarily mean that it is a leaf mutable. SomeLabeledMutable
can be further simplified into multiple leaf mutables. In the current implementation, there are basically two kinds ofLabeledMutable
:MutableSymbol
. This is usually referred to as a “parameter”. They produce a key-value in the sample.MutableAnnotation
. They function as some kind of hint, and do not generate a key-value in the sample. Sometimes they can also be simplified and theMutableSymbol
they depend on would appear in the simplified result.
- class nni.mutable.MutableSymbol(label)[source]¶
MutableSymbol
corresponds to the concept of a variable / hyper-parameter / dimension.For example, a learning rate with a uniform distribution between 0.1 and 1, or a convolution filter that is either 32 or 64.
MutableSymbol
is a subclass ofSymbol
. Therefore they support arithmetic operations. The operation results will be aMutableExpression
object.See also
- class nni.mutable.MutableExpression(function, repr_template, arguments)[source]¶
Expression of mutables. Common use cases include: summation of several mutables, binary comparison between two mutables.
The expression is defined by a operator and a list of operands, which must be one or several
MutableSymbol
orMutableExpression
.The expression can be simplified into a dict of
LabeledMutable
. It can also be evaluated to be a concrete value (viafreeze()
), when the values it depends on have been given.
Categorical¶
- class nni.mutable.Categorical(values, *, weights=None, default='__missing__', label=None)[source]¶
Choosing one from a list of categorical values.
- Parameters:
values (Iterable[Choice]) – The list of values to choose from. There are no restrictions on value types. They can be integers, strings, and even dicts and lists. There is no intrinsic ordering of the values, meaning that the order in which the values appear in the list doesn’t matter. The values can also be an iterable, which will be expanded into a list.
weights (list[float] | None) – The probability distribution of the values. Should be an array with the same length as
values
. The sum of the distribution should be 1. If not specified, the values will be chosen uniformly.default (Choice | str) – Default value of the mutable. If not specified, the first value will be used.
label (str) – The label of the mutable. If not specified, a label will be auto-generated.
Examples
>>> x = Categorical([2, 3, 5], label='x1') >>> x.simplify() {'x1': Categorical([2, 3, 5], label='x1')} >>> x.freeze({'x1': 3}) 3
- default(memo=None)[source]¶
The default() of
Categorical
is the first value unless default value is set.See also
- grid(memo=None, granularity=None)[source]¶
Return also values as a grid. Sorted by distribution from most likely to least likely.
See also
CategoricalMultiple¶
- class nni.mutable.CategoricalMultiple(values, *, n_chosen=None, weights=None, default='__missing__', label=None)[source]¶
Choosing multiple from a list of values without replacement.
It’s implemented with a different class because for most algorithms, it’s very different from
Categorical
.CategoricalMultiple
can be either treated as a atomicLabeledMutable
(i.e., simple format), or be further simplified into a series of more fine-grained mutables (i.e., categorical format).In categorical format, class:CategoricalMultiple can be broken down to a list of
Categorical
of true and false, each indicating whether the choice on the corresponding position should be chosen. A constraint will be added ifn_chosen
is not None. This is useful for some algorithms that only support categorical mutables. Note that the prior distribution will be lost in this process.- Parameters:
values (Iterable[Choice]) – The list of values to choose from. See
Categorical
.n_chosen (int | None) – The number of values to choose. If not specified, any number of values can be chosen.
weights (list[float] | None) – The probability distribution of the values. Should be an array with the same length as
values
. Whenn_chosen
is None, it’s the probability that each candidate is chosen. Whenn_chosen
is not None, the distribution should sum to 1.default (list[Choice] | str) – Default value of the mutable. If not specified, the first
n_chosen
value will be used.label (str) – The label of the mutable. If not specified, a label will be auto-generated.
Examples
>>> x = CategoricalMultiple([2, 3, 5, 7], n_chosen=2, label='x2') >>> x.random() [2, 7] >>> x.simplify() {'x2': CategoricalMultiple([2, 3, 5, 7], n_chosen=2, label='x2')} >>> x.simplify(lambda t: not isinstance(t, CategoricalMultiple)) { 'x2/0': Categorical([True, False], label='x2/0'), 'x2/1': Categorical([True, False], label='x2/1'), 'x2/2': Categorical([True, False], label='x2/2'), 'x2/3': Categorical([True, False], label='x2/3'), 'x2/n': ExpressionConstraint(...) } >>> x.freeze({'x2': [3, 5]}) [3, 5] >>> x.freeze({'x2/0': True, 'x2/1': False, 'x2/2': True, 'x2/3': False}) [2, 5]
- default(memo=None)[source]¶
The first
n_chosen
values. Ifn_chosen
is None, return all values.See also
- grid(memo=None, granularity=None)[source]¶
Iterate over all possible values.
If
n_chosen
is None, iterate over all possible subsets, in the order of increasing length. Otherwise, iterate over all possible combinations ofn_chosen
length, using the implementation ofitertools.combinations()
.See also
- leaf_mutables(is_leaf)[source]¶
If invoking
is_leaf
returns true, return self. Otherwise, further break it down to severalCategorical
andConstraint
.See also
Numerical¶
- class nni.mutable.Numerical(low=-inf, high=inf, *, mu=None, sigma=None, log_distributed=False, quantize=None, distribution=None, default='__missing__', label=None)[source]¶
One variable from a univariate distribution.
It supports most commonly used distributions including uniform, loguniform, normal, lognormal, as well as the quantized version. It also supports using arbitrary distribution from
scipy.stats
.- Parameters:
low (float) – The lower bound of the domain. Used for uniform and loguniform. It will also be used to clip the value if it is outside the domain.
high (float) – The upper bound of the domain. Used for uniform and loguniform. It will also be used to clip the value if it is outside the domain.
mu (float | None) – The mean of the domain. Used for normal and lognormal.
sigma (float | None) – The standard deviation of the domain. Used for normal and lognormal.
log_distributed (bool) – Whether the domain is log distributed.
quantize (float | None) – If specified, the final value will be postprocessed with
clip(round(uniform(low, high) / q) * q, low, high)
, where the clip operation is used to constrain the generated value within the bounds. For example, when quantize is 2.5, all the values will be rounded to the nearest multiple of 2.5. Note that, iflow
orhigh
is not a multiple ofquantize
, it will be clipped tolow
orhigh
after rounding.distribution (_distn_infrastructure.rv_frozen | None) – The distribution to use. It should be a
rv_frozen
instance, which can be obtained by callingscipy.stats.distribution_name(...)
. If specified,low
,high
,mu
,sigma
,log_distributed
will be ignored.default (float | str) – The default value. If not specified, the default value will be the median of distribution.
label (str) – The label of the variable.
Examples
To create a variable uniformly sampled from 0 to 1:
Numerical(low=0, high=1)
To create a variable normally sampled with mean 2 and std 3:
Numerical(mu=2, sigma=3)
To create a normally sampled variable with mean 0 and std 1, but always in the range of [-1, 1] (note that it’s not truncated normal though):
Numerical(mu=0, sigma=1, low=-1, high=1)
To create a variable uniformly sampled from 0 to 100, but always multiple of 2:
Numerical(low=0, high=100, quantize=2)
To create a reciprocal continuous random variable in the range of [2, 6]:
Numerical(low=2, high=6, log_distributed=True)
To create a variable sampled from a custom distribution:
from scipy.stats import beta Numerical(distribution=beta(2, 5))
- default(memo=None)[source]¶
If default value is not specified,
Numerical.default()
returns median.See also
- equals(other)[source]¶
Checks whether two distributions are equal by examining the parameters.
See also
- grid(memo=None, granularity=None)[source]¶
Yield a list of samples within the distribution.
Since the grid of continuous space is infinite, we use granularity to specify the number of samples to yield. If granularity = 1, grid only explores median point of the distribution. If granularity = 2, the quartile points of the distribution will also be generated. Granularity = 3 explores the 1/8th points of the distribution, and so on. If not specified, granularity defaults to 1.
Grid will eliminate duplicates within the same granularity. Duplicates across different granularity will be ignored.
Examples
>>> list(Numerical(0, 1).grid(granularity=2)) [0.25, 0.5, 0.75] >>> list(Numerical(0, 1).grid(granularity=3)) [0.125, 0.25, 0.375, 0.5, 0.625, 0.75, 0.875] >>> list(Numerical(mu=0, sigma=1).grid(granularity=2)) [-0.6744897501960817, 0.0, 0.6744897501960817] >>> list(Numerical(mu=0, sigma=1, quantize=0.5).grid(granularity=3)) [-1.0, -0.5, 0.0, 0.5, 1.0]
See also
Containers¶
- nni.mutable.container¶
alias of <module ‘nni.mutable.container’ from ‘/home/docs/checkouts/readthedocs.org/user_builds/nni/checkouts/latest/nni/mutable/container.py’>
Annotation¶
- class nni.mutable.annotation.Constraint[source]¶
Constraints put extra requirements to make one sample valid.
For example, a constraint can be used to express that a variable should be larger than another variable, or certain combinations of variables should be strictly avoided.
Constraint
is a subclass ofMutableAnnotation
, and thus can be used as a normal mutable. It has a specialcontains()
method, which is used to check whether a sample satisfies the constraint. A constraint is satisfied if and only ifcontains()
returnsNone
.In general, users should inherit from
Constraint
to implement customized constraints.ExpressionConstraint
is a special constraint that can be used to express constraints in a more concise way.See also
- check_contains(sample)[source]¶
Override this to implement customized constraint. It should return
None
if the sample satisfies the constraint. Otherwise return aConstraintViolation
exception.See also
- grid(memo=None, granularity=None)[source]¶
Yield all samples that satisfy the constraint.
If some samples the constraint relies on have not been frozen yet, it will be sampled here and put into the memo. After that, it checks whether the sample satisfies the constraint after sampling (via
contains()
). If the sample doesn’t satisfy the constraint, it will be discarded.Each yielded sample of the
Constraint.grid()
itself is None, becauseConstraint.freeze()
also returns None.
- class nni.mutable.annotation.ExpressionConstraint(expression, *, label=None)[source]¶
A constraint that is expressed as
MutableExpression
.The expression must evaluate to be true to satisfy the constraint.
- Parameters:
expression (MutableExpression) – A
MutableExpression
that evaluates to a boolean value.label (str) – The semantic name of the constraint.
Examples
>>> a = Categorical([1, 3]) >>> b = Categorical([2, 4]) >>> list(MutableList([a, b, ExpressionConstraint(a + b == 5)]).grid()) [[1, 4, None], [3, 2, None]]
- class nni.mutable.annotation.MutableAnnotation[source]¶
Provide extra annotation / hints for a search space.
Sometimes, people who authored search strategies might want to recognize some hints from the search space. For example,
Adding some extra constraints between two parameters in the space.
Marking some choices as “nodes” in a cell.
Some parameter-combinations should be avoided or explored at the very first.
MutableAnnotation
is defined to be transparent, i.e., it doesn’t generate any new dimension by itself, and thus typically doesn’t introduce a new key in the sample received bynni.mutable.Mutable.freeze()
, but it affects the sampling the freezing process implicitly.This class is useful for isinstance check.
Frozen¶
- nni.mutable.frozen.ensure_frozen(mutable, *, strict=True, sample=None, retries=1000)[source]¶
Ensure a mutable is frozen. Used when passing the mutable to a function which doesn’t accept a mutable.
If the argument is not a mutable, nothing happens. Otherwise,
freeze()
will be called if sample is given. If sample is None,ensure_frozen()
will also try to fill the sample with the content infrozen_context
. Or elserobust_default()
will be called on the mutable.- Parameters:
mutable (nni.mutable.Mutable or any) – The mutable to freeze.
strict (bool) – Whether to raise an error if sample context is not provided and not found.
sample (Sample | None) – The context to freeze the mutable with.
retries (int) – Control the number of retries in case
robust_default()
is called.
Examples
>>> with frozen_context({'a': 2}): ... ensure_frozen(Categorical([1, 2, 3], label='a')) 2 >>> ensure_frozen(Categorical([1, 2, 3]), strict=False) 1 >>> ensure_frozen(Categorical([1, 2, 3], label='a'), sample={'a': 2}, strict=False) 2 >>> ensure_frozen('anything', strict=False) 'anything'
- class nni.mutable.frozen.frozen_context(sample=None)[source]¶
Context manager to set a sample into context. Then the sample will be retrievable from an arbitrary level of function calls via
current_frozen_context()
.There are two use cases:
Setting a global sample so that some modules can directly create the frozen version, rather than first-create-and-freeze.
Sharing default / dry-run samples when the search space is dynamically created.
The implementation is basically adding another layer of empty dict on top of a global stack. When retrieved, all dicts in the stack will be merged, from the bottom to the top. When updated, only the dict on the top will be updated.
- Parameters:
sample (Sample | None) – The sample to be set into context.
- Return type:
Context manager that provides a frozen context.
Examples
- ::
- def some_func():
print(frozen_context.current()[‘learning_rate’]) # 0.1
- with frozen_context({‘learning_rate’: 0.1}):
some_func()
- static bypass()[source]¶
Ignore the most recent
frozen_context
.This is useful in creating a search space within a
frozen_context()
context. Under the hood, it only disables the most recent one frozen context, which means, if it’s currently in a nested with-frozen-arch context, multiplebypass()
contexts is required.Examples
>>> with frozen_context(arch_dict): ... with frozen_context.bypass(): ... model_space = ModelSpace()
- class nni.mutable.frozen.frozen_factory(callable, sample)[source]¶
Create a factory object that invokes a function with a frozen context.
- Parameters:
callable (Callable[..., Any]) – The function to be invoked.
sample (Sample | frozen_context) – The sample to be used as the frozen context.
Examples
>>> factory = frozen_factory(ModelSpaceClass, {"choice1": 3}) >>> model = factory(channels=16, classes=10)
Exception¶
- exception nni.mutable.exception.ConstraintViolation(msg, paths=None)[source]¶
Exception raised when constraint is violated.
Symbol¶
Infrastructure for symbolic execution.
Symbolic execution is a technique for executing programs on symbolic inputs. It supports arithmetic operations and comparisons on abstract symbols, and can be used to represent symbolic expressions for potential evaluation and optimization.
The symbolic execution is implemented by overriding the operators of the Symbol
class.
The operators are implemented in a way that they can be chained together to form a SymbolicExpression
.
The symbolic execution is lazy,
which means that the expression will not be evaluated until the final value is substituted.
Examples
>>> from nni.mutable.symbol import Symbol
>>> x, y = Symbol('x'), Symbol('y')
>>> expr = x * x + y * 2
>>> print(expr)
(x * x) + (y * 2)
>>> list(expr.leaf_symbols())
[Symbol('x'), Symbol('x'), Symbol('y')]
>>> expr.evaluate({'x': 2, 'y': 3})
10
- class nni.mutable.symbol.Symbol(label)[source]¶
The leaf node of a symbolic expression. Each
Symbol
represents one variable in the expression.Variable with the same
label
share the same value.Operations on symbols (e.g.,
a + b
) will result in a newSymbolicExpression
.- Parameters:
label (str) – Each symbol is bound with a label, i.e., the variable name.
- class nni.mutable.symbol.SymbolicExpression(function, repr_template, arguments)[source]¶
Implementation of symbolic execution.
Each instance of
SymbolicExpression
is a node on the expression tree, with a function and a list of children (i.e., function arguments).The expression is designed to be compatible with native Python expressions. That means, the static methods (as well as operators) can be also applied on plain Python values.
- static case(pred_expr_pairs)[source]¶
Return the first expression with predicate that is true.
For example:
if (x < y) return 17; else if (x > z) return 23; else (y > z) return 31;
Equivalent to:
SymbolicExpression.case([(x < y, 17), (x > z, 23), (y > z, 31)])
Notes
This function performs lazy evaluation. Only the expression will be recorded when the function is called. The real evaluation happens when the inner value choice has determined its final decision. If no value choice is contained in the parameter list, the evaluation will be intermediate.
- static condition(pred, true, false)[source]¶
Return
true
if the predicatepred
is true elsefalse
.Examples
>>> SymbolicExpression.condition(Symbol('x') > Symbol('y'), 2, 1)
Notes
This function performs lazy evaluation. Only the expression will be recorded when the function is called. The real evaluation happens when the inner value choice has determined its final decision. If no value choice is contained in the parameter list, the evaluation will be intermediate.
- property expr_cls: Type[SymbolicExpression]¶
The created expression will be using this class.
- leaf_symbols()[source]¶
Return a generator of all leaf symbols.
Useful for when you want to inspect when the symbols come from. No deduplication even if the symbols has duplicates.
- static max(arg0, *args)[source]¶
Returns the maximum value from a list of symbols. The usage should be similar to Python’s built-in symbols, where the parameters could be an iterable, or at least two arguments.
Notes
This function performs lazy evaluation. Only the expression will be recorded when the function is called. The real evaluation happens when the inner value choice has determined its final decision. If no value choice is contained in the parameter list, the evaluation will be intermediate.
- static min(arg0, *args)[source]¶
Returns the minimum value from a list of symbols. The usage should be similar to Python’s built-in symbols, where the parameters could be an iterable, or at least two arguments.
Notes
This function performs lazy evaluation. Only the expression will be recorded when the function is called. The real evaluation happens when the inner value choice has determined its final decision. If no value choice is contained in the parameter list, the evaluation will be intermediate.
- static switch_case(branch, expressions)[source]¶
Select the expression that matches the branch.
C-style switch:
switch (branch) { // c-style switch case 0: return 17; case 1: return 31; }
Equivalent to:
SymbolicExpression.switch_case(branch, {0: 17, 1: 31})
Notes
This function performs lazy evaluation. Only the expression will be recorded when the function is called. The real evaluation happens when the inner value choice has determined its final decision. If no value choice is contained in the parameter list, the evaluation will be intermediate.
- static to_float(obj)[source]¶
Convert the current value to a float. .. rubric:: Notes
This function performs lazy evaluation. Only the expression will be recorded when the function is called. The real evaluation happens when the inner value choice has determined its final decision. If no value choice is contained in the parameter list, the evaluation will be intermediate.
- static to_int(obj)[source]¶
Convert the current value to an integer. .. rubric:: Notes
This function performs lazy evaluation. Only the expression will be recorded when the function is called. The real evaluation happens when the inner value choice has determined its final decision. If no value choice is contained in the parameter list, the evaluation will be intermediate.
Utilities¶
- class nni.mutable.utils.ContextStack(key, value)[source]¶
This is to maintain a globally-accessible context environment that is visible to everywhere.
To initiate:
with ContextStack(namespace, value): ...
Inside the context, you can access the nearest value put into
with
:get_current_context(namespace)
Notes
ContextStack
is not multi-processing safe. Also, the values will get cleared for a new process.
- nni.mutable.utils.auto_label(name=None, scope=None)[source]¶
Automatically generate a formatted and reproducible label.
In case
name
is not set, the label name will use the uid sequence ofscope
. Ifscope
is not set, it will fetch the nearest scope. If no scope is found, it will use the global scope (label_scope.global_()
).If the scope is found and is not global scope, the scope’s full name will be prepended to the label, i.e., the label will then be formatted as
{scope}/{label}
. Otherwise, there are two cases. Firstly, if label is manually specified, it will be returned directly. Secondly, we rely on the global scope to generate the label name, and the scope name will still be prepended. The rules can be better explained with the examples below.Notes
We always recommend specifying the label manually.
- Parameters:
name (str | label | None) – The label name to use. If not specified, it will be generated by the scope.
scope (label_scope | None) – The scope to use. If not specified, the nearest scope will be used.
- Returns:
A string that represents the label.
For advanced users, please be advised that
the returned object would be a
label
object, rather than a string.This is to make sure
auto_label()
is idempotent.We expect some side effects, but it should work seamlessly with most of the code.
- Return type:
str
Examples
>>> label1 = auto_label('bar') # bar, because the scope is global >>> label2 = auto_label() # global/1, because label is not provided >>> with label_scope('foo'): ... label3 = auto_label() # foo/1, because in the scope "foo" >>> with label_scope(): # scope is global/2 ... label4 = auto_label() # global/2/1 >>> with label_scope('another'): ... label5 = auto_label() # another/1 ... label6 = auto_label('thing') # another/thing ... label7 = auto_label() # another/2
- class nni.mutable.utils.label_scope(basename=None, *, _path=None)[source]¶
To support automatic labeling of mutables.
Labels are named like a file-system. The analogy here is that: scope is like a directory, and label is like a file. The label name is like a file name. It can’t contain slash (
/
) or underscore (_
). The scope name is like a directory name. It also can’t contain/
or_
. When we refer to a “label”, we will usually use the full name, which is like an absolute file path.label_scope
is usually jointly used withauto_label()
, wherelabel_scope
is used to generate the “scope” (directory) part, andauto_label()
is used to generate the “name” (file) part. Alabel_scope
can be entered, and thenauto_label()
can be called inside. The labels as well as scopes generated inside can be automatically named with natural integers starting from 1 (see examples below), and we guarantee the generation of labels to be reproducible. It can also be naturally nested.label_scope
is NOT thread-safe. The behavior is undefined if multiple threads are trying to enter the scope at the same time.label_scope
is implemented based onContextStack
.- Parameters:
basename (str | label | label_scope | None) – The last part of current scope name. If not specified, it will be generated by the parent scope. If the parent scope is not found, the scope name will be
param
by default.label_scope
is idempotent, sobasename
also acceptslabel_scope
andlabel
, though it will return a new instance.
Examples
>>> with label_scope('model'): ... label1 = auto_label() # model/1 ... label2 = auto_label() # model/2 ... label3 = auto_label('foo') # model/foo ... with label_scope(): ... label4 = auto_label() # model/3/1 ... label5 = auto_label() # model/3/2 ... with label_scope('another'): ... label6 = auto_label() # model/another/1 ... with label_scope('model'): ... label7 = auto_label() # model/model/1 >>> with label_scope('model'): ... label8 = auto_label() # model/1, because the counter is reset >>> with label_scope(): ... label9 = auto_label() # global/1/1
- property absolute_scope: str¶
Alias of name.
- static current()[source]¶
Fetch the nearest label scope activated by
with
.If label scope is never used, or we are currently within no with-block, return none.
Examples
>>> with label_scope() as scope1: ... # somewhere in the middle of the code. ... label_scope.current() # Return scope1
- static global_()[source]¶
Fetch the global label scope.
This label scope can be created on-the-fly and can live without the with-blocks.
- property name: str¶
The full name of current namespace.
For example,
model/cell/2
.