NNI Client¶
NNI client is a python API of nnictl
, which implements the most commonly used commands. Users can use this API to control their experiments, collect experiment results and conduct advanced analyses based on experiment results in python code directly instead of using command line. Here is an example:
from nnicli import Experiment
# create an experiment instance
exp = Experiment()
# start an experiment, then connect the instance to this experiment
# you can also use `resume_experiment`, `view_experiment` or `connect_experiment`
# only one of them should be called in one instance
exp.start_experiment('nni/examples/trials/mnist-pytorch/config.yml', port=9090)
# update the experiment's concurrency
exp.update_concurrency(3)
# get some information about the experiment
print(exp.get_experiment_status())
print(exp.get_job_statistics())
print(exp.list_trial_jobs())
# stop the experiment, then disconnect the instance from the experiment.
exp.stop_experiment()
References¶
-
class
nnicli.
Experiment
[source]¶ -
connect_experiment
(endpoint)[source]¶ Connect to an existing experiment.
Parameters: endpoint (str) – The endpoint of nni rest server, i.e, the url of Web UI. Should be a format like http://ip:port.
-
export_data
()[source]¶ Return exported information for all trial jobs.
Returns: List of nnicli.TrialResult. Return type: list
-
get_experiment_profile
()[source]¶ Return experiment profile as a dict.
Returns: The profile of the experiment. Return type: dict
-
get_experiment_status
()[source]¶ Return experiment status as a dict.
Returns: Experiment status. Return type: dict
-
get_job_metrics
(trial_job_id=None)[source]¶ Return trial job metrics.
Parameters: trial_job_id (str) – trial job id. if this parameter is None, all trail jobs’ metrics will be returned. Returns: Each key is a trialJobId, the corresponding value is a list of nnicli.TrialMetricData. Return type: dict
-
get_job_statistics
()[source]¶ Return trial job statistics information as a dict.
Returns: Job statistics information. Return type: list
-
get_trial_job
(trial_job_id)[source]¶ Return a trial job.
Parameters: trial_job_id (str) – Trial job id. Returns: A nnicli.TrialJob instance corresponding to trial_job_id. Return type: nnicli.TrialJob
-
list_trial_jobs
()[source]¶ Return information for all trial jobs as a list.
Returns: List of nnicli.TrialJob. Return type: list
-
resume_experiment
(exp_id, port=None, debug=False)[source]¶ Resume a stopped experiment with specified experiment id
Parameters: - exp_id (str) – Experiment id.
- port (int) – The port of restful server, bigger than 1024.
- debug (boolean) – Set debug mode.
-
start_experiment
(config_file, port=None, debug=False)[source]¶ Start an experiment with specified configuration file and connect to it.
Parameters: - config_file (str) – Path to the config file.
- port (int) – The port of restful server, bigger than 1024.
- debug (boolean) – Set debug mode.
-
update_concurrency
(value)[source]¶ Update an experiment’s concurrency
Parameters: value (int) – New concurrency value.
-
update_duration
(value)[source]¶ Update an experiment’s duration
Parameters: value (str) – Strings like ‘1m’ for one minute or ‘2h’ for two hours. SUFFIX may be ‘s’ for seconds, ‘m’ for minutes, ‘h’ for hours or ‘d’ for days.
-
update_searchspace
(filename)[source]¶ Update the experiment’s search space.
Parameters: filename (str) – Path to the searchspace file.
-
-
class
nnicli.
TrialJob
(json_obj)[source]¶ TrialJob stores the information of a trial job.
Parameters: json_obj (dict) – json object that stores the hyper parameters -
trialJobId
¶ Trial job id.
Type: str
-
status
¶ Job status.
Type: str
-
hyperParameters
¶ See nnicli.TrialHyperParameters.
Type: list of nnicli.TrialHyperParameters
-
logPath
¶ Log path.
Type: str
-
startTime
¶ Job start time (timestamp).
Type: int
-
endTime
¶ Job end time (timestamp).
Type: int
-
finalMetricData
¶ See nnicli.TrialMetricData.
Type: list of nnicli.TrialMetricData
-
parameter_index
¶ Parameter index.
Type: int
-
-
class
nnicli.
TrialHyperParameters
(json_obj)[source]¶ TrialHyperParameters stores the hyper parameters of a trial job.
Parameters: json_obj (dict) – Json object that stores the hyper parameters. -
parameter_id
¶ Parameter id.
Type: int
-
parameter_source
¶ Parameter source.
Type: str
-
parameters
¶ Hyper parameters.
Type: dict
-
parameter_index
¶ Parameter index.
Type: int
-
-
class
nnicli.
TrialMetricData
(json_obj)[source]¶ TrialMetricData stores the metric data of a trial job. A trial job may have both intermediate metric and final metric.
Parameters: json_obj (dict) – Json object that stores the metric data. -
timestamp
¶ Time stamp.
Type: int
-
trialJobId
¶ Trial job id.
Type: str
-
parameterId
¶ Parameter id.
Type: int
-
type
¶ Metric type, PERIODICAL for intermediate result and FINAL for final result.
Type: str
-
sequence
¶ Sequence number in this trial.
Type: int
-
data
¶ Metric data.
Type: serializable object, usually a number, or a dict with key “default” and other extra keys
-
-
class
nnicli.
TrialResult
(json_obj)[source]¶ TrialResult stores the result information of a trial job.
Parameters: json_obj (dict) – Json object that stores the result information. -
parameter
¶ Hyper parameters for this trial.
Type: dict
-
value
¶ Final result.
Type: serializable object, usually a number, or a dict with key “default” and other extra keys
-
trialJobId
¶ Trial job id.
Type: str
-