NNI Client¶

NNI client is a python API of nnictl, which implements the most commonly used commands. Users can use this API to control their experiments, collect experiment results and conduct advanced analyses based on experiment results in python code directly instead of using command line. Here is an example:

from nnicli import Experiment

# create an experiment instance
exp = Experiment() 

# start an experiment, then connect the instance to this experiment
# you can also use `resume_experiment`, `view_experiment` or `connect_experiment`
# only one of them should be called in one instance
exp.start_experiment('nni/examples/trials/mnist-pytorch/config.yml', port=9090)

# update the experiment's concurrency
exp.update_concurrency(3)

# get some information about the experiment
print(exp.get_experiment_status())
print(exp.get_job_statistics())
print(exp.list_trial_jobs())

# stop the experiment, then disconnect the instance from the experiment.
exp.stop_experiment()

References¶

class nnicli.Experiment[source]¶

connect_experiment(endpoint)[source]¶

Connect to an existing experiment.

Parameters:	endpoint (str) – The endpoint of nni rest server, i.e, the url of Web UI. Should be a format like http://ip:port.

export_data()[source]¶

Return exported information for all trial jobs.

Returns:	List of nnicli.TrialResult.
Return type:	list

get_experiment_profile()[source]¶

Return experiment profile as a dict.

Returns:	The profile of the experiment.
Return type:	dict

get_experiment_status()[source]¶

Return experiment status as a dict.

Returns:	Experiment status.
Return type:	dict

get_job_metrics(trial_job_id=None)[source]¶

Return trial job metrics.

Parameters:	trial_job_id (str) – trial job id. if this parameter is None, all trail jobs’ metrics will be returned.
Returns:	Each key is a trialJobId, the corresponding value is a list of nnicli.TrialMetricData.
Return type:	dict

get_job_statistics()[source]¶

Return trial job statistics information as a dict.

Returns:	Job statistics information.
Return type:	list

get_trial_job(trial_job_id)[source]¶

Return a trial job.

Parameters:	trial_job_id (str) – Trial job id.
Returns:	A nnicli.TrialJob instance corresponding to trial_job_id.
Return type:	nnicli.TrialJob

list_trial_jobs()[source]¶

Return information for all trial jobs as a list.

Returns:	List of nnicli.TrialJob.
Return type:	list

resume_experiment(exp_id, port=None, debug=False)[source]¶

Resume a stopped experiment with specified experiment id

Parameters:	exp_id (str) – Experiment id. port (int) – The port of restful server, bigger than 1024. debug (boolean) – Set debug mode.

start_experiment(config_file, port=None, debug=False)[source]¶

Start an experiment with specified configuration file and connect to it.

Parameters:	config_file (str) – Path to the config file. port (int) – The port of restful server, bigger than 1024. debug (boolean) – Set debug mode.

stop_experiment()[source]¶: Stop the experiment.

update_concurrency(value)[source]¶

Update an experiment’s concurrency

Parameters:	value (int) – New concurrency value.

update_duration(value)[source]¶

Update an experiment’s duration

Parameters:	value (str) – Strings like ‘1m’ for one minute or ‘2h’ for two hours. SUFFIX may be ‘s’ for seconds, ‘m’ for minutes, ‘h’ for hours or ‘d’ for days.

update_searchspace(filename)[source]¶

Update the experiment’s search space.

Parameters:	filename (str) – Path to the searchspace file.

update_trailnum(value)[source]¶

Update an experiment’s maxtrialnum

Parameters:	value (int) – New trailnum value.

view_experiment(exp_id, port=None)[source]¶

View a stopped experiment with specified experiment id.

Parameters:	exp_id (str) – Experiment id. port (int) – The port of restful server, bigger than 1024.

class nnicli.TrialJob(json_obj)[source]¶

TrialJob stores the information of a trial job.

Parameters:	json_obj (dict) – json object that stores the hyper parameters

trialJobId¶

Trial job id.

Type:	str

status¶

Job status.

Type:	str

hyperParameters¶

See nnicli.TrialHyperParameters.

Type:	list of nnicli.TrialHyperParameters

logPath¶

Log path.

Type:	str

startTime¶

Job start time (timestamp).

Type:	int

endTime¶

Job end time (timestamp).

Type:	int

finalMetricData¶

See nnicli.TrialMetricData.

Type:	list of nnicli.TrialMetricData

parameter_index¶

Parameter index.

Type:	int

class nnicli.TrialHyperParameters(json_obj)[source]¶

TrialHyperParameters stores the hyper parameters of a trial job.

Parameters:	json_obj (dict) – Json object that stores the hyper parameters.

parameter_id¶

Parameter id.

Type:	int

parameter_source¶

Parameter source.

Type:	str

parameters¶

Hyper parameters.

Type:	dict

parameter_index¶

Parameter index.

Type:	int

class nnicli.TrialMetricData(json_obj)[source]¶

TrialMetricData stores the metric data of a trial job. A trial job may have both intermediate metric and final metric.

Parameters:	json_obj (dict) – Json object that stores the metric data.

timestamp¶

Time stamp.

Type:	int

trialJobId¶

Trial job id.

Type:	str

parameterId¶

Parameter id.

Type:	int

type¶

Metric type, PERIODICAL for intermediate result and FINAL for final result.

Type:	str

sequence¶

Sequence number in this trial.

Type:	int

data¶

Metric data.

Type:	serializable object, usually a number, or a dict with key “default” and other extra keys

class nnicli.TrialResult(json_obj)[source]¶

TrialResult stores the result information of a trial job.

Parameters:	json_obj (dict) – Json object that stores the result information.

parameter¶

Hyper parameters for this trial.

Type:	dict

value¶

Final result.

Type:	serializable object, usually a number, or a dict with key “default” and other extra keys

trialJobId¶

Trial job id.

Type:	str