Start and Manage a New Experiment

Configure Search Space

search_space = {
    "C": {"_type": "quniform", "_value": [0.1, 1, 0.1]},
    "kernel": {"_type": "choice", "_value": ["linear", "rbf", "poly", "sigmoid"]},
    "degree": {"_type": "choice", "_value": [1, 2, 3, 4]},
    "gamma": {"_type": "quniform", "_value": [0.01, 0.1, 0.01]},
    "coef0": {"_type": "quniform", "_value": [0.01, 0.1, 0.01]}
}

Configure Experiment

from nni.experiment import Experiment
experiment = Experiment('local')
experiment.config.experiment_name = 'Example'
experiment.config.trial_concurrency = 2
experiment.config.max_trial_number = 10
experiment.config.search_space = search_space
experiment.config.trial_command = 'python scripts/trial_sklearn.py'
experiment.config.trial_code_directory = './'
experiment.config.tuner.name = 'TPE'
experiment.config.tuner.class_args['optimize_mode'] = 'maximize'
experiment.config.training_service.use_active_gpu = True

Start Experiment

experiment.start(8080)

Out:

[2022-02-07 18:56:04] Creating experiment, Experiment ID: fl9vu67z
[2022-02-07 18:56:04] Starting web server...
[2022-02-07 18:56:05] Setting up...
[2022-02-07 18:56:05] Web UI URLs: http://127.0.0.1:8080 http://10.190.173.211:8080 http://172.17.0.1:8080 http://192.168.49.1:8080

Experiment View & Control

View the status of experiment.

experiment.get_status()

Out:

'RUNNING'

Wait until at least one trial finishes.

import time

for _ in range(10):
    stats = experiment.get_job_statistics()
    if any(stat['trialJobStatus'] == 'SUCCEEDED' for stat in stats):
        break
    time.sleep(10)

Export the experiment data.

experiment.export_data()

Out:

[TrialResult(parameter={'C': 0.9, 'kernel': 'rbf', 'degree': 4, 'gamma': 0.07, 'coef0': 0.03}, value=0.9733333333333334, trialJobId='dNOZt'), TrialResult(parameter={'C': 0.8, 'kernel': 'sigmoid', 'degree': 2, 'gamma': 0.01, 'coef0': 0.01}, value=0.9733333333333334, trialJobId='okYSD')]

Get metric of jobs

experiment.get_job_metrics()

Out:

{'okYSD': [TrialMetricData(timestamp=1644227777089, trialJobId='okYSD', parameterId='1', type='FINAL', sequence=0, data=0.9733333333333334)], 'dNOZt': [TrialMetricData(timestamp=1644227777357, trialJobId='dNOZt', parameterId='0', type='FINAL', sequence=0, data=0.9733333333333334)]}

Stop Experiment

experiment.stop()

Out:

[2022-02-07 18:56:25] Stopping experiment, please wait...
[2022-02-07 18:56:28] Experiment stopped