azuremlsdk (version 1.10.0)

hyperdrive_config: Create a configuration for a HyperDrive run

Description

The HyperDrive configuration includes information about hyperparameter space sampling, termination policy, primary metric, estimator, and the compute target to execute the experiment runs on.

To submit the HyperDrive experiment, pass the HyperDriveConfig object returned from this method to submit_experiment().

Usage

hyperdrive_config(
  hyperparameter_sampling,
  primary_metric_name,
  primary_metric_goal,
  max_total_runs,
  max_concurrent_runs = NULL,
  max_duration_minutes = 10080L,
  policy = NULL,
  estimator = NULL
)

Arguments

hyperparameter_sampling

The hyperparameter sampling space. Can be a RandomParameterSampling, GridParameterSampling, or BayesianParameterSampling object.

primary_metric_name

A string of the name of the primary metric reported by the experiment runs.

primary_metric_goal

The PrimaryMetricGoal object. This parameter determines if the primary metric is to be minimized or maximized when evaluating runs.

max_total_runs

An integer of the maximum total number of runs to create. This is the upper bound; there may be fewer runs when the sample space is smaller than this value. If both max_total_runs and max_duration_minutes are specified, the hyperparameter tuning experiment terminates when the first of these two thresholds is reached.

max_concurrent_runs

An integer of the maximum number of runs to execute concurrently. If NULL, all runs are launched in parallel. The number of concurrent runs is gated on the resources available in the specified compute target. Hence, you need to ensure that the compute target has the available resources for the desired concurrency.

max_duration_minutes

An integer of the maximum duration of the HyperDrive run. Once this time is exceeded, any runs still executing are cancelled. If both max_total_runs and max_duration_minutes are specified, the hyperparameter tuning experiment terminates when the first of these two thresholds is reached.

policy

The early termination policy to use. Can be either a BanditPolicy, MedianStoppingPolicy, or TruncationSelectionPolicy object. If NULL (the default), no early termination policy will be used.

The MedianStoppingPolicy with delay_evaluation of = 5 is a good termination policy to start with. These are conservative settings that can provide 25%-35% savings with no loss on primary metric (based on our evaluation data).

estimator

The Estimator object.

Value

The HyperDriveConfig object.

See Also

submit_experiment()

Examples

# NOT RUN {
# Load the workspace
ws <- load_workspace_from_config()

# Get the compute target
compute_target <- get_compute(ws, cluster_name = 'mycluster')

# Define the primary metric goal
goal = primary_metric_goal("MAXIMIZE")

# Define the early termination policy
early_termination_policy = median_stopping_policy(evaluation_interval = 1L,
                                                  delay_evaluation = 5L)

# Create the estimator
est <- estimator(source_directory = '.',
                 entry_script = 'train.R',
                 compute_target = compute_target)

# Create the HyperDrive configuration
hyperdrive_run_config = hyperdrive_config(
                                   hyperparameter_sampling = param_sampling,
                                   primary_metric_name = 'accuracy',
                                   primary_metric_goal = goal,
                                   max_total_runs = 100,
                                   max_concurrent_runs = 4,
                                   policy = early_termination_policy,
                                   estimator = est)

# Submit the HyperDrive experiment
exp <- experiment(ws, name = 'myexperiment')
run = submit_experiment(exp, hyperdrive_run_config)
# }