Setting up the Optimization


class sherpa.core.Continuous(name, range, scale='linear')[source]

Continuous parameter class.

class sherpa.core.Discrete(name, range, scale='linear')[source]

Discrete parameter class.

class sherpa.core.Choice(name, range)[source]

Choice parameter class.

class sherpa.core.Ordinal(name, range)[source]

Ordinal parameter class. Categorical, ordered variable.

class sherpa.core.Parameter(name, range)[source]

Defines a hyperparameter with a name, type and associated range.

  • name (str) – the parameter name.
  • range (list) – either [low, high] or [value1, value2, value3].
  • scale (str) – linear or log, defines sampling from linear or log-scale. Not defined for all parameter types.
static from_dict(config)[source]

Returns a parameter object according to the given dictionary config.

Parameters:config (dict) – parameter config.


{'name': '<name>',
 'type': '<continuous/discrete/choice>',
 'range': [<value1>, <value2>, ... ],
 'scale': <'log' to sample continuous/discrete from log-scale>}
Returns:the parameter range object.
Return type:sherpa.core.Parameter
static grid(parameter_grid)[source]

Creates a list of parameters given a parameter grid.

Parameters:parameter_grid (dict) – Dictionary mapping hyperparameter names lists of possible values.


{'parameter_a': [aValue1, aValue2, ...],
 'parameter_b': [bValue1, bValue2, ...],
Returns:list of parameter ranges for SHERPA.
Return type:list[sherpa.core.Parameter]


class sherpa.core.Study(parameters, algorithm, lower_is_better, stopping_rule=None, dashboard_port=None, disable_dashboard=False, output_dir=None)[source]

The core of an optimization.

Includes functionality to get new suggested trials and add observations for those. Used internally but can also be used directly by the user.

  • parameters (list[sherpa.core.Parameter]) – a list of parameter ranges.
  • algorithm (sherpa.algorithms.Algorithm) – the optimization algorithm.
  • lower_is_better (bool) – whether to minimize or maximize the objective.
  • stopping_rule (sherpa.algorithms.StoppingRule) – algorithm to stop badly performing trials.
  • dashboard_port (int) – the port for the dashboard web-server, if None the first free port in the range 8880 to 9999 is found and used.
  • disable_dashboard (bool) – option to not run the dashboard.
  • output_dir (str) – directory path for CSV results.
  • random_seed (int) – seed to use for NumPy random number generators throughout.
add_observation(trial, objective, iteration=1, context={})[source]

Add a single observation of the objective value for a given trial.

  • trial (sherpa.core.Trial) – trial for which an observation is to be added.
  • iteration (int) – iteration number e.g. epoch.
  • objective (float) – objective value.
  • context (dict) – other metrics or values to record.

Adds a trial into queue for next suggestion.

Trials added via this method forego the suggestions made by the algorithm and are returned by the get_suggestion method on a first in first out basis.

Parameters:trial (sherpa.core.Trial) – the trial to be enqueued.
finalize(trial, status='COMPLETED')[source]

Once a trial will not add any more observations it must be finalized with this function.

  • trial (sherpa.core.Trial) – trial that is completed.
  • status (str) – one of ‘COMPLETED’, ‘FAILED’, ‘STOPPED’.

Retrieve the best result so far.

Returns:row of the best result.
Return type:pandas.DataFrame

Obtain a new suggested trial.

This function wraps the algorithm that was passed to the study.

Returns:a parameter suggestion.
Return type:dict
keras_callback(trial, objective_name, context_names=[])[source]

Keras Callbacks to add observations to study

  • trial (sherpa.core.Trial) – trial to send metrics for.
  • objective_name (str) – the name of the objective e.g. loss, val_loss, or any of the submitted metrics.
  • context_names (list[str]) – names of all other metrics to be monitored.
static load_dashboard(path)[source]

Loads a study from an output dir without the algorithm.

Parameters:path (str) – the path to the output dir.
the study running the dashboard, note that
currently this study cannot be used to continue the optimization.
Return type:sherpa.core.Study

Stores results to CSV and attributes to config file.

Parameters:output_dir (str) – directory to store CSV to, only needed if Study output_dir is not defined.

Determines whether given trial should stop.

This function wraps the stopping rule provided to the study.

Parameters:trial (sherpa.core.Trial) – trial to be evaluated.
Return type:bool

Running the Optimization in Parallel

sherpa.core.optimize(parameters, algorithm, lower_is_better, scheduler, command=None, filename=None, output_dir='./output_20201018-075927', max_concurrent=1, db_port=None, stopping_rule=None, dashboard_port=None, resubmit_failed_trials=False, verbose=1, load=False, mongodb_args={}, disable_dashboard=False)[source]

Runs a Study with a scheduler and automatically runs a database in the background.

  • algorithm (sherpa.algorithms.Algorithm) – takes results table and returns parameter set.
  • parameters (list[sherpa.core.Parameter]) – parameters being optimized.
  • lower_is_better (bool) – whether lower objective values are better.
  • command (str) – the command to run for the trial script.
  • filename (str) – the filename of the script to run. Will be run as “python <filename>”.
  • output_dir (str) – where scheduler and database files will be stored.
  • scheduler (sherpa.schedulers.Scheduler) – a scheduler.
  • max_concurrent (int) – the number of trials that will be evaluated in parallel.
  • db_port (int) – port to run the database on.
  • stopping_rule (sherpa.algorithms.StoppingRule) – rule for stopping trials prematurely.
  • dashboard_port (int) – port to run the dashboard web-server on.
  • resubmit_failed_trials (bool) – whether to resubmit a trial if it failed.
  • verbose (int, default=1) – whether to print submit messages (0=no, 1=yes).
  • load (bool) – option to load study, currently not fully implemented.
  • mongodb_args (dict[str, any]) – arguments to MongoDB beyond port, dir, and log-path. Keys are the argument name without “–”.

Setting up the Trial


class sherpa.database.Client(host=None, port=None, test_mode=False, **mongo_client_args)[source]

Registers a session with a Sherpa Study via the port of the database.

This function is called from trial-scripts only.

  • host (str) – the host that runs the database. Passed host, host set via environment variable or ‘localhost’ in that order.
  • port (int) – port that database is running on. Passed port, port set via environment variable or 27010 in that order.

Returns the next trial from a Sherpa Study.

Returns:The trial to run.
Return type:sherpa.core.Trial
keras_send_metrics(trial, objective_name, context_names=[])[source]

Keras Callbacks to send metrics to SHERPA.

  • trial (sherpa.core.Trial) – trial to send metrics for.
  • objective_name (str) – the name of the objective e.g. loss, val_loss, or any of the submitted metrics.
  • context_names (list[str]) – names of all other metrics to be monitored.
send_metrics(trial, iteration, objective, context={})[source]

Sends metrics for a trial to database.

  • trial (sherpa.core.Trial) – trial to send metrics for.
  • iteration (int) – the iteration e.g. epoch the metrics are for.
  • objective (float) – the objective value.
  • context (dict) – other metric-values.