Tune Package Reference

ray.tune

Convenience method for specifying grid search over a value.

Parameters:values – An iterable whose parameters will be gridded.
ray.tune.register_env(name, env_creator)[source]

Register a custom environment for use with RLlib.

Parameters:
  • name (str) – Name to register.
  • env_creator (obj) – Function that creates an env.
ray.tune.register_trainable(name, trainable)[source]

Register a trainable function or class.

Parameters:
  • name (str) – Name to register.
  • trainable (obj) – Function or tune.Trainable class. Functions must take (config, status_reporter) as arguments and will be automatically converted into a class during registration.
ray.tune.run(run_or_experiment, name=None, stop=None, config=None, resources_per_trial=None, num_samples=1, local_dir=None, upload_dir=None, trial_name_creator=None, loggers=None, sync_to_cloud=None, sync_to_driver=None, checkpoint_freq=0, checkpoint_at_end=False, keep_checkpoints_num=None, checkpoint_score_attr=None, global_checkpoint_period=10, export_formats=None, max_failures=3, restore=None, search_alg=None, scheduler=None, with_server=False, server_port=4321, verbose=2, resume=False, queue_trials=False, reuse_actors=False, trial_executor=None, raise_on_failed_trial=True, return_trials=True, ray_auto_init=True, sync_function=None)[source]

Executes training.

Parameters:
  • run_or_experiment (function|class|str|Experiment) – If function|class|str, this is the algorithm or model to train. This may refer to the name of a built-on algorithm (e.g. RLLib’s DQN or PPO), a user-defined trainable function or class, or the string identifier of a trainable function or class registered in the tune registry. If Experiment, then Tune will execute training based on Experiment.spec.
  • name (str) – Name of experiment.
  • stop (dict) – The stopping criteria. The keys may be any field in the return result of ‘train()’, whichever is reached first. Defaults to empty dict.
  • config (dict) – Algorithm-specific configuration for Tune variant generation (e.g. env, hyperparams). Defaults to empty dict. Custom search algorithms may ignore this.
  • resources_per_trial (dict) – Machine resources to allocate per trial, e.g. {"cpu": 64, "gpu": 8}. Note that GPUs will not be assigned unless you specify them here. Defaults to 1 CPU and 0 GPUs in Trainable.default_resource_request().
  • num_samples (int) – Number of times to sample from the hyperparameter space. Defaults to 1. If grid_search is provided as an argument, the grid will be repeated num_samples of times.
  • local_dir (str) – Local dir to save training results to. Defaults to ~/ray_results.
  • upload_dir (str) – Optional URI to sync training results to (e.g. s3://bucket).
  • trial_name_creator (func) – Optional function for generating the trial string representation.
  • loggers (list) – List of logger creators to be used with each Trial. If None, defaults to ray.tune.logger.DEFAULT_LOGGERS. See ray/tune/logger.py.
  • sync_to_cloud (func|str) – Function for syncing the local_dir to and from upload_dir. If string, then it must be a string template that includes {source} and {target} for the syncer to run. If not provided, the sync command defaults to standard S3 or gsutil sync comamnds.
  • sync_to_driver (func|str) – Function for syncing trial logdir from remote node to local. If string, then it must be a string template that includes {source} and {target} for the syncer to run. If not provided, defaults to using rsync.
  • checkpoint_freq (int) – How many training iterations between checkpoints. A value of 0 (default) disables checkpointing.
  • checkpoint_at_end (bool) – Whether to checkpoint at the end of the experiment regardless of the checkpoint_freq. Default is False.
  • keep_checkpoints_num (int) – Number of checkpoints to keep. A value of None keeps all checkpoints. Defaults to None. If set, need to provide checkpoint_score_attr.
  • checkpoint_score_attr (str) – Specifies by which attribute to rank the best checkpoint. Default is increasing order. If attribute starts with min- it will rank attribute in decreasing order, i.e. min-validation_loss.
  • global_checkpoint_period (int) – Seconds between global checkpointing. This does not affect checkpoint_freq, which specifies frequency for individual trials.
  • export_formats (list) – List of formats that exported at the end of the experiment. Default is None.
  • max_failures (int) – Try to recover a trial from its last checkpoint at least this many times. Only applies if checkpointing is enabled. Setting to -1 will lead to infinite recovery retries. Defaults to 3.
  • restore (str) – Path to checkpoint. Only makes sense to set if running 1 trial. Defaults to None.
  • search_alg (SearchAlgorithm) – Search Algorithm. Defaults to BasicVariantGenerator.
  • scheduler (TrialScheduler) – Scheduler for executing the experiment. Choose among FIFO (default), MedianStopping, AsyncHyperBand, and HyperBand.
  • with_server (bool) – Starts a background Tune server. Needed for using the Client API.
  • server_port (int) – Port number for launching TuneServer.
  • verbose (int) – 0, 1, or 2. Verbosity mode. 0 = silent, 1 = only status updates, 2 = status and trial results.
  • resume (str|bool) – One of “LOCAL”, “REMOTE”, “PROMPT”, or bool. LOCAL/True restores the checkpoint from the local_checkpoint_dir. REMOTE restores the checkpoint from remote_checkpoint_dir. PROMPT provides CLI feedback. False forces a new experiment. If resume is set but checkpoint does not exist, ValueError will be thrown.
  • queue_trials (bool) – Whether to queue trials when the cluster does not currently have enough resources to launch one. This should be set to True when running on an autoscaling cluster to enable automatic scale-up.
  • reuse_actors (bool) – Whether to reuse actors between different trials when possible. This can drastically speed up experiments that start and stop actors often (e.g., PBT in time-multiplexing mode). This requires trials to have the same resource requirements.
  • trial_executor (TrialExecutor) – Manage the execution of trials.
  • raise_on_failed_trial (bool) – Raise TuneError if there exists failed trial (of ERROR state) when the experiments complete.
  • ray_auto_init (bool) – Automatically starts a local Ray cluster if using a RayTrialExecutor (which is the default) and if Ray is not initialized. Defaults to True.
  • sync_function – Deprecated. See sync_to_cloud and sync_to_driver.
Returns:

List of Trial objects.

Raises:

TuneError if any trials failed and raise_on_failed_trial is True.

Examples

>>> tune.run(mytrainable, scheduler=PopulationBasedTraining())
>>> tune.run(mytrainable, num_samples=5, reuse_actors=True)
>>> tune.run(
        "PG",
        num_samples=5,
        config={
            "env": "CartPole-v0",
            "lr": tune.sample_from(lambda _: np.random.rand())
        }
    )
ray.tune.run_experiments(experiments, search_alg=None, scheduler=None, with_server=False, server_port=4321, verbose=2, resume=False, queue_trials=False, reuse_actors=False, trial_executor=None, raise_on_failed_trial=True)[source]

Runs and blocks until all trials finish.

Examples

>>> experiment_spec = Experiment("experiment", my_func)
>>> run_experiments(experiments=experiment_spec)
>>> experiment_spec = {"experiment": {"run": my_func}}
>>> run_experiments(experiments=experiment_spec)
>>> run_experiments(
>>>     experiments=experiment_spec,
>>>     scheduler=MedianStoppingRule(...))
>>> run_experiments(
>>>     experiments=experiment_spec,
>>>     search_alg=SearchAlgorithm(),
>>>     scheduler=MedianStoppingRule(...))
Returns:List of Trial objects, holding data for each executed trial.
class ray.tune.Experiment(name, run, stop=None, config=None, resources_per_trial=None, num_samples=1, local_dir=None, upload_dir=None, trial_name_creator=None, loggers=None, sync_to_driver=None, checkpoint_freq=0, checkpoint_at_end=False, keep_checkpoints_num=None, checkpoint_score_attr=None, export_formats=None, max_failures=3, restore=None, repeat=None, trial_resources=None, custom_loggers=None, sync_function=None)[source]

Tracks experiment specifications.

Implicitly registers the Trainable if needed.

Examples

>>> experiment_spec = Experiment(
>>>     "my_experiment_name",
>>>     my_func,
>>>     stop={"mean_accuracy": 100},
>>>     config={
>>>         "alpha": tune.grid_search([0.2, 0.4, 0.6]),
>>>         "beta": tune.grid_search([1, 2]),
>>>     },
>>>     resources_per_trial={
>>>         "cpu": 1,
>>>         "gpu": 0
>>>     },
>>>     num_samples=10,
>>>     local_dir="~/ray_results",
>>>     checkpoint_freq=10,
>>>     max_failures=2)
classmethod from_json(name, spec)[source]

Generates an Experiment object from JSON.

Parameters:
  • name (str) – Name of Experiment.
  • spec (dict) – JSON configuration of experiment.
class ray.tune.function(func)[source]

Wraps func to make sure it is not expanded during resolution.

The use of function arguments in tune configs must be disambiguated by either wrapped the function in tune.sample_from() or tune.function().

Parameters:func – A function literal.
class ray.tune.sample_from(func)[source]

Specify that tune should sample configuration values from this function.

The use of function arguments in tune configs must be disambiguated by either wrapped the function in tune.sample_from() or tune.function().

Parameters:func – An callable function to draw a sample from.
ray.tune.uniform(*args, **kwargs)[source]

A wrapper around np.random.uniform.

ray.tune.choice(*args, **kwargs)[source]

A wrapper around np.random.choice.

ray.tune.randint(*args, **kwargs)[source]

A wrapper around np.random.randint.

ray.tune.randn(*args, **kwargs)[source]

A wrapper around np.random.randn.

class ray.tune.Trainable(config=None, logger_creator=None)[source]

Abstract class for trainable models, functions, etc.

A call to train() on a trainable will execute one logical iteration of training. As a rule of thumb, the execution time of one train call should be large enough to avoid overheads (i.e. more than a few seconds), but short enough to report progress periodically (i.e. at most a few minutes).

Calling save() should save the training state of a trainable to disk, and restore(path) should restore a trainable to the given state.

Generally you only need to implement _train, _save, and _restore here when subclassing Trainable.

Note that, if you don’t require checkpoint/restore functionality, then instead of implementing this class you can also get away with supplying just a my_train(config, reporter) function to the config. The function will be automatically converted to this interface (sans checkpoint functionality).

When using Tune, Tune will convert this class into a Ray actor, which runs on a separate process. Tune will also change the current working directory of this process to self.logdir.

classmethod default_resource_request(config)[source]

Returns the resource requirement for the given configuration.

This can be overriden by sub-classes to set the correct trial resource allocation, so the user does not need to.

classmethod resource_help(config)[source]

Returns a help string for configuring this trainable’s resources.

train()[source]

Runs one logical iteration of training.

Subclasses should override _train() instead to return results. This class automatically fills the following fields in the result:

done (bool): training is terminated. Filled only if not provided.

time_this_iter_s (float): Time in seconds this iteration took to run. This may be overriden in order to override the system-computed time difference.

time_total_s (float): Accumulated time in seconds for this entire experiment.

experiment_id (str): Unique string identifier for this experiment. This id is preserved across checkpoint / restore calls.

training_iteration (int): The index of this training iteration, e.g. call to train(). This is incremented after _train() is called.

pid (str): The pid of the training process.

date (str): A formatted date of when the result was processed.

timestamp (str): A UNIX timestamp of when the result was processed.

hostname (str): Hostname of the machine hosting the training process.

node_ip (str): Node ip of the machine hosting the training process.

Returns:A dict that describes training progress.
delete_checkpoint(checkpoint_dir)[source]

Removes subdirectory within checkpoint_folder

Parameters:checkpoint_dir – path to checkpoint
save(checkpoint_dir=None)[source]

Saves the current model state to a checkpoint.

Subclasses should override _save() instead to save state. This method dumps additional metadata alongside the saved path.

Parameters:checkpoint_dir (str) – Optional dir to place the checkpoint.
Returns:Checkpoint path or prefix that may be passed to restore().
save_to_object()[source]

Saves the current model state to a Python object.

It also saves to disk but does not return the checkpoint path.

Returns:Object holding checkpoint data.
restore(checkpoint_path)[source]

Restores training state from a given model checkpoint.

These checkpoints are returned from calls to save().

Subclasses should override _restore() instead to restore state. This method restores additional metadata saved with the checkpoint.

restore_from_object(obj)[source]

Restores training state from a checkpoint object.

These checkpoints are returned from calls to save_to_object().

export_model(export_formats, export_dir=None)[source]

Exports model based on export_formats.

Subclasses should override _export_model() to actually export model to local directory.

Parameters:
  • export_formats (list) – List of formats that should be exported.
  • export_dir (str) – Optional dir to place the exported model. Defaults to self.logdir.
Returns:

A dict that maps ExportFormats to successfully exported models.

reset_config(new_config)[source]

Resets configuration without restarting the trial.

This method is optional, but can be implemented to speed up algorithms such as PBT, and to allow performance optimizations such as running experiments with reuse_actors=True.

Parameters:new_config (dir) – Updated hyperparameter configuration for the trainable.
Returns:True if reset was successful else False.
stop()[source]

Releases all resources used by this trainable.

logdir

Directory of the results and checkpoints for this Trainable.

Tune will automatically sync this folder with the driver if execution is distributed.

Note that the current working directory will also be changed to this.

iteration

Current training iteration.

This value is automatically incremented every time train() is called and is automatically inserted into the training result dict.

get_config()[source]

Returns configuration passed in by Tune.

_train()[source]

Subclasses should override this to implement train().

The return value will be automatically passed to the loggers. Users can also return tune.result.DONE or tune.result.SHOULD_CHECKPOINT to manually trigger termination of this trial or checkpointing of this trial. Note that manual checkpointing only works when subclassing Trainables.

Returns:A dict that describes training progress.
_save(checkpoint_dir)[source]

Subclasses should override this to implement save().

Parameters:checkpoint_dir (str) – The directory where the checkpoint file must be stored.
Returns:
If string, the return value is
expected to be the checkpoint path or prefix to be passed to _restore(). If dict, the return value will be automatically serialized by Tune and passed to _restore().
Return type:checkpoint (str | dict)

Examples

>>> print(trainable1._save("/tmp/checkpoint_1"))
"/tmp/checkpoint_1/my_checkpoint_file"
>>> print(trainable2._save("/tmp/checkpoint_2"))
{"some": "data"}
_restore(checkpoint)[source]

Subclasses should override this to implement restore().

Parameters:checkpoint (str | dict) – Value as returned by _save. If a string, then it is the checkpoint path.
_setup(config)[source]

Subclasses should override this for custom initialization.

Parameters:config (dict) – Hyperparameters and other configs given. Copy of self.config.
_log_result(result)[source]

Subclasses can optionally override this to customize logging.

Parameters:result (dict) – Training result returned by _train().
_stop()[source]

Subclasses should override this for any cleanup on stop.

_export_model(export_formats, export_dir)[source]

Subclasses should override this to export model.

Parameters:
  • export_formats (list) – List of formats that should be exported.
  • export_dir (str) – Directory to place exported models.
Returns:

A dict that maps ExportFormats to successfully exported models.

class ray.tune.function_runner.StatusReporter(result_queue, continue_semaphore)[source]

Object passed into your function that you can report status through.

Example

>>> def trainable_function(config, reporter):
>>>     assert isinstance(reporter, StatusReporter)
>>>     reporter(timesteps_this_iter=1)
__call__(**kwargs)[source]

Report updated training status.

Pass in done=True when the training job is completed.

Parameters:kwargs – Latest training result status.

Example

>>> reporter(mean_accuracy=1, training_iteration=4)
>>> reporter(mean_accuracy=1, training_iteration=4, done=True)
Raises:StopIteration – A StopIteration exception is raised if the trial has been signaled to stop.

ray.tune.schedulers

class ray.tune.schedulers.TrialScheduler[source]

Bases: object

Interface for implementing a Trial Scheduler class.

CONTINUE = 'CONTINUE'

Status for continuing trial execution

PAUSE = 'PAUSE'

Status for pausing trial execution

STOP = 'STOP'

Status for stopping trial execution

on_trial_add(trial_runner, trial)[source]

Called when a new trial is added to the trial runner.

on_trial_error(trial_runner, trial)[source]

Notification for the error of trial.

This will only be called when the trial is in the RUNNING state.

on_trial_result(trial_runner, trial, result)[source]

Called on each intermediate result returned by a trial.

At this point, the trial scheduler can make a decision by returning one of CONTINUE, PAUSE, and STOP. This will only be called when the trial is in the RUNNING state.

on_trial_complete(trial_runner, trial, result)[source]

Notification for the completion of trial.

This will only be called when the trial is in the RUNNING state and either completes naturally or by manual termination.

on_trial_remove(trial_runner, trial)[source]

Called to remove trial.

This is called when the trial is in PAUSED or PENDING state. Otherwise, call on_trial_complete.

choose_trial_to_run(trial_runner)[source]

Called to choose a new trial to run.

This should return one of the trials in trial_runner that is in the PENDING or PAUSED state. This function must be idempotent.

If no trial is ready, return None.

debug_string()[source]

Returns a human readable message for printing to the console.

class ray.tune.schedulers.HyperBandScheduler(time_attr='training_iteration', reward_attr=None, metric='episode_reward_mean', mode='max', max_t=81)[source]

Bases: ray.tune.schedulers.trial_scheduler.FIFOScheduler

Implements the HyperBand early stopping algorithm.

HyperBandScheduler early stops trials using the HyperBand optimization algorithm. It divides trials into brackets of varying sizes, and periodically early stops low-performing trials within each bracket.

To use this implementation of HyperBand with Tune, all you need to do is specify the max length of time a trial can run max_t, the time units time_attr, the name of the reported objective value metric, and if metric is to be maximized or minimized (mode). We automatically determine reasonable values for the other HyperBand parameters based on the given values.

For example, to limit trials to 10 minutes and early stop based on the episode_mean_reward attr, construct:

HyperBand('time_total_s', 'episode_reward_mean', max_t=600)

Note that Tune’s stopping criteria will be applied in conjunction with HyperBand’s early stopping mechanisms.

See also: https://people.eecs.berkeley.edu/~kjamieson/hyperband.html

Parameters:
  • time_attr (str) – The training result attr to use for comparing time. Note that you can pass in something non-temporal such as training_iteration as a measure of progress, the only requirement is that the attribute should increase monotonically.
  • metric (str) – The training result objective value attribute. Stopping procedures will use this attribute.
  • mode (str) – One of {min, max}. Determines whether objective is minimizing or maximizing the metric attribute.
  • max_t (int) – max time units per trial. Trials will be stopped after max_t time units (determined by time_attr) have passed. The scheduler will terminate trials after this time has passed. Note that this is different from the semantics of max_t as mentioned in the original HyperBand paper.
on_trial_add(trial_runner, trial)[source]

Adds new trial.

On a new trial add, if current bracket is not filled, add to current bracket. Else, if current band is not filled, create new bracket, add to current bracket. Else, create new iteration, create new bracket, add to bracket.

on_trial_result(trial_runner, trial, result)[source]

If bracket is finished, all trials will be stopped.

If a given trial finishes and bracket iteration is not done, the trial will be paused and resources will be given up.

This scheduler will not start trials but will stop trials. The current running trial will not be handled, as the trialrunner will be given control to handle it.

on_trial_remove(trial_runner, trial)[source]

Notification when trial terminates.

Trial info is removed from bracket. Triggers halving if bracket is not finished.

on_trial_complete(trial_runner, trial, result)[source]

Cleans up trial info from bracket if trial completed early.

on_trial_error(trial_runner, trial)[source]

Cleans up trial info from bracket if trial errored early.

choose_trial_to_run(trial_runner)[source]

Fair scheduling within iteration by completion percentage.

List of trials not used since all trials are tracked as state of scheduler. If iteration is occupied (ie, no trials to run), then look into next iteration.

debug_string()[source]

This provides a progress notification for the algorithm.

For each bracket, the algorithm will output a string as follows:

Bracket(Max Size (n)=5, Milestone (r)=33, completed=14.6%): {PENDING: 2, RUNNING: 3, TERMINATED: 2}

“Max Size” indicates the max number of pending/running experiments set according to the Hyperband algorithm.

“Milestone” indicates the iterations a trial will run for before the next halving will occur.

“Completed” indicates an approximate progress metric. Some brackets, like ones that are unfilled, will not reach 100%.

class ray.tune.schedulers.AsyncHyperBandScheduler(time_attr='training_iteration', reward_attr=None, metric='episode_reward_mean', mode='max', max_t=100, grace_period=10, reduction_factor=4, brackets=1)[source]

Bases: ray.tune.schedulers.trial_scheduler.FIFOScheduler

Implements the Async Successive Halving.

This should provide similar theoretical performance as HyperBand but avoid straggler issues that HyperBand faces. One implementation detail is when using multiple brackets, trial allocation to bracket is done randomly with over a softmax probability.

See https://arxiv.org/abs/1810.05934

Parameters:
  • time_attr (str) – A training result attr to use for comparing time. Note that you can pass in something non-temporal such as training_iteration as a measure of progress, the only requirement is that the attribute should increase monotonically.
  • metric (str) – The training result objective value attribute. Stopping procedures will use this attribute.
  • mode (str) – One of {min, max}. Determines whether objective is minimizing or maximizing the metric attribute.
  • max_t (float) – max time units per trial. Trials will be stopped after max_t time units (determined by time_attr) have passed.
  • grace_period (float) – Only stop trials at least this old in time. The units are the same as the attribute named by time_attr.
  • reduction_factor (float) – Used to set halving rate and amount. This is simply a unit-less scalar.
  • brackets (int) – Number of brackets. Each bracket has a different halving rate, specified by the reduction factor.
on_trial_add(trial_runner, trial)[source]

Called when a new trial is added to the trial runner.

on_trial_result(trial_runner, trial, result)[source]

Called on each intermediate result returned by a trial.

At this point, the trial scheduler can make a decision by returning one of CONTINUE, PAUSE, and STOP. This will only be called when the trial is in the RUNNING state.

on_trial_complete(trial_runner, trial, result)[source]

Notification for the completion of trial.

This will only be called when the trial is in the RUNNING state and either completes naturally or by manual termination.

on_trial_remove(trial_runner, trial)[source]

Called to remove trial.

This is called when the trial is in PAUSED or PENDING state. Otherwise, call on_trial_complete.

debug_string()[source]

Returns a human readable message for printing to the console.

ray.tune.schedulers.ASHAScheduler

alias of ray.tune.schedulers.async_hyperband.AsyncHyperBandScheduler

class ray.tune.schedulers.MedianStoppingRule(time_attr='time_total_s', reward_attr=None, metric='episode_reward_mean', mode='max', grace_period=60.0, min_samples_required=3, hard_stop=True, verbose=True)[source]

Bases: ray.tune.schedulers.trial_scheduler.FIFOScheduler

Implements the median stopping rule as described in the Vizier paper:

https://research.google.com/pubs/pub46180.html

Parameters:
  • time_attr (str) – The training result attr to use for comparing time. Note that you can pass in something non-temporal such as training_iteration as a measure of progress, the only requirement is that the attribute should increase monotonically.
  • metric (str) – The training result objective value attribute. Stopping procedures will use this attribute.
  • mode (str) – One of {min, max}. Determines whether objective is minimizing or maximizing the metric attribute.
  • grace_period (float) – Only stop trials at least this old in time. The units are the same as the attribute named by time_attr.
  • min_samples_required (int) – Min samples to compute median over.
  • hard_stop (bool) – If False, pauses trials instead of stopping them. When all other trials are complete, paused trials will be resumed and allowed to run FIFO.
  • verbose (bool) – If True, will output the median and best result each time a trial reports. Defaults to True.
on_trial_result(trial_runner, trial, result)[source]

Callback for early stopping.

This stopping rule stops a running trial if the trial’s best objective value by step t is strictly worse than the median of the running averages of all completed trials’ objectives reported up to step t.

on_trial_complete(trial_runner, trial, result)[source]

Notification for the completion of trial.

This will only be called when the trial is in the RUNNING state and either completes naturally or by manual termination.

on_trial_remove(trial_runner, trial)[source]

Marks trial as completed if it is paused and has previously ran.

debug_string()[source]

Returns a human readable message for printing to the console.

class ray.tune.schedulers.FIFOScheduler[source]

Bases: ray.tune.schedulers.trial_scheduler.TrialScheduler

Simple scheduler that just runs trials in submission order.

on_trial_add(trial_runner, trial)[source]

Called when a new trial is added to the trial runner.

on_trial_error(trial_runner, trial)[source]

Notification for the error of trial.

This will only be called when the trial is in the RUNNING state.

on_trial_result(trial_runner, trial, result)[source]

Called on each intermediate result returned by a trial.

At this point, the trial scheduler can make a decision by returning one of CONTINUE, PAUSE, and STOP. This will only be called when the trial is in the RUNNING state.

on_trial_complete(trial_runner, trial, result)[source]

Notification for the completion of trial.

This will only be called when the trial is in the RUNNING state and either completes naturally or by manual termination.

on_trial_remove(trial_runner, trial)[source]

Called to remove trial.

This is called when the trial is in PAUSED or PENDING state. Otherwise, call on_trial_complete.

choose_trial_to_run(trial_runner)[source]

Called to choose a new trial to run.

This should return one of the trials in trial_runner that is in the PENDING or PAUSED state. This function must be idempotent.

If no trial is ready, return None.

debug_string()[source]

Returns a human readable message for printing to the console.

class ray.tune.schedulers.PopulationBasedTraining(time_attr='time_total_s', reward_attr=None, metric='episode_reward_mean', mode='max', perturbation_interval=60.0, hyperparam_mutations={}, quantile_fraction=0.25, resample_probability=0.25, custom_explore_fn=None, log_config=True)[source]

Bases: ray.tune.schedulers.trial_scheduler.FIFOScheduler

Implements the Population Based Training (PBT) algorithm.

https://deepmind.com/blog/population-based-training-neural-networks

PBT trains a group of models (or agents) in parallel. Periodically, poorly performing models clone the state of the top performers, and a random mutation is applied to their hyperparameters in the hopes of outperforming the current top models.

Unlike other hyperparameter search algorithms, PBT mutates hyperparameters during training time. This enables very fast hyperparameter discovery and also automatically discovers good annealing schedules.

This Tune PBT implementation considers all trials added as part of the PBT population. If the number of trials exceeds the cluster capacity, they will be time-multiplexed as to balance training progress across the population.

Parameters:
  • time_attr (str) – The training result attr to use for comparing time. Note that you can pass in something non-temporal such as training_iteration as a measure of progress, the only requirement is that the attribute should increase monotonically.
  • metric (str) – The training result objective value attribute. Stopping procedures will use this attribute.
  • mode (str) – One of {min, max}. Determines whether objective is minimizing or maximizing the metric attribute.
  • perturbation_interval (float) – Models will be considered for perturbation at this interval of time_attr. Note that perturbation incurs checkpoint overhead, so you shouldn’t set this to be too frequent.
  • hyperparam_mutations (dict) – Hyperparams to mutate. The format is as follows: for each key, either a list or function can be provided. A list specifies an allowed set of categorical values. A function specifies the distribution of a continuous parameter. You must specify at least one of hyperparam_mutations or custom_explore_fn.
  • quantile_fraction (float) – Parameters are transferred from the top quantile_fraction fraction of trials to the bottom quantile_fraction fraction. Needs to be between 0 and 0.5. Setting it to 0 essentially implies doing no exploitation at all.
  • resample_probability (float) – The probability of resampling from the original distribution when applying hyperparam_mutations. If not resampled, the value will be perturbed by a factor of 1.2 or 0.8 if continuous, or changed to an adjacent value if discrete.
  • custom_explore_fn (func) – You can also specify a custom exploration function. This function is invoked as f(config) after built-in perturbations from hyperparam_mutations are applied, and should return config updated as needed. You must specify at least one of hyperparam_mutations or custom_explore_fn.
  • log_config (bool) – Whether to log the ray config of each model to local_dir at each exploit. Allows config schedule to be reconstructed.

Example

>>> pbt = PopulationBasedTraining(
>>>     time_attr="training_iteration",
>>>     metric="episode_reward_mean",
>>>     mode="max",
>>>     perturbation_interval=10,  # every 10 `time_attr` units
>>>                                # (training_iterations in this case)
>>>     hyperparam_mutations={
>>>         # Perturb factor1 by scaling it by 0.8 or 1.2. Resampling
>>>         # resets it to a value sampled from the lambda function.
>>>         "factor_1": lambda: random.uniform(0.0, 20.0),
>>>         # Perturb factor2 by changing it to an adjacent value, e.g.
>>>         # 10 -> 1 or 10 -> 100. Resampling will choose at random.
>>>         "factor_2": [1, 10, 100, 1000, 10000],
>>>     })
>>> run_experiments({...}, scheduler=pbt)
on_trial_add(trial_runner, trial)[source]

Called when a new trial is added to the trial runner.

on_trial_result(trial_runner, trial, result)[source]

Called on each intermediate result returned by a trial.

At this point, the trial scheduler can make a decision by returning one of CONTINUE, PAUSE, and STOP. This will only be called when the trial is in the RUNNING state.

choose_trial_to_run(trial_runner)[source]

Ensures all trials get fair share of time (as defined by time_attr).

This enables the PBT scheduler to support a greater number of concurrent trials than can fit in the cluster at any given time.

debug_string()[source]

Returns a human readable message for printing to the console.

ray.tune.suggest

class ray.tune.suggest.SearchAlgorithm[source]

Bases: object

Interface of an event handler API for hyperparameter search.

Unlike TrialSchedulers, SearchAlgorithms will not have the ability to modify the execution (i.e., stop and pause trials).

Trials added manually (i.e., via the Client API) will also notify this class upon new events, so custom search algorithms should maintain a list of trials ID generated from this class.

See also: ray.tune.suggest.BasicVariantGenerator.

add_configurations(experiments)[source]

Tracks given experiment specifications.

Parameters:experiments (Experiment | list | dict) – Experiments to run.
next_trials()[source]

Provides Trial objects to be queued into the TrialRunner.

Returns:Returns a list of trials.
Return type:trials (list)
on_trial_result(trial_id, result)[source]

Called on each intermediate result returned by a trial.

This will only be called when the trial is in the RUNNING state.

Parameters:trial_id – Identifier for the trial.
on_trial_complete(trial_id, result=None, error=False, early_terminated=False)[source]

Notification for the completion of trial.

Parameters:
  • trial_id – Identifier for the trial.
  • result (dict) – Defaults to None. A dict will be provided with this notification when the trial is in the RUNNING state AND either completes naturally or by manual termination.
  • error (bool) – Defaults to False. True if the trial is in the RUNNING state and errors.
  • early_terminated (bool) – Defaults to False. True if the trial is stopped while in PAUSED or PENDING state.
is_finished()[source]

Returns True if no trials left to be queued into TrialRunner.

Can return True before all trials have finished executing.

class ray.tune.suggest.BasicVariantGenerator(shuffle=False)[source]

Bases: ray.tune.suggest.search.SearchAlgorithm

Uses Tune’s variant generation for resolving variables.

See also: ray.tune.suggest.variant_generator.

Example

>>> searcher = BasicVariantGenerator()
>>> searcher.add_configurations({"experiment": { ... }})
>>> list_of_trials = searcher.next_trials()
>>> searcher.is_finished == True
add_configurations(experiments)[source]

Chains generator given experiment specifications.

Parameters:experiments (Experiment | list | dict) – Experiments to run.
next_trials()[source]

Provides Trial objects to be queued into the TrialRunner.

Returns:Returns a list of trials.
Return type:trials (list)
is_finished()[source]

Returns True if no trials left to be queued into TrialRunner.

Can return True before all trials have finished executing.

class ray.tune.suggest.SuggestionAlgorithm[source]

Bases: ray.tune.suggest.search.SearchAlgorithm

Abstract class for suggestion-based algorithms.

Custom search algorithms can extend this class easily by overriding the _suggest method provide generated parameters for the trials.

To track suggestions and their corresponding evaluations, the method _suggest will be passed a trial_id, which will be used in subsequent notifications.

Example

>>> suggester = SuggestionAlgorithm()
>>> suggester.add_configurations({ ... })
>>> new_parameters = suggester._suggest()
>>> suggester.on_trial_complete(trial_id, result)
>>> better_parameters = suggester._suggest()
add_configurations(experiments)[source]

Chains generator given experiment specifications.

Parameters:experiments (Experiment | list | dict) – Experiments to run.
next_trials()[source]

Provides a batch of Trial objects to be queued into the TrialRunner.

A batch ends when self._trial_generator returns None.

Returns:Returns a list of trials.
Return type:trials (list)
_generate_trials(experiment_spec, output_path='')[source]

Generates trials with configurations from _suggest.

Creates a trial_id that is passed into _suggest.

Yields:Trial objects constructed according to spec
is_finished()[source]

Returns True if no trials left to be queued into TrialRunner.

Can return True before all trials have finished executing.

_suggest(trial_id)[source]

Queries the algorithm to retrieve the next set of parameters.

Parameters:trial_id – Trial ID used for subsequent notifications.
Returns:
Configuration for a trial, if possible.
Else, returns None, which will temporarily stop the TrialRunner from querying.
Return type:dict|None

Example

>>> suggester = SuggestionAlgorithm(max_concurrent=1)
>>> suggester.add_configurations({ ... })
>>> parameters_1 = suggester._suggest()
>>> parameters_2 = suggester._suggest()
>>> parameters_2 is None
>>> suggester.on_trial_complete(trial_id, result)
>>> parameters_2 = suggester._suggest()
>>> parameters_2 is not None

ray.tune.analysis

class ray.tune.analysis.ExperimentAnalysis(experiment_path, trials=None)[source]

Analyze results from a Tune experiment.

Parameters:experiment_path (str) – Path to where experiment is located. Corresponds to Experiment.local_dir/Experiment.name

Example

>>> tune.run(my_trainable, name="my_exp", local_dir="~/tune_results")
>>> analysis = ExperimentAnalysis(
>>>     experiment_path="~/tune_results/my_exp")
dataframe(refresh=False)[source]

Returns a pandas.DataFrame object constructed from the trials.

Parameters:refresh (bool) – Clears the cache which may have an existing copy.
stats()[source]

Returns a dictionary of the statistics of the experiment.

runner_data()[source]

Returns a dictionary of the TrialRunner data.

trial_dataframe(trial_id)[source]

Returns a pandas.DataFrame constructed from one trial.

get_best_trainable(metric, trainable_cls, mode='max')[source]

Returns the best Trainable based on the experiment metric.

Parameters:
  • metric (str) – Key for trial info to order on.
  • mode (str) – One of [min, max].
get_best_config(metric, mode='max')[source]

Retrieve the best config from the best trial.

Parameters:
  • metric (str) – Key for trial info to order on.
  • mode (str) – One of [min, max].
get_best_info(metric, mode='max', flatten=True)[source]

Retrieve the best trial based on the experiment metric.

Parameters:
  • metric (str) – Key for trial info to order on.
  • mode (str) – One of [min, max].
  • flatten (bool) – Assumes trial info is flattened, where nested entries are concatenated like info:metric.

ray.tune.logger

class ray.tune.logger.Logger(config, logdir)[source]

Logging interface for ray.tune.

By default, the UnifiedLogger implementation is used which logs results in multiple formats (TensorBoard, rllab/viskit, plain json, custom loggers) at once.

Parameters:
  • config – Configuration passed to all logger creators.
  • logdir – Directory for all logger creators to log to.
on_result(result)[source]

Given a result, appends it to the existing log.

update_config(config)[source]

Updates the config for all loggers.

close()[source]

Releases all resources used by this logger.

flush()[source]

Flushes all disk writes to storage.