ray.data.ExecutionOptions#

class ray.data.ExecutionOptions(resource_limits: ExecutionResources | None = None, exclude_resources: ExecutionResources | None = None, locality_with_output: bool | List[str] = False, preserve_order: bool = False, actor_locality_enabled: bool = False, verbose_progress: bool | None = None)[source]#

Common options for execution.

Some options may not be supported on all executors (e.g., resource limits).

resource_limits#

Set a soft limit on the resource usage during execution. Autodetected by default.

exclude_resources#

Amount of resources to exclude from Ray Data. Set this if you have other workloads running on the same cluster. Note, - If using Ray Data with Ray Train, training resources will be automatically excluded. - For each resource type, resource_limits and exclude_resources can not be both set.

locality_with_output#

Set this to prefer running tasks on the same node as the output node (node driving the execution). It can also be set to a list of node ids to spread the outputs across those nodes. Off by default.

preserve_order#

Set this to preserve the ordering between blocks processed by operators. Off by default.

actor_locality_enabled#

Whether to enable locality-aware task dispatch to actors (off by default). This parameter applies to both stateful map and streaming_split operations.

verbose_progress#

Whether to report progress individually per operator. By default, only AllToAll operators and global progress is reported. This option is useful for performance debugging. On by default.

DeveloperAPI: This API may change across minor Ray releases.

is_resource_limits_default()[source]#

Returns True if resource_limits is the default value.

validate() None[source]#

Validate the options.