Component run

Overview

In order to quickly verify the code of a component instead of submitting the component to AzureML, Component SDK supports running a component in the local host machine with component.run(). This document describes the scenarios of this feature and how to use it.

Scenarios

Component run covers the following scenarios:

  • Quickly verify the component with a small dataset when:

    • Deploying your own component.

    • Testing other’s component before submitting it.

  • Writing unittests of a component when developing it.

  • Run component in CI pipeline in a lightweight manner.

Features

When calling component.run(), it will help you to prepare the required environment for a component execution, download remote dataset and snapshot. You can also decide whether to upload run results and logs to run history and view it in workspace portal.

Component run supports three run modes, docker, conda and host. Their main difference is the execution environment.

  • Docker mode uses the same image as remote component execution, including the base image and the conda environment, could be used to simulate component run in remote. In docker mode, it will use “docker exec” to execute the component command in container. It is similar to the remote component execution.

  • Conda mode will build a conda environment according to the component’s conda config. In conda mode, its backend uses “subprocess.Popen(shell=True)” to execute the component command in shell.

  • Host mode directly uses current host environment(the python env you are calling component.run) to execute component, which is usually used to develop component for component author. In host mode, it’s similar to conda mode, calls “subprocess.Popen(shell=True)” to execute the component command.

Limitation

For component type:

  • Currently, component.run() support CommandComponent, ParallelComponent and DistributedComponent with MPI launcher. And other component types are all not supported.

For runsettings:

  • Component.run only supports execution in a single node.

  • For ParallelComponent, currently we provide a one process mock driver to simulate the real driver in AzureML(which could enable multi nodes, multi processes). So runsettings of ParallelComponent will be ignored.

  • For DistributedComponent, only process_count_per_node in runsetting is supported.

For component command:

  • If the component command contains special characters, such as “\n”, it may result in execution failure of component command.

For input dataset:

  • component.run() only supports download mode of inputs, so it supports FileDataset but doesn’t support TabularDataset.

  • Except for Dataset, it supports providing a local path in host machine(local path is not able to used for submitting to AzureML), this is useful to quickly verify the component logic.

For output dataset:

  • Component run outputs use the local file system as datastore, not support register as a dataset to the workspace.

  • Component run outputs will ignore parameters of output.configure(), including datastore, output_mode, path_on_datastore.

  • Component run outputs support using path_on_compute to specify output path on the host.

How to use Component run

Prerequisites

  • Install component SDK following the instructions here.

  • For docker mode, need to install Docker on your machine and add it to the system path.

  • For conda mode, need to install Conda on your machine and add it to the system path.

Component run parameters

Name Type Default Description
experiment_name str None The experiment_name will show in the portal. If not set, will use the component name.
working_dir str None The output path for component output info
mode str docker Currently, we support three component-run modes.
docker: Start a container with the component's image and run the component in it.
conda: Build a conda environment in the host with the component's conda definition and run the component in it.
host: Directly run the component in host environment.
track_run_history bool True Indicates whether run history tracks Component.run and view it in workspace portal. If track_run_history=True, run history will track run status. Outputs and log file will be uploaded to it.
show_output bool True Indicates whether to show the component run status on terminal.
raise_on_error bool True Indicates whether to raise an error when the Run is in a failed state.
skip_validation bool False Indicates whether to skip the validation of the component.

Sample

Component run with local/remote dataset

from azureml.core import Dataset
# Suppose we have a workspace as 'ws'
component_func = Component.load(ws, name=<Your component name>)

# Set remote dataset to component inputs.
remote_data = Dataset.get_by_name(ws, name=<Your remote dataset name>)
component = component_func(input_name=remote_data)
component.run()

# Set local dataset path to component inputs.
component = component_func(input_name=<Your local dataset path>)
component.run()

Component run in docker-based environment

component.run(mode='docker')

Component run in conda-based environment

component.run(mode='conda')

NOTE: Before run component in a conda-based environment, please make sure to enable conda in your shell. You can execute ‘conda init –all’ to enable conda.

Component run in user managed environment

component.run(mode='host')

Specific component run output path

# Suppose one output port of component named output_dir.
# Set local path to path_on_compute
component.outputs.output_dir.configure(path_on_compute=<Your output path>)
component.run()

NOTE: path_on_datastore which is parameter of Output.configure will be ignored.

Specific working directory

# If working_dir doesn't exist, component.run will create folder.
# If not set working_dir, component.run will create temp folder as working_dir.
component.run(working_dir=<Your working directory path>)

Show component run status on terminal

component.run(show_output=True)

Track component run

component.run(track_run_history=True)

NOTE: Since it needs to create a run in remote and upload the outputs, it will be slower than track_run_history=False.

Specific experiment name show in portal

# If not set experiment name, component run will use component name as experiment name.
component.run(experiment_name=<Your experiment name>, track_run_history=True)

When track_run_history is True, component.run log likes below. You can find run id and AzureML portal link from it.

working dir is <Your working_dir>
Start initialize run history tracker of component [ <Component name> ] to record execution information.
Finished initialize run history tracker of component [ <Component name> ] to record execution information.
[2021-02-05 16:52:27] RunId: <Component run id>
[2021-02-05 16:52:27] Link to Azure Machine Learning Portal: <AzureML portal link>

Component run working directory structure

Component run working directory structure like below:

working_dir
|   excutionlogs.txt
├───conda_logs (optional)
│       <Conda env name>.txt
├───image_logs (optional)
│       pull_image_log.txt
│       build_image_log.txt
├───<Remote dataset name>
├───outputs
│   └───<Output port name>
└───scripts

excutionlogs.txt records component run and component script execution log. The execution results are stored in the output folder according to the output port name. If using path_on_compute to specify output port path on the host, the output of port will be stored in path_on_compute. And the snapshot of the component will be downloaded to the scripts folder. If component inputs contain remote datasets, they will be downloaded in working_dir. When component execution environment doesn’t exist, it will build conda or image environment and record log in conda_logs or image_logs.

How to handle component run failure

Diagnostic failure reason

Build component environment failure

If conda environment build failed, it will raise the following error. You could find the error message through the build_log_path.

"error": {
    "code": "UserError",
    "message": "Build conda environment in host failed, please find failure reason from <build_log_path>"
}

To shorten the time to build the image, Component.run will perform operations, build image in local and pull remote image, at the same time. If docker image build failed, it will raise following error. You could find the error message through the build_log_path.

"error": {
    "code": "UserError",
    "message": "Failed to get component image, diagnostic failure reason by logs <build_log_path>"
}

Component command execution failure

If raise_on_error=True, it will raise an error like below when the Run is in a failed state. You could find the error message through execution_log_path.

"error": {
    "code": "UserError",
    "message": "Executing script failed with a non-zero exit code; see the <execution_log_path>"
}

If raise_on_error=False, execution_log_path will be printed in terminal. You could find the error message through it.

Debug failed component run

Currently, it only supports python script. If the component run in a docker-based-environment and failed, it will generate config files to debug in vscode. Debug parameters and environment are the same as Component.run.

You can debug component in vscode by steps:

  1. Pressing F1, click “Remote-Containers: Reopen in Container”.

  2. In Status Bar, selecting python interpreter path showed in Component.run terminal output.

  3. Pressing F5 to start debugging.

For more information about component debug, see Component debug