Additional includes

By default, The folder containing the component spec file will be treated as the base folder of the component running environment (a.k.a the snapshot). However, sometimes some common libraries lies on other top level folders. In this case, use additional includes to set the additional file or folders used by the component. The file or folders in additional includes will be copied (optionally as a zip package) to the snapshot folder by the component-cli when registering the component.

To specify additional includes, add a {spec_file_name}.additional_includes file next to the component spec file. For example, if the component spec file is named component_spec.yaml, the additional includes file should be named component_spec.additional_includes. There are two file formats for additional_includes, plain text and yaml format.

It would be helpful to test run the component before the component get registered to a workspace. For local testing, use az ml component build to build a local component snapshot with additional includes resolved.

Plain text format

As an example, for the following folder structure:

src/
  python/
    library1/
      hello.py
    library2/
      en_US/
        messages.json
      zh_CN/
        messages.json
      greetings.py
assets/
  LICENSE
component_entry/
  component_spec.additional_includes
  component_spec.yaml
  run.py

Inside the component_spec.additional_includes file:

../src/python/library1
../src/python/library2.zip
../assets/LICENSE

When packing snapshot, the contents inside the snapshot will be:

component_spec.yaml
run.py
library1/
  hello.py
library2.zip
LICENSE

library2 is copied as a zip package to the snapshot since it is specified with a .zip suffix in the additional_includes file. Inside library2.zip the files are:

library2/
  en_US/
    messages.json
  zh_CN/
    messages.json
  greetings.py

Notes

  • If additional includes file is a plain text file, place each item in one line. The line break could be either \r\n or \n, but it is recommended to use \n.

  • Specified additional file (or folder) will be copied directly into the base folder of the snapshot. No parent folders are included.

  • Each item in the additional include file:

    • Could either be a file or a folder. In the sample above, ../src/python/library1 and ../src/python/library2 are folders, while ../assets/LECENSE is a file.

    • Could be optionally specified with a .zip suffix. As an example, for entry item ../hello.zip:

      • Check whether ../hello.zip file exists. If it exists, copy hello.zip to snapshot folder.

      • If there is no ../hello.zip file exist, check whether ../hello folder exists. If so, compress the folder to a zip file and copy to snapshot folder as hello.zip.

      • If ../hello exists but it is a file (not a folder), an “../hello.zip not found” error will be raised.

    • Must be specified with relative file path from the folder containing the component spec file. Absolute file path not accepted.

      • Good: ../src

      • Bad: /, C:

    • Recommend using a specific folder or file name.

      • Recommended: ../src, ../src/python/library1, ../assets/LICENSE

      • Not recommended: ../../, ~, /

    • Recommend using linux-style path. Windows-style path will not work properly on Linux platforms.

      • Recommended: ../src/python/library1

      • Not recommended: ..\src\python\library1.

Yaml format

When the additional_includes file is yaml format, local paths and universal package of Azure Devops artifacts are supported as additional includes. A config list of additional includes is defined in additional_includes file. The yaml format additional_includes file likes below:

additional_includes:
 - your/local/path1
 - your/local/path2
 - type: artifact
   feed: <your_artifacts_feed_name>
   name: <your_universal_package_name>
   version: <your_package_version>
   scope: <your_feed_scope>
 - type: artifact
   organization: <your_devops_organization>
   project: <your_devops_project>
   feed: <your_artifacts_feed_name>
   name: <your_universal_package_name>
   version: <your_package_version>
   scope: <your_feed_scope>

The configuration of artifact is as follows:

Name Description Required
organization Azure DevOps organization URL. If not configured it will be inferred from current git repo (if this is a devops git). False
project Name of the project. If not configured it will be inferred from current git repo (if this is a devops git). False
feed Name or ID of the feed. Yes
name Name of the package. Yes
version Version of the package. Yes
scope Scope of the feed, accepted values are organization, project. Default value is organization. Yes

If the artifact configs exist in the additional_includes, component SDK will download the universal package of artifact then merge those packages to the component snapshots. Here is an example for additional_includes with artifact config:

  • Folder structure:

src/
  python/
    library1/
      hello.py
    library2/
      en_US/
        messages.json
      zh_CN/
        messages.json
      greetings.py
assets/
  LICENSE
component_entry/
  component_spec.additional_includes
  component_spec.yaml
  run.py

  • component_spec.additional_includes content:

additional_includes:
 - ../src/python/library1
 - ../src/python/library2.zip
 - ../assets/LICENSE
 - type: artifact
   feed: feed_name
   name: universal_package_name
   version: 0.0.1
   scope: project
 - type: artifact
   organization: https://your.organization.name/
   project: package
   feed: feed_name
   name: universal_package_name
   version: 0.0.2
   scope: project

  • Folder structure of universal_package_name:0.0.1

0.0.1/
  file1
file1

  • Folder structure of universal_package_name:0.0.2

0.0.2/
  file2
file2

  • Snapshot of this component:

The operation of local path in additional_includes config is the same as the plain text format. For the artifact configs, component SDK will download the universal package of artifact then merge those packages to the component snapshots.

component_spec.yaml
run.py
library1/
  hello.py
library2.zip
LICENSE
0.0.1/
  file1
file1
0.0.2/
  file2
file2

Sample component

Refer to here for a sample component.

FAQ

Permission error was occurred when downloading artifacts

Download artifact needs to depend on the Azure DevOps extension for Azure CLI. Only these login methods are supported:

If you meet permission error when downloading artifacts, you can try the above login methods to solve it. Reference Azure Devops CLI for more details.