Skip to content

Home

Jump to bottom
Sergei Silnov edited this page Sep 2, 2022 · 9 revisions

User guide

Creating a component for the registry

For components to be published in the registry idf_component.yml manifest is required.

Example of a component manifest idf_component.yml:

# Version of the component [Required, if not passed during upload]
version: "2.3.1"

# List of supported targets [Optional]
# If missing all targets are considered to be supported
targets:
  - esp32

# Short description for the project [Recommended]
description: Test project
# Github repo or a home  [Recommended]
url: https://github.com/espressif/esp-idf

# List of dependencies [Optional]
# All dependencies of the component should be published in the same registry.
dependencies:
  # Default namespace is `espressif`
  # Declaring dependency as `cool_component` is the same as `espressif/cool_component`
  cool_component: ">1.0.0"
  some_ns/some_component:
    version: "~1.2.7"

For most components good start will be:

version: "1.2.7"
description: Test component
url: https://github.com/espressif/my-component

It's also recommended to include documentation and license information with the component.

Version

Version format mainly follows specs of Semantic Versioning 2.0.0 with one extension, revision part. You can use revision to create components from third-party libraries that have versions like a.b.c. You can add one more section separated by tilde to the version to allow releasing multiple versions of a component while still keeping the original library version. I.e., for a library with version 1.2.3 a component version can be 1.2.3~1.

Requirement specifications format

For versions on the web service component manager mainly follows Semantic Versioning 2.0.0 specification. However, this specification doesn't tell how ranges of values should be represented, so the component manager accepts requirement specifications in a few different formats, to work in most cases:

Exact version

If your project or component depends on the exact version of some other component:

  • 1.0.0
  • =1.0.0
  • ==1.0.0

Pre-release versions

Pre-release versions won't satisfy regular specs. I.E. version 0.1.1-alpha2 won't satisfy 0.1.1 requirement/

Build numbers

Build numbers are equivalents to the regular requirements. I.e. version 0.1.1+210119001 will satisfy 0.1.1 requirement.

Version range

To specify ranges of versions you can use

For example: >=0.2.0,<0.5.0. This requirement will be satisfied by versions 0.2.0 and 0.4.34, but won't be satisfied by 0.1.19 and 0.5.0.

Fixed bugfix or minor version:

If you want to restrict the requirement to any bugfix version with fixed minor and major default syntax will be >=1.2.0,<1.3.0. In this case, few alternatives are available:

  • ~1.2.0 or ~1.2
  • ~=1.2.0 or ~=1.2
  • 1.2.x or 1.2.X
  • 1.2.*

Similar to fixed major:

  • >=1.0.0,<2.0.0
  • ^1.0.0, ^1.0 or ^1
  • 1.x
  • 1.*

Documenting components

Component registry automatically processes and renders in the Web UI documentation provided in these files:

  • README.md - General information
  • CHANGELOG.md - Version history
  • API.md - Programming interface

Only markdown (*.md) files are supported. Filenames are not case-sensitive.

Providing a license

There is no field in the manifest to put license name, instead, it's possible to create a file name license or license.txt (case insensitive) in the root of the component with the full text of the license agreement. Commonly used licenses will be detected automatically and displayed with the component. The original text of the license is always delivered with the component.

Uploading a component to the registry

To upload a component run upload-component command. If the component doesn't exist in the registry it will be created automatically. This command requires an API token to be configured.

idf.py upload-component --namespace=username --name=component

Choosing what files to upload

As a component developer, you may want to choose what files from the component directory will be uploaded to the registry. So the idf_component.yml manifest may have include and exclude filters. For example:

files:
  exclude:
    - "*.py" # Exclude all Python files
    - "**/*.list" # Exclude `.list` files in all directories
    - "big_dir/**/*" # Exclude files in `big_dir` directory (but empty directory will be added to archive anyway)
  include:
    - "**/.DS_Store" # Include files excluded by default

Files and directories that are excluded by default can be found here: https://github.com/espressif/idf-component-manager/blob/main/idf_component_tools/file_tools.py#L12

The file field is only taken into account during the preparation of the archive before uploading to the registry.

Add examples to the component

To add some examples to your component place them in the examples directory inside your component. Examples are discovered recursively in subdirectories at this path. A directory with CMakeLists.txt that registers a project is considered as an example.

When an archive with the component is uploaded to the registry all examples are repacked to individual archives and then examples directory is removed from the component archive. So every example must be self-sufficient, i.e. doesn't depend on any files in the examples directory except its own directory.

Adding dependency on the component for examples

When a component repo is cloned from a git repository, then it's essential that for the example in the examples directory to use the component that lays right here in the tree. However, when a single example is downloaded using CLI from the registry, and there is no dependency laying around it must be downloaded from the registry.

This behavior can be achieved by setting override_path for dependency in the manifest file. When override_path is defined for a dependency from the registry it will be used with higher priority. From the examples downloaded from the registry override_paths are removed, so during the build process, it won't try to look for the component nearby.

I.E. for a component named cmp published in the registry as watman/cmp the idf_component.yml manifest in the examples/hello_world/main may look like:

version: "1.2.7"
description: My hello_world example
dependencies:
  watman/cmp:
    version: '~1.0.0'
    override_path: '../../../' # three levels up, pointing the directory with the component itself

You shouldn't add your component's directory to EXTRA_COMPONENT_DIRS to example's CMakeLists.txt, because it will break the examples downloaded from the repo.

Using component manager in CI

Some component manager commands commonly used for CI pipelines use are available without IDF.

  • pack-component - Prepare an archive with the component
  • upload-component - Upload component to the registry
  • upload-component-status - Check status of the processing of the component
  • delete-version - Mark component version as deleted.
  • create-project-from-example - Get example of the component and create a project

These commands can be executed by running component manager as a python package:

python -m idf_component_manager upload-component --namespace robertpaulson --name mayhem

If the component version is stored in the manifest then you probably want to add --allow-existing flag to the upload command to make it work on subsequent requests. Otherwise component manager will exit with error code 144 when the version already exists in the registry.

It's also possible to remove version from the manifest at all and pass it with CLI parameter like --version=1.2.3 or pass --version=git to get version from the git tag.

Uploading components with Github Action

Github Action to upload components to the registry is available as part of Espressif's GitHub actions: https://github.com/espressif/upload-components-ci-action

Authentication in the registry using environment variables

To run a command that changes data in the registry like upload-component an authentication token should be provided. The most common way is to set an environment variable IDF_COMPONENT_API_TOKEN.

To upload a component to a non-default registry a DEFAULT_COMPONENT_SERVICE_URL env variable should be set. If you upload to production this variable should be unset.

NB: staging and production use different access tokens

Authentication in the registry with the config file

It's also possible to save the access token to the config file. It has to be named idf_component_manager.yml and place in the directory with ESP-IDF tools. By default, it is ~/.espressif on macOS and Linux and C:/Users/username/.espressif on Windows. This directory can be changed by setting IDF_TOOLS_PATH environment variable.

Values provided for default profile will be used by default.

Configurable options:

  • api_token - Access token to the registry. Required for all operations modifying data in the registry.
  • default_namespace - Namespace used for the creation of component or upload of a new version. Default is not set.
  • service_url - URL of the component registry API. Default: https://api.components.espressif.com

Example idf_component_manager.yml:

profiles:
  default:
    api_token: some_token
    default_namespace: example

  staging:
    url: https://api.example-service.com
    api_token: my_long_long_token
    default_namespace: my_namespace

All idf.py subcommands that work with the component registry accept --service-profile parameter. I.e. if you have a staging profile configured in the idf_component_manager.yml you can upload to it using:

idf.py upload-component --service-profile=staging --name=my_component --namespace=espressif
Clone this wiki locally