Skip to content

Superflore OE Recipe Generation Scheme

Jump to bottom
Rob Woolley edited this page Apr 24, 2024 · 52 revisions

2024-04-24: The content on this has been preserved while we work to revise the content

For up-to-date information about the meta-ros layer please visit the main page: https://github.com/ros/meta-ros

For simplified instructions on how to get started please refer to these build instructions: https://github.com/ros/meta-ros/blob/build/kas/README.md

To chat about meta-ros, come find us on the OSRF Discord #cwg-openembedded channel.

You are also welcome to join our biweekly meetings. Catch up on our meeting minutes here: ROS OpenEmbedded Working Group Minutes - Google Docs

Superflore OE Recipe Generation Scheme

Revision: 52 Published: 2022-06-05

General

  • It appears that the ros-infrastructure convention is to have the data for all of the releases (ROS calls them "distros") available in [master] instead of having separate branches for each release/distro. meta-ros follows this convention. However, it also follows the OpenEmbedded (OE) convention of layers having different branches for different OpenEmbedded release series with linear history (no merges), having [master] be for the release series currently under development, and having -next suffixed rebaseable branches containing commits that are pending being merged into their corresponding unsuffixed branches. It deviates from the convention for OpenEmbedded layers by having annotated tags along the OpenEmbedded release series branches of the form: <BRANCH>/milestones/<N>.
  • The design allows the generation of recipes for ROS 1 distros as well as ROS 2 distros.
  • The layer defines OE DISTRO-s having the same names as the distribution type from REP 153 ie, ros1 or ros2. Use of these OE DISTRO-s will be optional so that the layer can be used other OE DISTRO-s, eg, webOS.
  • OE allows the assignment of versions to its layers. The meta-ros with recipes generated by superflore in separate layers for each ROS distro is considered version 3 and will be the focus of ongoing development; the meta-ros with all recipes generated in the same layer is version 2 and is retained on the [legacy-v2] branch. The meta-ros with manually created recipes is considered version 1 and is retained on the [legacy] branch.
  • meta-ros only supports ROS distros for which there are generated recipes, which are only made/retained for ROS distros that have not reached their EOLs.
  • superflore by default uses http://repo.ros2.org/rosdistro_cache/<ROS_DISTRO>-cache.yaml.gz (or http://repositories.ros.org/rosdistro_cache/<ROS_DISTRO>-cache.yaml.gz) as input. Unfortunately, the <ROS_DISTRO>-cache.yaml.gz files aren't versioned. To allow reproducibility, a script scripts/ros-generate-cache.sh that wraps rosdistro_build_cache is used generate a cache.yaml for a specified commit of https://github.com/ros/rosdistro and commit it to a new local branch. A second script scripts/ros-generate-recipes.sh, which invokes superflore with the ROSDISTRO_INDEX_URL environment variable pointing to this cache.yaml, is then run to generate and commit the recipes. Also, it passes --no-branch and--dry-run to superflore so that it does not create a new branch and does not create a pull request (because manual modficiations will inevitably be required).
  • The term baseline platform is defined to be the value of distribution_file.release_platforms from <ROS_DISTRO>-cache.yaml chosen as the platform release to which the OE metadata for a ROS distro is targeted. This is always a release of Ubuntu. The packages provided by the baseline platform are known as platform packages.
  • In order to reduce the number of recipes for Python platform packages that will need to be manually created, the optional use of Python 3 with ROS 1 will not be supported.
  • Commits that are specific to a particular ROS_DISTRO have {<ROS_DISTRO>} at the beginning of their summary lines.
  • By default, you will need to source ros_setup.<SHELL> before using ROS 2 or /opt/ros/ROS_DISTRO/setup.<SHELL> before using ROS 1. This can be done implicitly for login shells by adding "ros-implicit-workspace" to EXTRA_IMAGE_FEATURES in your conf/local.conf.

Usage

Installation

  • Follow the instructions here to install superflore. Version 0.3.2-19-g4105ad8 or later must be used. Then issue:

    sudo rosdep init
rosdep update

  • Clone https://github.com/ros/rosdistro.git into a local rosdistro directory.

  • Edit the https://raw.githubusercontent.com/ros/rosdistro/master/rosdep/*.yaml lines in /etc/ros/rosdep/sources.list.d/20-default.list (created by rosdep init) to point to the local directory.

  • Setup a build environment for meta-ros by following the instructions here.

Recipe Generation

  • Generate and commit a meta-ros[12]-<ROS_DISTRO>/files/<ROS_DISTRO>/generated/cache.yaml for a ROS_DISTRO on the branch [superflore/<ROS_DISTRO>/<RELEASE-YYYYMMDD>] from a specified commit of ros/rosdistro.git by issuing:

    cd meta-ros
sh scripts/ros-generate-cache.sh ROS_DISTRO RELEASE-YYYYMMDD PATH-TO-LOCAL-ROS-ROSDISTRO ROS-ROSDISTRO-COMMIT
cd ..

    RELEASE-YYYYMMDD is the value for ROS_DISTRO_RELEASE_DATE, which is taken from the release announcement or the last field of the release-ROS_DISTRO-YYYYMMDD tag. Prior to the first release of ROS_DISTRO, specify "pre-release" for RELEASE-YYYYMMDD. The cache.yaml is generated from the ROS-ROSDISTRO-COMMIT commit of the local ros/rosdistro.git created in the previous step at PATH-TO-LOCAL-ROS-ROSDISTRO. The platform package dependency mapping files used by superflore (base.yaml, python.yaml, and ruby.yaml) are copied from rosdistro to the meta-ros[12]-<ROS_DISTRO>/files/<ROS_DISTRO>/generated/rosdep/ directory to ensure that if recipes are regenerated. the same same dependency mappings will be used.

  • Generate and commit the recipes and other files from the meta-ros[12]-<ROS_DISTRO>/files/<ROS_DISTRO>/generated/cache.yaml generated in the previous step by issuing:

    cd meta-ros
[SUPERFLORE_GEN_OE_RECIPES=/path/to/superflore-gen-oe-recipes] sh scripts/ros-generate-recipes.sh ROS_DISTRO
cd ..

  • If there are entries in meta-ros[12]-<ROS_DISTRO>/files/<ROS_DISTRO>/generated/rosdep-resolve.yaml whose values are of the form ${ROS_UNRESOLVED_DEP-*} (see below ), add openembedded stanzas to ros/rosdistro/rosdep/base.yaml, ros/rosdistro/rosdep/python.yaml, or ros/rosdistro/rosdep/ruby.yaml for them if they are actually platform packages:

    <PLATFORM-PKG>:
  openembedded:
      packages: [ <OE-RECIPE>@<OE-LAYER> ... ]

    and regenerate the recipes. <OE-RECIPE> is the name of a recipe, not the name of a package built by a recipe. It must never have a -native suffix. <OE-LAYER> is only a hint: a recipe can be in different layers in different OpenEmbedded release series.

    If <PLATFORM-PKG> is a Python package that is not prefixed with python3-, then <OE-RECIPE> is prefixed with ${PYTHON_PN}- (after dropping python-, if present) so that it will map to the package for the version of Python specified by ROS_PYTHON_VERSION for the ROS_DISTRO.

    A value of [] will prevent the generation of a dependency on <PLATFORM-PKG>. But because the ros/rosdistro/rosdep/*.yaml files are used for all ROS_DISTRO-s, do not use [] to eliminate a dependency that might be needed in another ROS_DISTRO. Instead, leave it unmapped and set ROS_UNRESOLVED_DEP-<PLATFORM-PKG> to "" in conf/ros-distro/include/<ROS_DISTRO>/ros-distro.inc.

  • Build the images, which will be done using the newly generated recipes, and run the sanity test -- instructions are here.

  • Push the branch upstream and create a pull request. Use the contents of .pr-title.tmp for the title of the pull request and the contents of .pr-message.tmp for the description of the pull request.

  • If any of the ros/rosdistro/rosdep/*.yaml files were updated, issue a pull request for https://github.com/ros/rosdistro. However, it is advisable to wait until the mappings have been used to generate recipes for all of the ROS distros and OpenEmbedded release series before doing so.

Layout

meta-ros-backports-warrior          # Layer with metadata backported from a newer OpenEmbedded release series, in this case
                                    # 2.7 Warrior. The advantage of keeping them in a separate layer is that you can just drop
                                    # this whole layer when upgrading to the newer OpenEmbedded release series and you won't loose
                                    # any of the functionality provided by it.
meta-ros-common                     # Common recipes, configuration files and bbclasses shared by all ROS distros.
meta-ros1                           # Recipes, configuration files and bbclasses specific to ROS 1 distros.
meta-ros1-<ROS1-DISTRO>             # Layer with all recipes and bbappends specific to the ROS 1 distro <ROS1-DISTRO>.
meta-ros2                           # Recipes, configuration files and bbclasses specific to ROS 2 distros.
meta-ros2-<ROS2-DISTRO>             # Layer with all recipes and bbappends specific to the ROS 2 distro <ROS2-DISTRO>.

files/examples/webos-device-config  # An example of how to perform per-device configuration with webOS OSE.

# Each meta-ros[12]-<ROS-DISTRO> layer has this structure:
classes                             # bbclasses
conf                                # configuration files
files                               # other files not used by bitbake such as cache.yaml, cache.diffme, rosdep files etc.
generated-recipes                   # recipes generated by superflore without any manual modifications
recipes-bbappends                   # bbappends for generated recipes
recipes-*                           # other recipes and bbappends which are specific to <ROS-DISTRO>

# Files generated by ros-generate-cache.sh under each meta-ros[12]-<ROS-DISTRO> layer:
files/<ROS_DISTRO>/generated/cache.yaml
files/<ROS_DISTRO>/generated/cache.diffme
files/<ROS_DISTRO>/generated/rosdep
files/<ROS_DISTRO>/generated/rosdep/base.yaml
files/<ROS_DISTRO>/generated/rosdep/python.yaml
files/<ROS_DISTRO>/generated/rosdep/ruby.yaml

# Files generated by ros-generate-recipes.sh under each meta-ros[12]-<ROS-DISTRO> layer:
conf/ros-distro/include/<ROS_DISTRO>/generated/superflore-datetime.inc
conf/ros-distro/include/<ROS_DISTRO>/generated/superflore-ros-distro.inc
files/<ROS_DISTRO>/generated/index-v4.yaml
files/<ROS_DISTRO>/generated/newer-platform-components.list
files/<ROS_DISTRO>/generated/rosdep-resolve.yaml

Files Generated

  • All generated files are placed under separate generated directory trees so that there's no intermingling of generated and manually created files. This also makes it unnecessary for superflore to explicitly track recipes that are renamed. It simply does git rm -r meta-ros[12]-<ROS_DISTRO>/generated-recipes meta-ros[12]-<ROS_DISTRO>/conf/ros-distro/include/<ROS_DISTRO>/generated/ before starting generation and git add meta-ros[12]-<ROS_DISTRO>/generated-recipes meta-ros[12]-<ROS_DISTRO>/conf/ros-distro/include/<ROS_DISTRO>/generated/ when done.

  • Values from <ROS_DISTRO>-cache.yaml:

    • <COMPONENT> = <ROS_DISTRO>-cache.repositories.<ROS-COMPONENT>.replace('_','-').tolower()
    • <PV> = <ROS_DISTRO>-cache.repositories.<ROS-COMPONENT>.release.version.replace('_','-').tolower()
    • <BPN1>,<BPN2>, ... = <ROS_DISTRO>-cache.repositories.<ROS-COMPONENT>.release.packages[].replace('_','-').tolower()
    • <RELEASE-REPO> = <ROS_DISTRO>-cache.repositories.<ROS-COMPONENT>.release.url
    • <RELEASE-COMMIT> = <ROS_DISTRO>-cache.repositories.<ROS-COMPONENT>.release.tags.release.commitof()
    • <RELEASE-BRANCH> = <RELEASE-BRANCH>.dirname()
    • <BASELINE_PLATFORM> = ubuntu- + <ROS_DISTRO>-cache.distribution_file.release_platforms['ubuntu'][0]

    <BASELINE_PLATFORM> is assigned to ROS_DISTRO_BASELINE_PLATFORM manually in conf/ros-distro/include/<ROS_DISTRO>/ros-distro.inc.

  • The .bbappend files are where additional metadata for and overrides to the settings in the generated recipes are placed (but see the next item for an exception). *_<PV>*.bbappend is where the metadata that will need to be checked for validity with each new <PV> is placed, eg, missing dependencies or the patch files that are appended to SRC_URI. More version-agnostic modifications for generated recipes might be placed in *_%.bbappend; their validity is usually checked only when creating a layer for new a <ROS_DISTRO>. Always use bitbake -e to confirm that the modifications applied are what is expected.

  • There is one exception to always placing overrides to the generated recipe settings in .bbappend-s: modifications to ROS_BUILD_TYPE. This variable needs to be assigned before the recipe is parsed (because ROS_BUILD_TYPE decides which .bbclass to inherit and a recipe's .bbappend files parsed after it has been), so place these modifications in conf/ros-distro/include/<ROS_DISTRO>/ros-distro.inc using a _pn- override, eg, ROS_BUILD_TYPE_pn-sros2-cmake = "ament_cmake".

  • _<ROS_DISTRO_BASELINE_PLATFORM>, _<ROS_DISTRO_TYPE>-distro, and _<ROS_DISTRO> overrides can be used on variables in any recipe that inherits ros_distro.bbclass, but they should only be necessary in those under meta-ros-common.

  • Be careful when using the standard OE _<DISTRO> override because this layer is intended to also be usable with other OE distro layers, eg, meta-webosose.

  • If the <PV> of a package changes as the result of generating for <ROS_DISTRO> from a new version of cache.yaml and there exists a *_<PV>.bbappend file for it, either it is renamed to contain the new <PV>, or, if it is no longer needed, it is removed. One advantage of using .bbappend files instead of the .inc files used in layer version 2 is that when <PV> no longer matches, a parsing error will be shown, while the .inc file would just silently stop being applied.

  • If there are many .bbappend files adding the same non-trivial modifications, then the .bbappend-s can require a common .inc file to prevent unnecessary duplication. For simple few line modifications, the .bbappend-s might be identical with just different filenames to apply to different recipes.

  • Do not place *_%.bbappend-s that are common to multiple ROS distros under meta-ros[12]. By keeping all of the .bbappend-s for a <ROS_DISTRO> under meta-ros[12]-<ROS_DISTRO>, it guarantees that changes made for a new release of it will not affect other ROS distros.

  • If a package appears in <buildtool_depends> or <buildtool_export_depend>, it is a package that is meant to run on the machine that is doing the build, ie, a -native OE package. superflore generates a variable ROS_SUPERFLORE_GENERATED_BUILDTOOLS whose value is a list of these packages. The generated recipes then append native to BBCLASSEXTEND if <BPN>-native appears in ROS_SUPERFLORE_GENERATED_BUILDTOOLS.

  • Rename the superflore entry point superflore-gen-meta-pkgs to superflore-gen-oe-recipes.

  • superflore implements the semantics of rosdep --skip-keys "<PACKAGES-TO-SKIP>", but as of Milestone 12, it is no longer used to exclude packages that do not build. Instead, recipes are generated for all packages in cache.yaml and those that do not build are excluded using the OpenEmbedded blacklisting mechanism (see blacklist.bbclass) in combination with values in the variable ROS_WORLD_SKIP_GROUPS.

  • Specifying the --ros-distro and --dry-run options to superflore-gen-oe-recipes is required.

  • The --upstream-repo option to superflore-gen-oe-recipes is enhanced to allow the specification of the branch from which to fetch.

  • The first line of generated files is always: # Generated by superflore -- DO NOT EDIT

  • Generated files have a single final newline.

  • The summary line of the commit superflore creates is:

      {<ROS_DISTRO>} Sync to <ROS_DISTRO>-cache.yaml as of <DATETIME>

    where <DATETIME> is the value that is assigned to ROS_SUPERFLORE_GENERATION_DATETIME (see below). The body of the commit message contains the command line used and a list of the packages changed. Unless given the --dry-run or --no-branch options, superflore pushes the commit to an upstream branch named superflore/<ROS_DISTRO>/<RELEASE-YYYYMMDD> and creates a pull request for the repo and branch specified by the values given for --upstream-repo and --upstream-branch with a title and description duplicating the commit message, but with Markdown formatting added.

  • superflore enforces generating recipes for a single ROS_DISTRO at a time.

  • Will need to create ros_<ROS_BUILD_TYPE>.bbclass. Most likely they will just inherit from an existing bbclass.

  • A ros_distro_<ROS_DISTRO>.bbclass is manually created for each new ROS_DISTRO.

  • ROS package names that are for the target can end with _native. This clashes with the OE convention where <BPN>-native packages run on the build machine => superflore must change the <BPN>-s of the recipes generated for these packages to end with -rosnative. (This can't be done in rosdep/base.yaml because it's only for platform packages.)

  • ROS package names can end with _dev. This clashes with the OE convention where <BPN>-dev packages contain the files only needed when another component uses <BPN> in its build => superflore must change the <BPN>-s of the recipes generated for these packages to end with -rosdev. (This can't be done in rosdep/base.yaml because it's only for platform packages.)

  • OE does not allow a package to have the same name as DISTRO => superflore must append --distro-renamed to any packages named ros1 or ros2. (This can't be done in rosdep/base.yaml because it's only for platform packages.)

  • superflore must handle <*depends> condition attributes.

  • It appears that the values for ROS_EXPORT_DEPENDS of ament_cmake should actually be in ROS_BUILDTOOL_EXPORT_DEPENDS, ie, they should have a -native suffix and not appear in the RDEPENDS list of packagegroup-ros-world. Special case code is added to superflore to do this until the maintainer of ament_cmake has been convinced to change its packages.xml.

generated-recipes/<COMPONENT>/<BPN>_<PV>.bb

# Generated by superflore -- DO NOT EDIT
#
# Copyright Open Source Robotics Foundation

inherit ros_distro_${ROS_DISTRO}
inherit ros_superflore_generated

DESCRIPTION = "<description-with-XHTML-expanded>"
AUTHOR =  "<maintainer>"                    # <maintainer> more closely matches the semantics of AUTHOR than <author>.
ROS_AUTHOR = "<author>"                     # If no <author> item, don't output this line.
HOMEPAGE = "<url>"
SECTION = "[as generated by superflore v0.2.1]"
LICENSE = "[as generated by superflore v0.2.1, except sequences of spaces and punctuation other than '-', '_', '.', '+' => a single '-']"
LIC_FILES_CHKSUM "[as generated by superflore v0.2.1]"

ROS_CN = "<ROS-COMPONENT>"                  # As found in ROS_DISTRO/cache.yaml, ie, underscores NOT converted to hyphens.
ROS_BPN = "<ROS-PKG>"

ROS_BUILD_DEPENDS = " \
    <build_depend>
    <depend>
"

ROS_BUILDTOOL_DEPENDS = " \
    <buildtool_depend>-native
"

ROS_EXPORT_DEPENDS = " \
    <build_export_depend>
    <depend>
"

ROS_BUILDTOOL_EXPORT_DEPENDS = " \
    <buildtool_export_depend>-native
"

ROS_EXEC_DEPENDS = " \
    <exec_depend>
    <depend>
"

# Currently informational only -- see http://www.ros.org/reps/rep-0149.html#dependency-tags.
ROS_TEST_DEPENDS = " \
    <test_depend>
"

DEPENDS = "${ROS_BUILD_DEPENDS} ${ROS_BUILDTOOL_DEPENDS}"
# Bitbake doesn't support the "export" concept, so build them as if we needed them to build this package (even though we actually
# don't) so that they're guaranteed to have been staged should this package appear in another's DEPENDS.
DEPENDS += "${ROS_EXPORT_DEPENDS} ${ROS_BUILDTOOL_EXPORT_DEPENDS}"

RDEPENDS_${PN} += "${ROS_EXEC_DEPENDS}"

# matches with: [SRC_URI generated by superflore v0.2.1]
ROS_BRANCH ?= "branch=<RELEASE-BRANCH>"
SRC_URI = "<RELEASE-REPO>;${ROS_BRANCH};protocol=https"
SRCREV = "<RELEASE-COMMIT>"
S = "${WORKDIR}/git"

ROS_BUILD_TYPE = "<build_type>"             # Underscores NOT converted to hyphens. If multiple, use the last one; if not
                                            # specified in ROS_DISTRO-cache.yaml, defaults to "catkin" (see
                                            # http://www.ros.org/reps/rep-0149.html#build-type-multiple) if ROS 1 and
                                            # "ament_cmake" if ROS 2 (empirically determined). Can not be overridden in a
                                            # .bbappend; instead, add a _pn- override in
                                            # conf/ros-distro/include/ROS_DISTRO/ros-distro.inc .

inherit ros_${ROS_BUILD_TYPE}

conf/ros-distro/include/<ROS_DISTRO>/generated/superflore-ros-distro.inc

# <ROS_DISTRO>/generated/superflore-ros-distro.inc
# Generated by superflore -- DO NOT EDIT (except ROS_DISTRO_METADATA_VERSION_REVISION)
#
# Copyright Open Source Robotics Foundation

# Increment every time meta-ros is released because of a manually created change, ie, NOT as a result of a superflore run (which
# resets it to "0").
ROS_DISTRO_METADATA_VERSION_REVISION = "0"

ROS_SUPERFLORE_PROGRAM_VERSION = "<__version__>"
ROS_SUPERFLORE_GENERATION_SCHEME = "2"

ROS_DISTRO_TYPE = "ros1"                                <- ROS 1
ROS_VERSION = "1"                                       <- ROS 1
# DO NOT OVERRIDE ROS_PYTHON_VERSION                    <- ROS 1
ROS_PYTHON_VERSION = "2"                                <- ROS 1

ROS_DISTRO_TYPE =  = "ros2"                             <- ROS 2
ROS_VERSION = "2"                                       <- ROS 2
# DO NOT OVERRIDE ROS_PYTHON_VERSION                    <- ROS 2
ROS_PYTHON_VERSION = "3"                                <- ROS 2

ROS_SUPERFLORE_GENERATION_SKIP_LIST = " \
    <PACKAGES-SKIPPED>
"

# Superflore was unable to generate recipes for these packages, eg, because their repositories are not on GitHub.
ROS_SUPERFLORE_GENERATION_NOT_POSSIBLE = " \
    <PACKAGES-NOT-GENERATED>
"

# Number of commits that will be returned by "git log files/ROS_DISTRO/cache.yaml" when the generated files are committed. This is
# used for the fourth version field of ROS_DISTRO_METADATA_VERSION.
ROS_NUM_CACHE_YAML_COMMITS = "<COUNT>"

# Iterated values of ROS_DISTRO-cache.distribution_file.release_platforms.<LINUX-DISTRO>.[ <NAME> ... ] .
ROS_DISTRO_RELEASE_PLATFORMS = " \
    <LINUX-DISTRO>-<NAME>
"

ROS_SUPERFLORE_GENERATED_RECIPES = " \
    <LIST-OF-ALL-GENERATED-RECIPES>
"

ROS_SUPERFLORE_GENERATED_RECIPE_BASENAMES_WITH_COMPONENT = " \
    <LIST-OF-ALL-GENERATED-RECIPE-COMPONENT/BASENAMES>  # Leave off the .bb . Sort solely on the recipe name; ignore the component
                                                        # and the underscore and what follows it. Add leading spaces to align the
                                                        # "/"-s.
"

# What's built by packagegroup-ros-world. Does not include packages that appear solely in ROS_SUPERFLORE_GENERATED_BUILDTOOLS
# (with a -native suffix) or ROS_SUPERFLORE_GENERATED_TESTS.
ROS_SUPERFLORE_GENERATED_WORLD_PACKAGES = " \
    <LIST-OF-GENERATED-WORLD-PACKAGES>
"

# Packages found in the <buildtool_depend> and <buildtool_export_depend> items, ie, ones for which a -native is built. Does not
# include those found in the ROS_EXEC_DEPENDS values in the recipes of build tools.
ROS_SUPERFLORE_GENERATED_BUILDTOOLS_<ROS_DISTRO> = " \
    <LIST-OF-NATIVE-PACKAGES>
"

ROS_SUPERFLORE_GENERATED_BUILDTOOLS_append = " ${ROS_SUPERFLORE_GENERATED_BUILDTOOLS_<ROS_DISTRO>}"

ROS_SUPERFLORE_GENERATED_PLATFORM_PACKAGE_DEPENDENCIES = " \
    <LIST-OF-PLATFORM-PACKAGES-DEPENDENCIES>
"

# Packages found in only the <test_depend> items. Does not include those found only in the ROS_*_DEPENDS of recipes of tests.
ROS_SUPERFLORE_GENERATED_TESTS = " \
    <LIST-OF-TEST-PACKAGES>
"

ROS_SUPERFLORE_GENERATED_RECIPES_FOR_COMPONENTS = " \
    <LIST-OF-COMPONENTS-WITH-GENERATED-RECIPES>
"

# Platform packages without a OE-RECIPE@OE-LAYER mapping in base.yaml, python.yaml, or ruby.yaml. Until they are added, override
# the settings in ros-distro.inc .
ROS_UNRESOLVED_DEP-<PLATFORM_PKG1> = "ROS_UNRESOLVED_DEP-<PLATFORM_PKG1>"
ROS_UNRESOLVED_DEP-<PLATFORM_PKG2> = "ROS_UNRESOLVED_DEP-<PLATFORM_PKG2>"
...

# From the release announcement or the last field of the "release-ROS_DISTRO-YYYYMMDD" tag for the release in
# https://github.com/ros2/ros2/releases. Prior to the first release of a ROS_DISTRO, it is set to "".
ROS_DISTRO_RELEASE_DATE = "<YYYYMMDD>"

# The commit of ros/rosdistro/ROS_DISTRO/distribution.yaml from which the recipes were generated.
ROS_SUPERFLORE_GENERATION_COMMIT = "<ROS-ROSDISTRO-COMMIT>"

conf/ros-distro/include/<ROS_DISTRO>/generated/superflore-datetime.inc

  • If git diff reports that the only change after superflore is run is to this file or to files/<ROS_DISTRO>/newer-platform-components.list, their previous versions are checked out and the run is considered a NOP (because the commit would be empty).

    # <ROS_DISTRO>/generated/superflore-datetime.inc
# Generated by superflore -- DO NOT EDIT
#
# Copyright Open Source Robotics Foundation

# The time, in UTC, associated with the last superflore run that resulted in a change to the generated files. The date portion is
# used as the third version field of ROS_DISTRO_METADATA_VERSION prior to the first release of a ROS_DISTRO.
ROS_SUPERFLORE_GENERATION_DATETIME = "<YYYYMMDDHHmmss>"

files/<ROS_DISTRO>/generated/cache.yaml

  • This is the output from ros-generate-cache. The comment

    # <ROS_DISTRO>/cache.yaml <RELEASE-YYYYMMDD> <ROS-ROSDISTRO-COMMIT> <ROS-ROSDISTRO-COMMIT-DATETIME>

    is added as the first line of the file. The values are used by ros-generate-recipes to generate the assignments to ROS_DISTRO_RELEASE_DATE, ROS_SUPERFLORE_GENERATION_COMMIT, and ROS_SUPERFLORE_GENERATION_DATETIME.

files/<ROS_DISTRO>/generated/cache.diffme

  • This is files/<ROS_DISTRO>/cache.yaml filtered to make it easier to discover the differences between versions of the file. Created by ros-generate-cache using:

    sed -e 's@{\([^ }][^ }]*\)}@[[\1]]@g' -e 's@{@{\n@g' -e 's@}@\n}@g' -e 's@\[\[@{@g' -e 's@]]@}@g' -e 's@, @,\n@g' \
    -e 's@^    @-----\n@' -e 's@<version>[^<]*</version>@@' -e 's@><@>\n<@g' -e 's@^  @-----\n@' \
    -e 's@^\(source_repo_package_xmls:\)@-----\n\1@' \
    $ROS_DISTRO/cache.yaml > $ROS_DISTRO/cache.diffme

    The <version> XML item is removed to prevent confusion; it's the version value from the component that's used for <PV>.

files/<ROS_DISTRO>/generated/index-v4.yaml

  • This is a copy of the ros/rosdistro/index-v4.yaml used by ros-generate-cache when it last generated files/<ROS_DISTRO>/cache.yaml.

files/<ROS_DISTRO>/generated/newer-platform-components.list

  • This is a list of the platform components the version of whose packages might be newer than those provided by Ubuntu bionic. It is created by issuing:

    #! /bin/bash
echo "# $ROS_DISTRO/newer-platform-components.list" > files/$ROS_DISTRO/newer-platform-components.list
wget http://packages.ros.org/${ROS_DISTRO_TYPE/1/}/ubuntu/dists/bionic/main/source/Sources.gz
gunzip Sources.gz
egrep '^(Package|Version|Build-Depends):' Sources \
  | awk '$1 ~ /^Package:/ && $2 !~ /^ros-/ { printf "%s;", $2; getline; printf "%s;", $2; getline; print}' \
  | sed -e 's/Build-Depends: //' -e 's/, /,/g' | LC_ALL=C sort -t ';' -k 1,1 >> files/$ROS_DISTRO/newer-platform-components.list

files/<ROS_DISTRO>/generated/rosdep-resolve.yaml

  • This contains the mappings used by superflore to resolve platform package dependency names, including the @<OE-LAYER> and including ones where the mapping is to []. If there is no mapping specified for a name, its value is ${ROS_UNRESOLVED_DEP-<DEPENDENCY>} and a ROS_UNRESOLVED_DEP-<DEPENDENCY> = "ROS_UNRESOLVED_DEP-<DEPENDENCY>" line is added to conf/ros-distro/include/<ROS_DISTRO>/generated/superflore-ros-distro.inc.
  • The first line of the file is the comment: # <ROS_DISTRO>/rosdep-resolve.yaml.
  • Can convert into one line per item using sed -n -e '/\[\]$/ p' -e '/:$/ {;N;s/\n-//p;}'.

files/<ROS_DISTRO>/generated/rosdep/*.yaml

  • This directory contains copies of the platform package mapping files used by superflore when the recipes for <ROS_DISTRO> were last generated.

Manually Created Files under recipes-appends

  • Metadata that will need to be checked for validity with each new <PV>, especially the appending of patch files to SRC_URI and the altering of anything whose value can be changed by updating package.xml (eg, missing dependencies), is always placed in *_<PV>.bbappend.
  • Other metadata that is expected to be constant across versions, eg, inheriting of additional bbclasses, overridding of generated variable values, overridding or appending to tasks or FILES_${PN}*, are placed in the appropriate .inc file.
  • Note that ros_<ROS_BUILD_TYPE>.bbclass can assign to FILES_${PN}* => to be safe, always use _prepend instead of =+ or += when adding to these variables in .bbappend files.
  • The .bbclass files deliberately have underscores in their name instead of hyphens. (Open webOS used this convention for a good reason; I believe there was some advantage to having functions defined in the .bbclass be prefixed identically with the basename of the file.)

<BPN>_%.bbappend

  • Additional version-independent metadata and overrides only for package <BPN>, eg, adjustments to FILES_*.

<BPN>_<PV>.bbappend

  • Additional metadata and overrides only for version <PV> of package <BPN>, eg, dependency adjustments. Add patches using this pattern:

    FILESEXTRAPATHS_prepend := "${THISDIR}/${BPN}:"

SRC_URI_append = " \
    file://FILE1.patch \
    ...
"

# Other metadata that will need to be checked for validity with each new <PV>.

Other Manually Created Files

meta-ros-common/conf/milestone.conf

# Copyright (c) 2020 LG Electronics, Inc.
#
# Last meta-ros milestone completed. The commit that changes this value must be the last one done for a milestone.
ROS_DISTRO_METADATA_MILESTONE_NUMBER = "<N>"
ROS_DISTRO_METADATA_MILESTONE_RELEASE_DATE = "<YYYYMMDD>"

Layer Version 3 (ROS_SUPERFLORE_GENERATION_SCHEME 2)

Phase 1

  • Restructure meta-ros.git into sublayers: meta-ros-common, meta-<ROS_DISTRO_TYPE>, meta-<ROS_DISTRO_TYPE>-<ROS_DISTRO>. Thanks, Windel Bouwman, for this excellent suggestion. Also, this will make is much simpler to support building packages from a ROS 1 distro for inclusion with a ROS 2 distro.
  • Move webos-compat.inc and webos-image-*into a separate integration layer with its own repo ros/meta-ros-webos.git. Its layer priority is set so that it is parsed before the layers for webOS and the other ROS layers so that it can override settings in them (ie, to a higher value than those of the webOS and other ROS layers).
  • Move Python 3.7 (and others?) into a meta-ros-backports-warrior sublayer.
  • Move recipes imported from other layers into meta-ros-common/recipes-imported-<source-layer>.
  • Collapse and rename includes-ros*/*/*.inc -> recipes-appends/*/*.bbappends. (Because there are now separate layers for each ROS_DISTRO, each recipes-appends tree only provide metadata for a single version of a recipe, which means that .inc files are no longer needed.) This change causes there not to be a use for ROS_COMPONENT_TYPE => drop it, ros<N>_component.bbclass, and the ${ROS_COMPONENT_TYPE}-component field from DISTROOVERRIDES, and have superflore not generate inherit {ROS_COMPONENT_TYPE}_component (instead, add inherit ros_component to ros_superflore_generated.bbclass).
  • Generate recipes that use the Git fetcher instead of fetching the GitHub archive tarballs (whose checksums are not guaranteed to stay the same). See here. Eventually, meta-ros will support a OpenEmbedded release series that has a QA check for this. Also, there are manually created recipes which will need to be changed.
  • Simplify the image names: core-image-ros-* -> ros-image-* and *-roscore -> *-core.
  • Eliminate the requirement to set DISTRO in conf/bblayers.conf.
  • Make these changes to the OE distro name and version for DISTRO = "ros1" and DISTRO = "ros2":
    • Append with OpenEmbedded Linux to DISTRO_NAME.
    • Append +<YOCTO-VERSION-TAG>-<COMMITS-BEYOND-TAG> to DISTRO_VERSION. This will have to be done manually, so set the value the value ros*.mcf files.

Phase 2

  • Move the recipes for components found in files/<ROS_DISTRO>/generated/newer-platform-components.list to meta-ros-common/recipes-infrastructure.
  • Convert from using DISTRO_FEATURES values to remove groups of packages that do not build from the *-image-*-world images to using values in a new variable ROS_WORLD_SKIP_GROUPS. Add conditional PNBLACKLIST-s for them so that they are excluded from bitbake world.
  • Copy <ROS_BUILD_TYPE>.bbclass to ros_<ROS_BUILD_TYPE>.bbclass; deprecate the former.
  • Copy .bbclass-s with hyphens in their names to files with names that have underscores instead and are prefixed with ros_; deprecate the former.

Phase 3

  • Eliminate the requirement to set ROS_DISTRO in conf/bblayers.conf and, hence, drop its setting in build/files/*.mcf.

Phase 4

  • Install the packages for ROS 2 distros under /opt/ros/<ROS_DISTRO> instead of /usr and drop /usr/bin/ros_setup.sh.
  • Have ros-generate-recipes move newer-platform-components.list to meta-ros[12]/files/generated/ros[12] and append generated PREFERRED_VERSION_* settings for the components to meta-ros[12]-<ROS_DISTRO>/conf/ros-distro/include/<ROS_DISTRO>/generated/superflore-ros-distro.inc. This is needed because ROS 1 and ROS 2 have different archives for these packages and there is nothing that enforces that they have the same version in both. For noetic, foxy and rolling, generate newer-platform-components.list from http://packages.ros.org/ros2/ubuntu/dists/focal/main/source/Sources.gz.

Migration of Layer Version 2 Files

Layer Version 2 Layer Version 3
classes/ {meta-ros-common,meta-ros[12]}/classes/
classes/ament.bbclass (deprecated) meta-ros2/classes/ros_ament_cmake.bbclass
classes/ament-python.bbclass (deprecated) meta-ros2/classes/ros_ament_python.bbclass
classes/catkin.bbclass (deprecated) meta-ros1/classes/ros_catkin.bbclass
classes/faulty-solibs.bbclass (deprecated) meta-ros-common/classes/ros_faulty_solibs.bbclass
classes/ros-insane.bbclass (deprecated) meta-ros-common/classes/ros_insane_dev_so.bbclass
conf/distro/ros1.conf meta-ros1/conf/distro/ros1.conf
conf/distro/ros2.conf meta-ros2/conf/distro/ros2.conf
conf/distro/include/ meta-ros-common/conf/distro/include/
conf/ros-distro/include/ubuntu-bionic/ meta-ros-common/conf/ros-distro/include/ubuntu-bionic/ (subseqently removed)
conf/ros-distro/include/<ROS_DISTRO>/ meta-ros[12]-<ROS_DISTRO>/conf/ros-distro/include/<ROS_DISTRO>/
conf/ros-distro/include/<ROS_DISTRO>/generated-superflore-datetime.inc meta-ros[12]-<ROS_DISTRO>/conf/ros-distro/include/<ROS_DISTRO>/generated/superflore-datetime.inc
conf/ros-distro/include/<ROS_DISTRO>/generated-ros-distro.inc meta-ros[12]-<ROS_DISTRO>/conf/ros-distro/include/<ROS_DISTRO>/generated/superflore-ros-distro.inc
files/*.mcf separate [build-draft] branch in meta-ros
files/examples/webos-device-config/ meta-ros-webos/files/examples/webos-device-config/
files/<ROS_DISTRO>/ meta-ros[12]-<ROS_DISTRO>/files/<ROS_DISTRO>/generated/
generated-recipes-<ROS_DISTRO>/ meta-ros[12]-<ROS_DISTRO>/generated-recipes/
includes-ros[12]/*/*.inc meta-ros[12]-<ROS_DISTRO>/recipes-bbappends/*/*.bbappend
recipes-*/ {meta-ros-common,meta-ros[12],meta-ros-backports-warrior,meta-ros-webos}/recipes-*/
recipes-core/packagegroups/packagegroup-ros-world-<ROS_DISTRO>.inc meta-ros[12]-<ROS_DISTRO>/recipes-core/packagegroups//packagegroup-ros-world-<ROS_DISTRO>.bb
scripts/mcf separate [build-draft] branch in meta-ros

Defer Until ROS_SUPERFLORE_GENERATION_SCHEME 3

  • Support building and adding ROS 1 packages to a ROS 2 image.
  • Support building a custom version of ros1-bridge. Will need to source the setup.sh for both ROS distros.
  • Generation of recipes for components whose repos are not on GitHub.
  • Generation of <BPN>*_git.bb with a SRC_URI that fetches from HEAD of branch repositories.<COMPONENT>.release.source.version of repo repositories.<COMPONENT>.release.source.url. These will allow the latest HEAD of a component to be built before a new version of it is added to <COMPONENT>-release.git and <ROS_DISTRO>-cache.yaml updated.
  • The technique for determining if a package is a build tool misses those packages that appear in the ROS_EXEC_DEPENDS of recipes for build tools. Enhance it to find them.
  • Support for test targets (use of ROS_TEST_DEPENDS). Add platform packages found in the <test_depend> to ROS_SUPERFLORE_GENERATED_PLATFORM_PACKAGE_DEPENDENCIES.
  • The technique for determining if a package is a test misses those packages that appear only in the dependencies of test packages. Enhance it to find them.
  • Support for nativesdk.
  • Support for <conflicts>and <replaces> items in packages.xml.
  • Support for <architecture_independent/> items in packages.xml.
  • Support for <metapackage/> items in packages.xml`.
  • Support for <member_of_group> and <group_depend> items in packages.xml. Perhaps this implementation:
    • Introduce "dependency groups" which are the analogue of "package groups" except they have DEPENDS instead of RDEPENDS.
    • Have superflore generate recipes dependencygroups/dependencygroup-<DEPENDENCY-GROUP-WITH-UNDERSCORES>.bb whose DEPENDS values contains the packages that have <member_of_group> items specifying the groups.
    • For packages with a <group_depend> item, add dependencygroup-<DEPENDENCY-GROUP-WITH-UNDERSCORES> to their ROS_BUILD_DEPENDS setting.
    • A new package can add itself to a dependency group by creating a dependencygroup-<DEPENDENCY-GROUP-WITH-UNDERSCORES>.bbappend in its recipe subdirectory.
  • Support for version_* attributes in packages.xml.
  • Support for file attribute of <license> in packages.xml.
  • Support for type attribute of <url> in packages.xml.
  • Create the value for LAYERDEPENDS_ros-layer from what's in rosdep/base.yaml NO: some of the layer dependencies are conditional upon what's enabled in DISTRO_FEATURES => add a variable in superflore-ros-distro.inc that lists them all.
  • Automatically generate RDEPENDS for Python dependencies using https://bitbucket.org/blais/snakefood, modelling it after how automatic dependencies on .so-s are currently generated. Once available, have superflore remove any platform packages from the values for ROS_EXEC_DEPENDS that it generates.
  • Better handling of formatting in <description> when generating DESCRIPTION, including expanding any XHTML found in it.
  • Outputting the entire value of <license>. Currently, everything after the first comma is ignored.
  • Enhance catkin-pkg and rosdistro to optionally return the list of <depends> values instead of merging them into others.
  • Generate recipes for the ROS infrastructure packages (those in http://packages.ros.org/{ros,ros2}/ubuntu/dists/bionic/main/binary-amd64/Packagesand http://packages.ros.org/{ros,ros2}/ubuntu/dists/bionic/main/source/Sources.gz whose names do not start with ros-).
  • Abort the build if ROS_DISTRO is not in ROS_DISTRO_COMPAT.
  • Consider the use of BBFILES_DYNAMIC to skip the inclusion of recipes when the upstream layers they require aren't present.

Milestones

Milestone 1:

ROS 2 with webOS OSE on Beanbird Bot (ROS 2 bouncy and OpenEmbedded morty)

  • Use the latest [devel] branch of build-ros2-lgsvl.git to select the commits to build from for the layers other than meta-ros2.

  • Use bitbake-layers show-overlayed to find bbclass-s and recipes for platform packages in meta-ros2 that are no longer needed because they are now provided by OpenEmbedded morty. Don't remove python-pyyaml_3.11.bb as morty doesn't provide a recipe for python3-pyyaml.

  • Use superflore to generate recipes for bouncy from http://repo.ros2.org/rosdistro_cache/bouncy-cache.yaml.gz.

  • Create a new branch [morty-superflore-bouncy-beanbirdbot] from [devel] of https://github.com/lgsvl/meta-ros2. git mv recipes-{ros1,ros2}/<ROS_SUPERFLORE_GENERATED_RECIPES_FOR_COMPONENTS> to v1-recipes-{ros1,ros2}. Use bitbake-layers show-recipes --multiple to ensure that there are no duplicated recipes: it uncovered that the recipes for these components that appear in ROS_SUPERFLORE_GENERATED_RECIPES_FOR_COMPONENTS are in different subdirectories from recipes-{ros1,ros2}/<COMPONENT>:

    COMPONENT Subdirectory
    angles recipes-ros1/geometry-angles-utils
    common-interfaces recipes-ros2/common-interfaces/ros2-*
    demos recipes-ros2/ros2-demos
    ecl-core recipes-ros1/ecl-navigation
    launch recipes-ros2/ros2-launch
    urdf recipes-ros1/robot-model

    Also create a new branch [morty-superflore-bouncy-beanbirdbot] from [devel] of https://github.com/lgsvl/meta-ros2-lgsvl, as changes must be made to it to work with the generated recipes. Use these branches when building webos-image from [devel] of https://github.com/lgsvl/build-ros2-lgsvl (Beanbird Bot).

  • Drop the ros2- prefixes from the packages in RDEPENDS_${PN} in packagegroup-ros2-*.bb because the recipes that superflore generates do not have them.

  • Migrate patches, etc. from v1-recipes-{ros1,ros2} into the appropriate .inc file under includes-{ros1,ros2}. Create other .inc files where necessary, especially for build tools. Only do this for the packages needed for webos-image.

  • Set LAYERSERIES_COMPAT_ros-layer to morty as it will be the only OpenEmbedded release series with which the bouncy meta-ros will be tested.

  • Enhance mcf to recognize an optional BblayersConfExtraLines variable in weboslayers.py that will contain a tuple of strings to be appended to the conf/bblayers.conf that it generates.

  • Done when the image for Beanbird Bot works.

  • COMPLETED

Milestone 2:

ROS Core on qemux86 and raspberrypi3 (ROS 2 crystal and OpenEmbedded morty)

  • Use bitbake-layers show-overlayed to find bbclass-s and recipes for platform packages in meta-ros that are no longer needed because they are now provided by OpenEmbedded morty.

  • Upgrade the recipes included in meta-ros for the platform packages needed for crystal to the versions found in Ubuntu bionic and files/<ROS_DISTRO>/newer-platform-components.list.

  • Add entries to base.yaml or python.yaml for platform packages whose mappings in files/crystal/rosdep-resolve.yaml do not contain @<OE-LAYER>.

  • Create recipes for the platform packages required for crystal that do not appear in https://layers.openembedded.org/layerindex/branch/morty/recipes/. Copy recipes from layers that only supply one into meta-ros, ie, meta-sca, meta-virtualization, and meta-openstack. Add them to the appropriate recipes-* tree. Build the versions that appear in Ubuntu bionic and files/<ROS_DISTRO>/newer-platform-components.list.

  • Use superflore to generate recipes for melodic, bouncy, crystal, and dashing. This will allow the next step to operate on all of the manually created recipes that will eventually have generated recipes.

  • git mv the subdirectories recipes-{ros1,ros2}/<ROS_SUPERFLORE_GENERATED_RECIPES_FOR_COMPONENTS> for each of the four ROS_DISTROs. Use bitbake-layers show-recipes --multiple to ensure that there are no duplicated recipes: it uncovered that the recipes for these components that appear in ROS_SUPERFLORE_GENERATED_RECIPES_FOR_COMPONENTS are in different subdirectories from recipes-{ros1,ros2}/<COMPONENT>:

    COMPONENT Subdirectory
    angles recipes-ros1/geometry-angles-utils
    ar-track-alvar recipes-ros1/{ar-track-alvar,ar-track-alvar-msgs}
    bfl recipes-extended/bfl
    common-interfaces recipes-ros2/common-interfaces/ros2-*
    demos recipes-ros2/ros2-demos
    ecl-core recipes-ros1/{ecl-core,ecl-navigation}
    launch recipes-ros2/ros2-launch
    message-filters recipes-ros1/ros-comm/message-filters* (single package)
    ompl recipes-extended/ompl
    orocos-kinematics-dynamics recipes-extended/kdl
    urdf recipes-ros1/robot-model

  • If a package that previously had a manually created recipe now has a generated one, add inherit ros_recipe_now_generated to its manually created recipe. Use bitbake-layers show-recipes --multiple to find these packages. Move any remaining recipes in recipes-{ros1,ros2} to unused-recipes-{ros1,ros2} except those in recipes-{ros1,ros2}/packagegroups, which are moved to recipes-core/packagegroups.

  • Drop the ros2- prefixes from the packages in RDEPENDS_${PN} in packagegroup-ros2-*.bb because the recipes that superflore generates do not have them.

  • Determine if any of the existing PR-s from https://github.com/bmwcarit/meta-ros should be applied.

  • Examine the additions to [devel] of https://github.com/lgsvl/meta-ros2 since it was branched from from [master] of https://github.com/bmwcarit/meta-ros, cherry-picking any that are still needed.

  • Cherry-pick the changes from [morty-superflore-bouncy-beanbirdbot], confirming that they are still needed. Update any <PV>-specific files for the new version of ROS packages in crystal.

  • Migrate patches, etc. from v1-recipes-{ros1,ros2} into the appropriate .inc file under includes-{ros1,ros2} for packages added to crystal since bouncy. Create other .inc files where necessary, especially for new build tools. Only do this for packages needed for core-image-ros-roscore.

  • Confirm that all of the builds of the *-vendor packages do the right thing.

  • Prefix the setup scripts provided by ros-workspace with ros_ and install them under /usr/bin.

  • Done when the core-image-ros-world images for qemux86 and raspberrypi3 boot and can perform these basic ROS 2 actions:

      # /etc/init.d/networking start     # Without a network connection, there's no output from "ros2 topic echo /chatter"
  # ping lge.com
  # source ros_setup.sh
  # ros2 topic list
  /parameter_events
  /rosout

  # ros2 msg list
  ...

  # (sleep 5; ros2 topic pub /chatter std_msgs/String "data: Hello world") &
  # ros2 topic echo /chatter
  publisher: beginning loop
  publishing #1: std_msgs.msg.String(data='Hello world')

  data: Hello world

  publishing #2: std_msgs.msg.String(data='Hello world')

  data: Hello world

  • COMPLETED

Milestone 3:

ROS World with webOS OSE on qemux86 and raspberrypi3 (ROS 2 crystal and OpenEmbedded morty)

  • Use the absence of DISTRO_FEATURES values to conditionally remove the "optional" packages associated with Gazebo, Navigation2, Mobile Robot Programming Toolkit 2, RealSense, Qt, rqt, and RViz from packagegroup-ros-world. Adding the value for BehaviorTree.CPP v3 to DISTRO_FEATURES removes the Navigation2 packages because they require v2 of BehaviorTree.CPP . Do this in recipes-core/packagegroups/packagegroup-ros-world-<ROS_DISTRO>.inc. Conditionalize the packages in packagegroup-ros-world that depend on mesa to require opengl in DISTRO_FEATURES.
  • For the remainder of the packages that are in packagegroup-ros-world when no DISTRO_FEATURES are enabled, create any necessary patches and add them and any other metadata to <BPN>-<PV>.inc under includes-{ros1,1ros2}. Create other .inc-s where necessary.
  • Remove any licenses under licenses that are now supplied upstream.
  • Remove any scripts under scripts and lib that are no longer applicable.
  • Done when core-image-ros-world (DISTRO = "ros2") and webos-image-ros-world (DISTRO = "webos") build successfully for qemux86 and raspberrypi3 and the basic ROS 2 actions shown in Milestone 2 run successfully on them. Additionally, when running webos-image-ros-world, the webOS OSE menu must appear after pressing the Windows key and allow the settings for Network to be launched. Note that webOS OSE automatically starts networking; issuing /etc/init.d/networking start is unnecessary.
  • COMPLETED

Milestone 4:

ROS World with webOS OSE on qemux86 and raspberrypi3 (ROS 2 crystal and OpenEmbedded thud)

  • Set LAYERSERIES_COMPAT_ros-layer to thud as it will be the only OpenEmbedded release series with which the crystal meta-ros will continue to be tested.
  • Add dashing to ROS_DISTRO_COMPAT.
  • Drop the recipes for platform packages now provided by thud and revert the import of recipes done for morty that are no longer needed. Use bitbake-layers show-recipes --multiple to find these recipes.
  • Update files/*-crystal.mcf to fetch from the thud branches.
  • Done when core-image-ros-world (DISTRO = "ros2") and webos-image-ros-world (DISTRO = "webos") build successfully for qemux86 and raspberrypi3 and the basic ROS 2 actions shown in Milestone 2 run successfully on them. Additionally, when running webos-image-ros-world, the webOS OSE menu must appear after pressing the Windows key and allow the settings for Network to be launched. Note that webOS OSE automatically starts networking; issuing /etc/init.d/networking start is unnecessary.
  • COMPLETED

Milestone 5:

ROS World with webOS OSE on qemux86 and raspberrypi3 (ROS 2 dashing and OpenEmbedded thud)

  • Add entries to base.yaml or python.yaml for platform packages whose mappings in files/dashing/rosdep-resolve.yaml have names that do not contain @<OE-LAYER>.
  • Update any <PV>-specific files for the new version of ROS packages in dashing.
  • For new packages added since crystal, create any necessary patches and add them and any other metadata to <BPN>-<PV>.inc under includes-{ros1,ros2}. Create other .inc, where necessary, especially for new build tools.
  • If a package that previously had a manually created recipe now has a generated one, add inherit ros_recipe_now_generated to its manually created recipe. Use bitbake-layers show-recipes --multiple to find these packages.
  • For existing packages whose versions have changed, determine whether to create copies of the existing <PV>-specific .inc files for the new version.
  • Create recipes for the platform packages newly required by dashing that do not appear in https://layers.openembedded.org/layerindex/branch/thud/recipes/. Add them to the appropriate recipes-* tree. Build the versions that appear in Ubuntu bionic and files/dashing/newer-platform-components.list.
  • git rm -r v1-recipes-ros1 v1-recipes-ros2 unused-recipes-ros1
  • Done when core-image-ros-world (DISTRO = "ros2") and webos-image-ros-world (DISTRO = "webos") build successfully for qemux86 and raspberrypi3 and the basic ROS 2 actions shown in Milestone 2 run successfully on them. Additionally, when running webos-image-ros-world, the webOS OSE menu must appear after pressing the Windows key and allow the settings for Network to be launched. Note that webOS OSE automatically starts networking; issuing /etc/init.d/networking start is unnecessary.
  • COMPLETED 2019-04-17

Milestone 6:

ROS Core with webOS OSE on qemux86 and raspberrypi3 (ROS 1 melodic and OpenEmbedded thud)

  • Decide whether the ROS packages should installed under /opt/ros/<ROS_DISTRO> or folded into the rest of the OS under /usr -- see Issue #3. Until this is decided. ROS 1 distros will install them under /opt/ros/<ROS_DISTRO> and ROS 2 distros will continue to install them under /usr.

  • Set LAYERSERIES_COMPAT_ros-layer to thud as it will be the only OpenEmbedded release series with which the melodic meta-ros will be tested.

  • Add melodic to ROS_DISTRO_COMPAT.

  • Don't support EusLisp and Lisp => add Lisp-related packages to the skip list for melodic (and kinetic).

  • Temporarily add settings for the ROS_UNRESOLVED_PLATFORM_PKG_* variables to conf/ros-distro/include/melodic/ros-distro.inc until bitbake core-image-core parses successfully. Use the OpenEmbedded Layer Index to see which recipes are provided by the thud branches of the layers in open-embedded-core and meta-openembedded. Import recipes that are found in other branches or layers. Create recipes for the remainder. Where possible, use the version of the component that appears in Ubuntu bionic-updates; if not, record the differences in the version differences tables.

  • Add entries to base.yaml or python.yaml for platform packages for which ROS_UNRESOLVED_PLATFORM_PKG_* settings were added to conf/ros-distro/include/melodic/ros-distro.inc. Remove the settings, re-run ros-generate-recipes, and confirm that bitbake core-image-ros-roscore parses successfully. Create a PR for the base.yaml and python.yaml changes.

  • If a package that previously had a manually created recipe now has a generated one, add inherit ros_recipe_now_generated to its manually created recipe. Use bitbake-layers show-recipes --multiple after configuring with ros1-melodic.mcf to find these packages.

  • Initially blacklist all packages that appear with different versions in multiple layers in the appropriate ros-distro*-recipe-blacklist.inc file. Use bitbake-layers show-recipes --multiple after configuring with webos-melodic.mcf to find these packages. When a dependency on a blacklisted package is found, select the version of it to use, make an entry in ros-distro*-preferred-versions.inc, and remove the entry from ros-distro*-recipe-blacklist.inc.

  • Update the versions of recipes under recipes-devtools/python if any have changed in files/melodic/newer-platform-components.list.

  • Migrate patches, etc. from [legacy] into the appropriate .inc file under includes-ros1. Create other .inc files where necessary, especially for build tools. Only do this for the packages needed for core-image-ros-roscore.

  • Create any necessary patches and add them and any other metadata to <BPN>-<PV>.inc under includes-ros1. Create other .inc, where necessary, especially for new build tools.

  • Update catkin.bbclass and ros_opt_prefix.bbclass for meta-ros layer version 2.

  • Add connman to the images when DISTRO = "ros1" or DISTRO = "ros2" so that networking will start automatically when booting. Drop issuing /etc/init.d/networking start in the ROS sanity tests.

  • Re-run ros-generate-recipes for crystal and dashing, replacing the existing commits. (Recipes generated from the latest upstream dashing-cache.yaml do not yet build because of missing dependencies, so use the existing files/dashing/cache.yaml for the 2019-06-14 release instead.)

  • Drop all support for bouncy as its EOL has passed.

  • Done when the core-image-ros-roscore (DISTRO = "ros1") and webos-image-ros-roscore (DISTRO = "webos") images for qemux86 and raspberrypi3 boot and can perform these basic ROS 1 actions:

      # ping lge.com
  # source /opt/ros/melodic/setup.sh
  # roscore &
  ...
  started core service [/rosout]
  # rosnode list
  /rosout
  # rosnode info /rosout
  --------------------------------------------------------------------------------
  Node [/rosout]
  Publications:
  * /rosout_agg [rosgraph_msgs/Log]

  Subscriptions:
  * /rosout [unknown type]

  Services:
  * /rosout/get_loggers
  * /rosout/set_logger_level


  contacting node http://127.0.0.1:34643/ ...
  Pid: 1363

    (from http://wiki.ros.org/ROS/Tutorials/UnderstandingNodes). Additionally, when running webos-image-ros-roscore, the webOS OSE menu must appear after pressing the Windows key and allow the settings for Network to be launched.

  • COMPLETED 2019-06-27

Milestone 7:

TurtleBot 3 Waffle Pi (using model B not B+) with webOS OSE (ROS 1 melodic and OpenEmbedded thud)

  • Split off the generation of files/<ROS_DISTRO>/cache.yaml into own script ros-generate-cache.
  • Change the name of the branch created during recipe generation from superflore/<DATETIME> to superflore/<ROS_DISTRO>/<RELEASE-YYYYMMDD>.
  • Introduce new images core-image-ros-turtlebot3-core and webos-image-ros-turtlebot3-core that contain the packages required for core TurtleBot 3 operation listed here and here. Introduce a turtlebot3-remote-roscore value for IMAGE_FEATURES that adds turtlebot3-bringup instead of turtlebot3 to IMAGE_INSTALL.
  • Introduce new images core-image-ros-turtlebot3-all and webos-image-ros-turtlebot3-all that contain all the non-simulation related turtlebot3- prefixed packages in ROS 1 melodic and their prerequisities mentioned in the TurtleBot3 e-Manual.
  • Temporarily add settings for the ROS_UNRESOLVED_PLATFORM_PKG_* variables to conf/ros-distro/include/melodic/ros-distro.inc until bitbake core-image-ros-turtlebot3-all parses successfully. Use the OpenEmbedded Layer Index to see which recipes are provided by the thud branches of the layers in open-embedded-core and meta-openembedded. Import recipes that are found in other branches or layers. Create recipes for the remainder. Where possible, use the version of the component that appears in Ubuntu bionic-updates; if not, record the differences in the version differences tables.
  • Add entries to base.yaml, python.yaml, and ruby.yaml for the platform packages for which ROS_UNRESOLVED_PLATFORM_PKG_* settings were added to conf/ros-distro/include/melodic/ros-distro.inc. Remove the settings, re-run ros-generate-recipes, and confirm that bitbake core-image-ros-turtlebot3-all parses successfully. Create a PR for the *.yaml changes.
  • Re-run ros-generate-recipes for melodic, replacing the existing commits.
  • Now that [morty-draft] is gone, drop all of the morty-related commits from [thud-draft].
  • Introduce a scheme under webOS OSE for per-device configuration using a USB flash drive.
  • Navigation2 now builds for crystal => drop having its inclusion in -world being conditional on ros-behaviortree-cpp-v3 not appearing in DISTRO_FEATURES and unconditionally exclude ros-behaviortree-cpp-v3.
  • Done when core-image-ros-turtlebot3-all (DISTRO = "ros1") and webos-image-ros-turtlebot3-all (DISTRO = "webos") build successfully for raspberrypi3 and both the basic ROS 1 actions shown in Milestone 6 and the TurtleBot 3 bringup instructions run successfully on them. (If [WARN] [<timestamp>]: Failed to get param: timeout expired is output, restart and try again.) Additionally, when running webos-image-ros-turtlebot3-all, the webOS OSE menu must appear after pressing the Windows key and allow the settings for Network to be launched.
  • COMPLETED 2019-09-18

Milestone 8:

Implement layer version 3 phase 1

  • Implement the items under Phase 1 in the Layer Version 3 section.
  • Update this document with the file layout and content changes for layer version 3.
  • Update https://github.com/ros/meta-ros/wiki/OpenEmbedded-Build-Instructions with the new image names.
  • Update superflore to implement ROS_SUPERFLORE_GENERATION_SCHEME 2 and submit PR-s for the changes.
  • Create a new branch [legacy-v2], whose files/*.mcf fetch from [legacy-v2], from the existing [thud-draft] branch. Its files will not be changed nor its commits rebased.
  • Move scripts/mcf and files/*.mcf into a new independently-rooted branch [build-draft]. Give the .mcf file basenames a -thud suffix.
  • Add a rebaseable [master-draft] that is coincident with [thud-draft].
  • Done when:
    • ros-image-world (DISTRO = "ros2") and webos-image-ros-world (DISTRO = "webos") build successfully for qemux86 and raspberrypi3 for crystal, and dashing;
    • ros-image-turtlebot3-all (DISTRO = "ros1") and webos-image-ros-turtlebot3-all (DISTRO = "webos") build successfully for raspberrypi3 for melodic;
    • The images pass the sanity tests.
  • COMPLETED 2019-12-11

Milestone 9:

Support ROS 2 eloquent

  • Use the initial release from 2019-11-22 (see ROS 2 Eloquent Elusor
  • Upgrade webOS OSE a bit from build 84 to 91 so that it doesn't use the old libmpc from meta-gplv2. This allows many components to be unblacklisted which weren't available because of that (most importantly, python3-cryptography, used by sros2 in eloquent).
  • Update the pins for the OpenEmbedded repos in files/ros*.mcf to match those now used in files/webos-*.mcf for webOS OSE build 91.
  • Done when:
    • ros-image-world (DISTRO = "ros2") builds successfully for qemux86 and raspberrypi3 for eloquent;
    • webos-image-ros-world (DISTRO = "webos") builds successfully for qemux86 and raspberrypi3 for crystal, dashing, and eloquent;
    • webos-image-ros-turtlebot3-all (DISTRO = "webos") builds successfully for raspberrypi3 for melodic;
    • The images pass the sanity tests.
  • COMPLETED 2019-12-19

Milestone 10:

Support December 2019 releases of melodic, crystal, dashing, and eloquent

  • Generate recipes for the latest melodic release (2019-12-10) and update its .bbappend-s so that it builds. Add support for qemux86.
  • Generate recipes for the latest crystal release (2019-12-12) and update its .bbappend-s so that it builds.
  • Generate recipes for the latest dashing release (2019-12-12) and update its .bbappend-s so that it builds.
  • Generate recipes for the latest eloquent release (2019-12-14) and update its .bbappend-s so that it builds.
  • Move the recipes for components found in files/<ROS_DISTRO>/newer-platform-components.list to meta-ros-common/recipes-infrastructure.
  • Update the recipes for new versions of the infrastructure packages found in newer-platform-components.list.
  • Have ros-generate-cache.sh write its version to the line 1 comments of cache.* and have ros-generate-recipes.sh write its version to a variable ROS_GENERATE_RECIPES_PROGRAM_VERSION appended to superflore-ros-distro.inc.
  • Done when:
    • ros-image-{core,world} (DISTRO = "ros2") and webos-image-ros-{core,world} (DISTRO = "webos") build successfully for qemux86 and raspberrypi3 for crystal, dashing, and eloquent;
    • ros-image-{core,turtlebot3-all} (DISTRO = "ros1") and webos-image-ros-{core,turtlebot3-all} (DISTRO = "webos") build successfully for qemux86 and raspberrypi3 for melodic;
    • The images pass the sanity tests.
  • COMPLETED 2020-01-07

Milestone 11:

Support newer OpenEmbedded release series and webOS OSE v2.2.0

  • Update to webOS OSE v2.2.0, build 143. Add configuration for its swupdate feature to meta-ros-webos/files/examples/webos-device-config/v1/rc.local.
  • Add support for warrior, zeus, and the currently under development dunfell OpenEmbedded release series for DISTRO = "ros1" and DISTRO = "ros2" only; webOS OSE is not supported. Create [warrior-draft] and [zeus-draft]; add dunfell support in [master-draft].
  • Publish the support policies for OpenEmbedded release series and MACHINE-s on a Policies wiki page.
  • Update Package Version Differences for the platform packages provided by warrior, zeus, and dunfell (as of Milestone 11).
  • Update the tables in REP-2000 to show the versions for OpenEmbedded zeus instead of thud. Mention that multiple OpenEmbedded release series are supported and provide a link to Package Version Differences.
  • Change the policy in warrior and later for selecting the version of the kernel to build: instead of choosing it based on that provided by ROS_DISTRO_BASELINE_PLATFORM (currently ubuntu-bionic), use the default version provided by the BSP layer for the OpenEmbedded release series being used. Note that webOS OSE sets the preferred version to whatever works with it.
  • Change the repo used to determine the OpenEmbedded version portion of DISTRO_VERSION for DISTRO = "ros1" and DISTRO = "ros2" from git.yoctoproject.org/poky.git to github.com/openembedded/openembedded-core.git.
  • Drop support for ROS 2 crystal because it's now past its EOL:
    • Move build/files/*-crystal-*.mcf to build/files-contrib.
    • Drop meta-ros2-crystal as first commit for dunfell. Backport to the other branches once they are no longer rebaseable in Milestone 13 [now Milestone 12].
    • Drop build/files-contrib/*-crystal-*.mcf as last commit pushed to [build-draft].
  • Done when:
    • ros-image-{core,world} (DISTRO = "ros2") build successfully for their supported MACHINES-s for dashing, and eloquent with each of thud, warrior, zeus, and dunfell;
    • webos-image-ros-{core,world} (DISTRO = "webos") build successfully for their supported MACHINES-s for dashing, and eloquent with thud;
    • ros-image-{core,turtlebot3-all} (DISTRO = "ros1") build successfully for their supported MACHINES-s for melodic with each of thud, warrior, zeus, and dunfell;
    • webos-image-ros-{core,turtlebot3-all} (DISTRO = "webos") build successfully for their supported MACHINES-s for melodic with thud;
    • The images pass the sanity tests.
  • COMPLETED 2020-02-07

Milestone 12:

Support OpenEmbedded dunfell, webOS OSE v2.5.0, and Q1 2020 releases of melodic, dashing, and eloquent

  • OpenEmbedded dunfell has been released. Finalize support for building all current ROS distros with it. [master-draft] now targets the next OpenEmbedded release under active development: gatesgarth.
  • Update to webOS OSE v2.5.0, build 197.
  • Generate recipes for the March melodic release (2020-03-22) and update its .bbappend-s so that it builds.
  • Generate recipes for the March dashing release (2020-03-19) and update its .bbappend-s so that it builds.
  • Generate recipes for the February eloquent release (2020-02-20) and update its .bbappend-s so that it builds.
  • Split off [thud-draft], [warrior-draft], [zeus-draft], and [dunfell-draft] from being along [master-draft]. Changes common to all OpenEmbedded release series must now be applied to all five branches.
  • Change how packages that do not build are excluded from the the *-image-*-world images. Instead of passing to superflore a list of packages for which not to generate recipes and using values in DISTRO_FEATURES to control removal of additional groups of packages, have superflore generate recipes for all packages in cache.yaml and use values in a new variable ROS_WORLD_SKIP_GROUPS to remove groups of packages based the reason(s) they don't build, including the absence of auxililary metadata layers and values defined by OpenEmbedded in DISTRO_FEATURES. Drop all other ros- DISTRO_FEATURES values.
  • Use conditional PNBLACKLIST-s based on the values in ROS_WORLD_SKIP_GROUPS to exclude recipes for packages that do not build from bitbake world.
  • Split off a number of items under Phase 2 in the Layer Version 3 section into a new Phase 3 to be implemented in Milestone 14 [now split again to Phase 3 and Phase 4, the former to be implemented in Milestone 15]. Implement those that remain in Phase 2.
  • Change the policy for selecting versions of platform packages provided by OpenEmbedded: instead of attempting to match the versions provided by ROS_DISTRO_BASELINE_PLATFORM (ubuntu-bionic), use the default version provided by the OpenEmbedded release series being used and drop the meta-ros-common/conf/ros-distro/include/ubuntu-bionic subdirectory. Note that webOS OSE sets the preferred version to whatever works with it. When importing a recipe from another OpenEmbedded layer or creating a new recipe, use the latest version available at the time of import or creation, unless the ROS package(s) that are dependent on it require a different one in order to build. Update the Policies wiki page for this change.
  • Introduce a ROS_OE_RELEASE_SERIES setting in meta-ros-common/conf/ros-distro/ros-distro.conf and modify the conf/layer.conf-s to use it to set LAYERSERIES_COMPAT_<LAYER>. Also modify the example conf/local.conf to use it instead of MCF_OE_RELEASE_SERIES but don't drop setting MCF_OE_RELEASE_SERIES in build/files/*.mcf until Milestone 14 [now Milestone 15].
  • Stop considering the branches to be rebaseable:
    • Finalize the commit messages:
      • Examine those with DRAFT, SQUASH, WIP, and TEMP summary line prefixes.
      • Add missing {<ROS_DISTRO>} summary line prefixes.
      • Remove [<NAME>] and ((pr/*)) summary lines prefixes.
      • Shorten summary lines whose length(git log --oneline) > 130.
      • Fix up those containing XXX.
      • etc.
    • Replace the commits that change LAYERSERIES_COMPAT_<LAYER> with ones that change ROS_OE_RELEASE_SERIES.
    • Add commits that set ROS_DISTRO_METADATA_MILESTONE_* in meta-ros-common/conf/ros-distro/milestone.conf and remove the {{milestones/N}} summary line prefix from existing commits.
    • Pin the commits of meta-ros and meta-ros-webos in build/files/*.mcf.
    • Create branches without the -draft suffix coincident with the -draft suffixed ones and change OpenEmbedded Build Instructions to reference them. Update the README.md-s along each branch for the new branch names.
    • Remove meta-ros2-crystal as the first commit of [thud], [warrior], and [zeus] branches after they are split off from being along [master].
  • Done when:
    • bitbake world builds successfully for the supported MACHINES-s for melodic, dashing, and eloquent with each of thud, warrior, zeus, and dunfell;
    • ros-image-{core,world} (DISTRO = "ros2") build successfully for their supported MACHINES-s for dashing, and eloquent with each of thud, warrior, zeus, and dunfell;
    • webos-image-ros-{core,world} (DISTRO = "webos") build successfully for their supported MACHINES-s for dashing, and eloquent with thud;
    • ros-image-{core,turtlebot3-all,world} (DISTRO = "ros1") build successfully for their supported MACHINES-s for melodic with each of thud, warrior, zeus, and dunfell;
    • webos-image-ros-{core,turtlebot3-all,world} (DISTRO = "webos") build successfully for their supported MACHINES-s for melodic with thud;
    • The images pass the sanity tests.
  • COMPLETED 2020-07-01

Milestone 13:

Support ROS 2 foxy and OpenEmbedded gatesgarth

  • Add support for foxy using the initial release from 2020-06-05 (see ROS 2 Foxy Fitzroy only building DISTRO = "ros2" with OpenEmbedded dunfell and gatesgarth.
  • Generate recipes for the second June foxy release (2020-06-25) and update its .bbappend-s so that it builds.
  • Subsequently, add support for foxy with the remaining OpenEmbedded release series and DISTRO = "webos". Tag the [build] branch with build/milestones/13-all.
  • Keep the -draft suffixed branches coincident with the unsuffixed ones.
  • Keep the pins to the HEAD commits of the upstream OpenEmbedded layers current in build/files/*.mcf.
  • Done when:
    • bitbake world builds successfully for the supported MACHINES-s for foxy with each of thud, warrior, zeus, dunfell, and gatesgarth;
    • ros-image-{core,world} (DISTRO = "ros2") build successfully for their supported MACHINES-s for foxy with each of thud, warrior, zeus, dunfell, and gatesgarth;
    • webos-image-ros-{core,world} (DISTRO = "webos") build successfully for their supported MACHINES-s for foxy with thud;
    • The images pass the sanity tests.
  • COMPLETED 2020-07-04

Milestone 14:

Support mid-year 2020 ROS distro releases (including ROS 2 rolling) with dunfell and gatesgarth, and support webOS OSE v2.6.0

  • Generate recipes for these ROS distro releases and update their .bbappend-s so that they build DISTRO = "ros1" or DISTRO = "ros2" with dunfell and gatesgarth only:
    • melodic 2020-04-22
    • melodic 2020-05-28
    • melodic 2020-06-12
    • melodic 2020-07-07
    • melodic 2020-08-07
    • dashing 2020-05-29
    • dashing 2020-08-06
    • eloquent 2020-05-29
    • eloquent 2020-08-13
    • foxy 2020-07-10
    • foxy 2020-07-23
    • foxy 2020-08-07
  • Add support for rolling using a snapshot from 2020-07-29 and only build DISTRO = "ros2" with only dunfell and gatesgarth.
  • Update to webOS OSE v2.6.0, build 218, but only for the ROS distro releases as of Milestone 13.
  • Keep the pins to the HEAD commits of the upstream OpenEmbedded layers current in build/files/*.mcf.
  • Drop the -draft suffixed branches.
  • Introduce -next suffixed branches. They are used in the same way as the -next suffixed branches in other OpenEmbedded layers: to hold commits that are pending being merged into their corresponding unsuffixed branch, but whose history can be rewritten before this happens.
  • Allow the use of rpm as the OpenEmbedded package manager.
  • Done when:
    • bitbake world builds successfully for the supported MACHINES-s for melodic, dashing, eloquent, and foxy with each of thud, warrior, zeus, dunfell, and gatesgarth;
    • bitbake world builds successfully for the supported MACHINES-s for rolling with dunfell and gatesgarth;
    • ros-image-{core,world} (DISTRO = "ros2") build successfully for their supported MACHINES-s for dashing, eloquent. and foxy with each of thud, warrior, zeus, dunfell, and gatesgarth;
    • ros-image-{core,world} (DISTRO = "ros2") build successfully for their supported MACHINES-s for rolling with dunfell and gatesgarth;
    • webos-image-ros-{core,world} (DISTRO = "webos") build successfully for their supported MACHINES-s for dashing, eloquent, and foxy with thud;
    • ros-image-{core,turtlebot3-all,world} (DISTRO = "ros1") build successfully for their supported MACHINES-s for melodic with each of thud, warrior, zeus, dunfell, and gatesgarth;
    • webos-image-ros-{core,turtlebot3-all,world} (DISTRO = "webos") build successfully for their supported MACHINES-s for melodic with thud;
    • The images pass the sanity tests.
  • COMPLETED 2020-09-08

Milestone 15:

Support webOS OSE v2.8.0 and OpenEmbedded hardknott, drop/reduce support for OpenEmbedded thud, warrior, and zeus

  • Update to webOS OSE v2.8.0, build 263, which is built with OpenEmbedded dunfell instead of thud.
  • With OpenEmbedded thud no longer used by webOS OSE and warrior, and zeus no longer at the Stable Support Level, drop or reduce support for them as per Policies.
  • OpenEmbedded gatesgarth has been released. Finalize support for building all current ROS distros with it. [master] now targets the next OpenEmbedded release under active development: hardknott.
  • Update Package Version Differences for the changes introduced by webOS OSE v2.8.0 and for newer packages from the upstream OpenEmbedded layers.
  • Generate recipes for the new ROS distro releases since Milestone 14 and update their .bbappend-s so that they build with dunfell, gatesgarth, and hardknott:
    • melodic 2020-08-26
    • melodic 2020-09-15
    • melodic 2020-10-12
    • dashing 2020-10-27
    • foxy 2020-08-27
    • foxy 2020-09-10
    • foxy 2020-09-12
    • foxy 2020-10-09
    • foxy 2020-11-05
    • rolling 2020-10-12
  • Use a new version (0.3.2-19-g4105ad8) of superflore to generate the recipes for last of the ROS distro releases. This version:
    • improves the accuracy of the LICENSE values generated from the values found in package.xml, especially for those which are a variant of GPL version 3 (which had been misclassified as being GPL version 2);
    • correctly parses conditional <build_type> tags;
    • changes the ROS_UNRESOLVED_PLATFORM_PKG_ variable prefix to ROS_UNRESOLVED_DEP- because what's unresolved is not always a platform package and to prevent it from being interpreted as an override by bitbake, and changes the prefix of their generated values from UNRESOLVED- to ROS_UNRESOLVED_DEP- so as to make searching for where it's assigned easier;
    • hardcodes ROS_PYTHON_VERSION to be 2 for melodic and 3 otherwise.
  • Keep the pins to the HEAD commits of the upstream OpenEmbedded layers current in build/files/*.mcf.
  • Add entries to the base.yaml and python.yaml of ros/rosdistro.git for the platform packages for which ROS_UNRESOLVED_DEP-* settings were added to conf/ros-distro/include/<ROS_DISTRO>/ros-distro.inc. Remove most of the settings, re-run ros-generate-cache and ros-generate-recipes, and confirm that bitbake world parses successfully. Also update the mappings of the Python 2 platform packages in python.yaml to follow what's specified in this document, ie, change them to have a ${PYTHON_PN}- prefix so that they map to the packages for the version of Python specified by ROS_PYTHON_VERSION for the ROS distro. Create a PR for ros/rosdistro.git with the *.yaml changes.
  • Move all but one of the items in Phase 3 of Layer Version 3 to a new Phase 4. Implement the remaining item: eliminate the requirement to set ROS_DISTRO in conf/bblayers.conf, and hence, in build/files/*.mcf
  • Drop setting MCF_OE_RELEASE_SERIES in build/files/*.mcf.
  • Adjust the default BBFILE_PRIORITY_* settings in */conf/layer.conf so that layers are searched for recipes in the order of their dependency. This behavior will be more intuitive for those who aren't using mcf (which already arranges for this to happen).
  • Drop the .bbclass-s deprecated in Milestone 12.
  • Update COPYING.MIT with LG's copyright notice; drop CONTENTS.
  • Update .travis.yml so that it does somthing useful; drop .yottaci.yml and its assocated GitHub Webhook (https://bbcifunctions.azurewebsites.net/api/github_trigger (pull_request and push).
  • Now that ROS distro releases are being tagged with <ROS_DISTRO>/YYYY-MM-DD in ros/rosdistro.git (see Guidelines for gating a sync to main), modify ros-generate-cache and ros-generate-recipes to follow this convention and start including hyphens in the names meta-ros uses for ROS distro releases.
  • Done when:
    • ros-image-{core,world} (DISTRO = "ros1" or DISTRO = "ros2") build successfully for their supported MACHINES-s for melodic, dashing, eloquent, foxy, and rolling with dunfell, gatesgarth, and hardknott;
    • webos-image-ros-{core,world} (DISTRO = "webos") build successfully for their supported MACHINES-s for melodic, dashing, eloquent, foxy, and rolling with dunfell;
    • ros-image-turtlebot3-all (DISTRO = "ros1") builds successfully for its supported MACHINES-s for melodic with dunfell, gatesgarth, and hardknott;
    • webos-image-ros-turtlebot3-all (DISTRO = "webos") builds successfully for its supported MACHINES-s for melodic with dunfell;
    • bitbake world builds successfully for the supported MACHINES-s for melodic, dashing, eloquent, foxy, and rolling with dunfell, gatesgarth, and hardknott, or if it doesn't, the packages that fail to build are judged likely to be shortly fixed upstream and unlikely to be used by the ROS community;
    • the images pass the sanity tests;
    • a "best effort" has been made to have ros-image-{core,world} (DISTRO = "ros1" or DISTRO = "ros2") build successfully for their supported MACHINES-s for melodic, dashing, and foxy with zeus;
    • a "best effort" has been made to have ros-image-turtlebot3-all (DISTRO = "ros1") build successfully for its supported MACHINES-s for melodic with zeus.
  • COMPLETED 2020-12-23

Milestone 16:

Support webOS OSE v2.10.0, OpenEmbedded honister, ROS 1 noetic, ROS 2 galactic, and end of June 2021 releases of other ROS distros

  • Update to webOS OSE v2.10.0, build 325.
  • OpenEmbedded hardknott has been released. Finalize support for building all current ROS distros with it. [master] now targets the next OpenEmbedded release under active development: honister.
  • Update Package Version Differences for:
    • the changes introduced by webOS OSE v2.10.0,
    • newer packages from the upstream OpenEmbedded layers,
    • the release of hardknott by adding a honister column.
  • Add support for ROS 1 noetic (see ROS 1 Noetic Ninjemys with OpenEmbedded dunfell, gatesgarth, hardknott, and honister.
    • noetic 2020-10-09
    • noetic 2020-11-30
    • noetic 2020-12-14
    • noetic 2021-01-22
    • noetic 2021-02-08
    • noetic 2021-03-12
    • noetic 2021-05-04
    • noetic 2021-05-24
    • noetic 2021-05-28
  • Add support for ROS 2 galactic (see ROS 2 Galactic Geochelone with OpenEmbedded dunfell, gatesgarth, hardknott, and honister.
  • Generate recipes for the new releases of other ROS distro releases since Milestone 15 and update their .bbappend-s so that they build with dunfell, gatesgarth, hardknott, and honister:
    • melodic 2020-11-11
    • melodic 2020-12-07
    • melodic 2020-12-21
    • melodic 2021-01-26
    • melodic 2021-02-04
    • melodic 2021-03-18
    • melodic 2021-04-47
    • melodic 2021-05-17
    • melodic 2021-06-09
    • dashing 2020-12-02
    • dashing 2021-01-14
    • dashing 2021-02-04
    • dashing 2021-03-04
    • dashing 2021-06-03 (final release)
    • eloquent 2020-12-14 (final release)
    • foxy 2020-12-11
    • foxy 2021-01-07
    • foxy 2021-02-18
    • foxy 2021-04-16
    • foxy 2021-04-26
    • foxy 2021-06-03
    • rolling 2020-12-19
    • rolling 2021-02-19
    • rolling 2021-03-17
    • rolling 2021-04-09
    • rolling 2021-06-08
  • Keep the pins to the HEAD commits of the upstream OpenEmbedded layers current in build/files/*.mcf.
  • Done when:
    • ros-image-{core,world} (DISTRO = "ros1" or DISTRO = "ros2") build successfully for their supported MACHINES-s for melodic, noetic, dashing, foxy, galactic, and rolling with dunfell, gatesgarth, hardknott, and honister.
    • webos-image-ros-{core,world} (DISTRO = "webos") build successfully for their supported MACHINES-s for melodic, noetic, dashing, foxy, galactic, and rolling with dunfell;
    • ros-image-turtlebot3-all (DISTRO = "ros1") builds successfully for its supported MACHINES-s for melodic with dunfell, gatesgarth, hardknott, and honister;
    • webos-image-ros-turtlebot3-all (DISTRO = "webos") builds successfully for its supported MACHINES-s for melodic with dunfell;
    • bitbake world builds successfully for the supported MACHINES-s for melodic, noetic, dashing, foxy, galactic, and rolling with dunfell, gatesgarth, hardknott, and honister, or if it doesn't, the packages that fail to build are judged likely to be shortly fixed upstream and unlikely to be used by the ROS community;
    • the images pass the sanity tests;
    • a "best effort" has been made to have ros-image-{core,world} (DISTRO = "ros1" or DISTRO = "ros2") build successfully for their supported MACHINES-s for eloquent with dunfell, gatesgarth, hardknott, and honister.
    • a "best effort" has been made to have webos-image-ros-{core,world} (DISTRO = "webos") build successfully for their supported MACHINES-s for eloquent with dunfell;
    • a "best effort" has been made to have bitbake world build successfully for the supported MACHINES-s for eloquent with dunfell, gatesgarth, hardknott, and honister,
    • a "best effort" has been made to have ros-image-{core,world} (DISTRO = "ros1" or DISTRO = "ros2") build successfully for their supported MACHINES-s for melodic, dashing, and foxy with zeus;
    • a "best effort" has been made to have ros-image-turtlebot3-all (DISTRO = "ros1") build successfully for its supported MACHINES-s for melodic with zeus.
  • COMPLETED 2021-06-24

Milestone 17:

Support webOS OSE v2.14.0, OpenEmbedded kirkstone (as of December 2021), and mid-November 2021 releases of ROS distros (except rolling)

  • Update to webOS OSE v2.14.0, build 411.

  • OpenEmbedded honister has been released. Finalize support for building all current ROS distros with it. [master] now targets kirkstone. As of 2021-06-08, gatesgarth is EOL, so reduce support for it to "best effort" as per Policies. Drop support for zeus (this could have happened as far back as 2021-02).

  • Generate recipes for the new releases of current ROS distros since Milestone 16 until the middle of November 2021 and update their .bbappend-s so that they build with dunfell, hardknott, honister, and kirkstone:

    • melodic 2021-08-04
    • melodic 2021-09-27
    • melodic 2021-11-08
    • noetic 2021-06-25
    • noetic 2021-07-26
    • noetic 2021-08-13
    • noetic 2021-09-08
    • noetic 2021-09-27
    • noetic 2021-10-14
    • noetic 2021-11-14
    • foxy 2021-06-30
    • foxy 2021-08-04
    • foxy 2021-09-02
    • foxy 2021-10-07
    • foxy 2021-10-13
    • foxy 2021-11-04
    • galactic 2021-07-16
    • galactic 2021-08-25
    • galactic 2021-09-20
    • galactic 2021-11-11
    • rolling 2021-07-06
    • rolling 2021-07-15
    • rolling 2021-08-06
    • rolling 2021-08-19
    • rolling 2021-09-10
    • rolling 2021-09-30
    • rolling 2021-11-02
    • rolling 2021-11-11

  • Keep the pins to the HEAD commits of the upstream OpenEmbedded layers current in build/files/*.mcf until the beginning of December 2021, including adding support for the new BitBake override syntax. Note that pins of [master] for kirkstone are as of this date; a few more changes are needed to make it compatible with the final kirkstone released in April 2022 (eg, https://github.com/ros/meta-ros/pull/1009).

  • Add recipes to build colcon packages.

  • Update Package Version Differences for:

    • newer packages from the upstream OpenEmbedded layers as of 2021-08-27,
    • the release of hardknott by adding a honister column.

  • Change the MACHINES-s for which ROS 2 distros are built to be the 32-bit variants -- see Policies for the rationale.

  • Introduce mcf file format version 3 which:

    • adds a SupportLevel variable whose values are documented in Policies,
    • renames Distribution to OE_Distribution,
    • adds OE_Branch, Bitbake_Branch, and ROS_Distribution variables.

  • Update build/files*/*.mcf for the dropping of support for the git:// protocol by GitHub.

  • Done when:

    • ros-image-{core,world} (DISTRO = "ros1" or DISTRO = "ros2") build successfully for their supported MACHINES-s for melodic, noetic, foxy, galactic, and rolling with dunfell, hardknott, honister, and kirkstone (exception: rolling can no longer be built because the commits referenced by its recipes have been removed from the [release/rolling/*]branches of the release repositories);
    • webos-image-ros-{core,world} (DISTRO = "webos") build successfully for their supported MACHINES-s for melodic, noetic, foxy, galactic, and rolling with dunfell (exception: rolling can no longer be built because the commits referenced by its recipes have been removed from the [release/rolling/*]branches of the release repositories);
    • ros-image-turtlebot3-all (DISTRO = "ros1") builds successfully for its supported MACHINES-s for melodic with dunfell, hardknott, honister, and kirkstone;
    • webos-image-ros-turtlebot3-all (DISTRO = "webos") builds successfully for its supported MACHINES-s for melodic with dunfell;
    • bitbake world builds successfully for the supported MACHINES-s for melodic, noetic, foxy, galactic, and rolling with dunfell, hardknott, honister, and kirkstone, or if it doesn't, the packages that fail to build are judged likely to be shortly fixed upstream and unlikely to be used by the ROS community (NOT DONE);
    • the images pass the sanity tests (NOT DONE on the images for raspberrypi4, raspberrypi4-64 nor on any of the webos-image-ros-* images)
    • a "best effort" has been made to have ros-image-{core,world} (DISTRO = "ros1" or DISTRO = "ros2") build successfully for their supported MACHINES-s for melodic, noetic, foxy, galactic, and rolling with gatesgarth, but some of the packages that are used to create the images do not build;
    • a "best effort" has been made to have ros-image-{core,world} (DISTRO = "ros1" or DISTRO = "ros2") build successfully for their supported MACHINES-s for dashing and eloquent with dunfell, gatesgarth, hardknott, honister, and kirkstone, but some of the packages that are used to create the ros-image-world image do not build;
    • a "best effort" has been made to have webos-image-ros-{core,world} (DISTRO = "webos") build successfully for their supported MACHINES-s for dashing and eloquent with dunfell, but some of the packages that are used to create the webos-image-world image do not build;
    • a "best effort" has been made to have bitbake world builds successfully for the supported MACHINES-s for melodic, noetic, foxy, galactic, and rolling with gatesgarth (NOT DONE);
    • a "best effort" has been made to have bitbake world build successfully for the supported MACHINES-s for dashing and eloquent with dunfell, gatesgarth, hardknott, honister, and kirkstone (NOT DONE).

    NOTE: Builds are done using mcf and the files/*.mcf configuration files from the [build] branch.

  • This is the final milestone worked on by LG Electronics.

  • COMPLETED 2022-06-05

Unscheduled Features

  • Update to the latest releases of the current ROS distros. This should make rolling be buildable again.
  • Update to the latest commits on the branches for the current OpenEmbedded release series. This should allow PR 1013 to be reverted.
  • Reduce support to "best effort" for OpenEmbedded hardknott and honister.
  • Support the release of OpenEmbedded kirkstone and the under-development release series langdale.
  • Support ROS 2 humble.
  • Update to webOS OSE v2.16.0 .
  • Update Package Version Differences for newer packages from the upstream OpenEmbedded layers since 2021-08-27 and for the changes introduced by webOS OSE v2.16.0 .
  • Correct the value of OpenEmbeddedVersion in build/files*/*-kirkstone.mcf to be 4.0*.
  • Implement the items under Phase 4 in the Layer Version 3 section and update this document for the new files generated by ros-generate-recipes.
  • Drop from superflore its generation of ROS_DISTRO_METADATA_VERSION_REVISION in conf/ros-distro/include/<ROS_DISTRO>/generated/superflore-ros-distro.inc.
  • Support building from a locally modified source tree: add inherit ${@ 'externalsrc' if d.getVar('EXTERNALSRC') else ''} to ros_componment.bbclass.
  • Support regenerating recipes when package.xml is modified: implement ros-generate-recipes --from-local-source /path/to/package.xml.
  • mcf: Support multiple -f options and recognize a FetchOverrides dict: { '<layer-name>': { <fetch> } } for ease in doing CI from the weboslayers.py of webOS OSE; this will be file format 4.
  • Finalize the IMAGE_FEATURES values.
    • Convert per-device configuration from a USB flash drive code in webos-compat.inc to a bbappend that creates a package that's per-device configuration from a USB flash drive and conditionally included based on an IMAGE_FEATURES
  • Update README.md with contributions, attributions, governace, etc. sections (make similar to the one in [legacy]).
  • Move the build instructions from wiki to [build]README.md.
  • Determine the root cause as to why the dependencies from package.xml are not always correct.
  • Submit PR-s to add the dependencies found to be missing in the package.xml files of the upstream ROS components.
  • Support use of the Connext DDS implementation.
  • Drop the second argument to d.getVar() when it's True (now the default).
  • Document what was done in Milestone 15 on a How to Support a New OpenEmbedded Release wiki page.
  • Document what was done in Milestone 16 on a How to Support a New ROS Distro wiki page.

Issues

This section has been moved into Resolved Legacy Issues. New issues will be posted to meta-ros GitHub issues.

Change Log

Revision 52

  • Document these exceptions to the "done when" items of Milestone 17:
    • None of the items were done for rolling because it does not build.
    • Some of the packages use to create the gatesgarth images do not build.
    • Some of the packages use to create the ros-image-world images for dashing and eloquent do not build.
    • None of the bitbake world items were done.
    • The sanity tests were not run on the images for raspberrypi4 and raspberrypi4-64 nor on any of the webos-image-ros-* images.
  • Indicate that Milestone 17 has been completed.
  • Add more items to the Unscheduled Features section and update for the release of webOS OSE v2.16.0

Revision 51

  • Redefine Milestone 17 to be what was merged by the end of Decemeber 2021 (when LG Electronics stopped maintaining the meta-ros and meta-ros-webos). Move its uncompleted features to a new Unscheduled Features section. Update its "done when" item for the support reduction to "best effort" for gatesgarth and dashing, and for the dropping of support for zeus.

Revision 50

  • Indicate that Milestone 15 has been completed.
  • Add a Milestone 16 item to support webOS OSE v2.10.0.
  • Add items to support OpenEmbedded honister, and ROS 2 galactic to Milestone 16 and update the milestone "done when" items for the additions.
  • Bring the list of the other ROS distro releases supported by Milestone 16 up-to-date.
  • Support for EOL ROS 2 eloquent will no longer be dropped in Milestone 17. Instead, support for it will be continue to be "best effort".
  • Support for EOL ROS 2 dashing will be "best effort" as of Milestone 17.
  • Add a note that the <OE-LAYER> in ros/rosdistro/rosdep/*.yaml is only a hint.
  • Drop OpenSplice DDS from the list of DDS implementations to eventually support.
  • Correct the description of Milestone 14: it supported a ROS 1 distro (melodic), but only supported webOS OSE v2.6.0 for the ROS distro releases as of Milestone 13.
  • Indicate that Milestone 16 has been completed.

Revision 49

  • Add a Milestone 16 item to introduce support for ROS 1 noetic and to add support for the end of 2020 releases of the other ROS distros.
  • Repurpose Milestone 15 to introduce support for webOS OSE v2.8.0, build 263, and OpenEmbedded hardknott, and to drop/reduce support for OpenEmbedded thud, warrior, and zeus. Move most of the Milestone 15 technical debt and cleanup items to a new Milestone 17, but add a few new ones to Milestone 15. Update the milestone "done when" items for the support changes.
  • Drop support for ROS 2 eloquent in Milestone 17.
  • Use a new version (0.3.2-19-g4105ad8) of superflore when generating the recipes for last of the Milestone 15 ROS distro releases. Update this document for its changes of the ROS_UNRESOLVED_PLATFORM_PKG_ variable prefix to ROS_UNRESOLVED_DEP- and the prefix of their generated values from UNRESOLVED- to ROS_UNRESOLVED_DEP-.
  • Add to Milestone 15 updating base.yaml and python.yaml for the platform packages for which ROS_UNRESOLVED_DEP-* settings have been generated and creating a PR for ros/rosdistro.git with the changes.
  • Recommend waiting until the new platform package mappings have been used to generate recipes for all of the ROS distros and OpenEmbedded release series before creating the ros/rosdistro.git PR for them.
  • Change the milestone "done when" items to not require that bitbake world always build successfully and create separate items for the TurtleBot 3 image, which will initially not be required to build for noetic.
  • Split off a Phase 4 from Phase 3 of Layer Version 3 so that the items can be worked on in different milestones. Include in Phase 4 an item to generate newer-platform-components.list from http://packages.ros.org/ros2/ubuntu/dists/focal/main/source/Sources.gz for noetic, foxy and rolling.
  • Indicate that Milestone 14 has been completed.
  • Fix typos and formatting.

Revision 48

  • Introduce the practice of adding tags along the OpenEmbedded release series and build branches of the form: <BRANCH>/milestones/<N>. Update build/files/*.mcf to use them.
  • Introduce rebaseable -next suffixed branches.
  • Mention in each milestone that the pins to the HEAD commits of the upstream OpenEmbedded layers in build/files/*.mcf are kept current.
  • Extend Milestone 13 to include supporting the second June foxy release with all of the OpenEmbedded release series and webOS OSE (not just dunfell) and to support building all of the ROS distros (including foxy) with OpenEmbedded gatesgarth.
  • Extend Milestone 14 to include supporting rolling with OpenEmbedded dunfell and gatesgarth, supporting the releases of other ROS distros announced as of mid-August, and updating to webOS OSE v2.6.0, build 218, but reduce its scope to just supporting newer ROS distro releases (with dunfell and gatesgarth only) and webOS OSE releases. Its other items have been moved into Milestone 15.
  • Correct the milestone "done when" items: webOS OSE is only supported with OpenEmbedded thud.
  • Indicate that Milestones 12 and 13 have been completed. Add the completion dates of several earlier milestones.
  • Fix typos.

Revision 47

  • Indicate that Milestone 11 has been completed. Add the date of announcement to completed milestones for which one was posted.
  • Split up Milestone 12, changing its focus, and reorganize the subsequent milestones. In particular, support for foxy is now in Milestone 13 and the branches are separate and no longer considered to be rebaseable after Milestone 12.
  • Generate recipes using superflore v0.3.2. It no longer creates files/<ROS_DISTRO>/generated/superflore-change-summary.txt and changes the default value of ROS_BUILD_TYPE for a ROS 2 distro to be ament_cmake instead of catkin.
  • Add updating to webOS OSE v2.5.0, build 197 to Milestone 12.
  • Add generating recipes for the Q1 2020 ROS distro releases to Milestone 12.
  • As of Milestone 12, change the policy for selecting versions of platform packages provided by OpenEmbedded.
  • Change how packages that do not build are excluded from the the *-image-*-world images. This allows the *-image-*-world images for melodic to build successfully (albeit with many packages excluded).
  • As of Milestone 12, bitbake world must build successfully before a milestone is considered to have been completed.
  • Split off a number of items under Phase 2 in the Layer Version 3 section into a new Phase 3 to be implemented in Milestone 14 [[now Layer Version 4, Phase 1 to be implemented in Milestone 15]. Change the renaming of various bbclass-s in what's to be done for Phase 2 to copying and deprecating (with dropping to be done in Milestone 14 [now Milestone 15]), and drop the item to rename DISTRO_FEATURES and IMAGE_FEATURES values ros-* to ros-trial-*.
  • Introduce a ROS_OE_RELEASE_SERIES setting in meta-ros-common/conf/ros-distro/ros-distro.conf. Deprecate the MCF_OE_RELEASE_SERIES variable set in files/*.mcf.
  • Change how the last commit of a milestone is marked. Currently, its commit summary has a {{milestones/N}} prefix. Change it to be a commit that sets the value of ROS_DISTRO_METADATA_MILESTONE_* in a new file meta-ros-common/conf/ros-distro/milestone.conf.
  • Change the revision portion of ROS_DISTRO_METADATA_VERSION to be ROS_DISTRO_METADATA_MILESTONE_NUMBER instead of the unchanging value in conf/ros-distro/include/<ROS_DISTRO>/generated/superflore-ros-distro.inc.
  • Revert the documentation changes for conf/ros-distro-type/include/ros[12]/generated/newer-platform-components-preferred-versions.inc and files/ros[12]/generated/newer-platform-components.list; ros-generate-recipes.sh will not be changed to create them until Milestone 14.
  • Mention additional characteristics of the branches following OpenEmbedded layer conventions.
  • Generated files don't have copyright years; drop them from the examples shown in this document.
  • Split the milestone headings in this document into two so that their description can be updated without invalidating links to them.
  • Fix typos.

Revision 46

  • Drop the defining of "developer" images; it makes more sense for users to create their own with whatever packages they want. Instead of trying to guess which ROS packages users might want installed on their images and what the various flavors of images might be, change the focus to be having the *-image-*-world images build and install as many packages much as possible.
  • Add changing newer-platform-components.list to be ROS_DISTRO_TYPE-specific and renaming various bbclass-s to what's to be done for layer version 3 phase 2.
  • Add support for building melodic for qemux86 to Milestone 10.
  • Update the requirements for considering Milestones 10 and later to be "done" to include building additional images and building melodic for qemux86.
  • Indicate that Milestone 10 has been completed.
  • Publish the support policies for OpenEmbedded release series and MACHINE-s to Policies as part of Milestone 11.
  • Add support for OpenEmbedded release series zeus and not yet released dunfell to Milestone 11. Update the features of the later milestones for this change.
  • Move allowing ROS 1 packages to be built and added a ROS 2 image out of what's to be done for layer version 3 phase 2 into ROS_SUPERFLORE_GENERATION_SCHEME 3. Therefore, it is no longer a feature of Milestone 12.
  • Add more details to Milestones 11, 12, and 13 and shuffle various items between them.
  • Drop the milestones after Milestone 13. Add a new placeholder Milestone 14 [now Milestone 13] to add support for ROS 2 foxy.
  • Change how the OE version portion of DISTRO_VERSION for DISTRO = "ros1" and DISTRO = "ros2" is determined for OpenEmbedded release series thud to be the same as that for the later OpenEmbedded release series.
  • It doesn't make sense for the default value of ROS_BUILD_TYPE to be catkin for a ROS 2 distro => change superflore have it be ament_cmake.
  • Change the scheme used to map Python 2 platform packages in python.yaml so that they map to the packages for the version of Python specified by ROS_PYTHON_VERSION for the ROS_DISTRO.
  • Split off the version differences tables onto a separate wiki page: Package Version Differences. Close the two issues associated with them.
  • Close the item concerning MACHINE support under Issues after creating a meta-ros GitHub issue for it and soliciting community input.
  • Close other items under Issues and create meta-ros GitHub issues for the remaining two, which is where any new issues will be posted.
  • Split off the Issues section into a separate wiki page: Resolved Legacy Issues.
  • Drop marking commits that are specific to a particular OpenEmbedded release series with the branch name in their summary lines.
  • Replace "OE-Core" with "OpenEmbedded" everywhere.
  • Clarify from where the value for ROS_DISTRO_BASELINE_PLATFORM is obtained and that is always a release of Ubuntu.
  • Fix typos and line wrapping.

Revision 45

  • Add supporting the latest eloquent release to Milestone 10.
  • Add moving the recipes for the components found in files/<ROS_DISTRO>/newer-platform-components.list to Milestone 10.
  • Change how the OE version portion of DISTRO_VERSION for DISTRO = "ros1" and DISTRO = "ros2" is determined for OpenEmbedded release series warrior and later.
  • Add the missing layer version 3 mapping for recipes-core/packagegroups/packagegroup-ros-world-<ROS_DISTRO>.inc.
  • Indicate that Milestones 8 and 9 have been completed.
  • Change how completed milestones are indicated so that their links aren't changed.

Revision 44

  • Update this document for the many changes introduced by layer version 3.
  • Drop the New Files and Changes to Existing Files section because it describes the changes from version 1 to version 2.
  • Drop information about component type; with layer version 3, there's no need to distinguish between them.
  • Indicate the earliest version of superflore that must be used to generate the recipes for layer version 3.
  • Split the layer version 3 implementation into two phases and place in different milestones. Additions to its features include:
    • Move webos-compat.inc, webos-image-*, and files/examples/webos-device-confg into a separate integration layer with its own repo: ros/meta-ros-webos.git.
    • Eliminate the requirement to set DISTRO and ROS_DISTRO in conf/bblayers.conf.
    • Make changes to the OE distro name and version for DISTRO = "ros1" and DISTRO = "ros2".
  • Drop a couple of items that appear elsewhere from the ROS_SUPERFLORE_GENERATION_SCHEME 3 list.
  • Add two new issues.
  • Add an item to the version differences tables for a platform package used by melodic.
  • Add new three milestones, including one for cleanup, and restructure and reorder the existing ones.
  • Indicate that Milestone 7 has been completed.

Revision 43

  • Update the document for the splitting off of ros-generate-cache and the new files it creates.
  • Change the name of the branch created during recipe generation from superflore/<DATETIME> to superflore/<ROS_DISTRO>/<RELEASE-YYYYMMDD>.
  • Add rosdoc-lite to the skip list for melodic (and kinetic): generation of ROS package documentation will never be done on the target.
  • Resolve 3 issues and add a new one.
  • Add several items to Milestone 7.
  • Move the item that removes the morty-related commits from [thud-draft] from Milestone 8 to Milestone 7.
  • Improve the descriptions of various items in various milestones.
  • Swap the order of the eloquent- and warrior-related milestones.
  • Fix typos.

Revision 42

  • Add <ROS_SUPERFLORE_GENERATION_DATETIME> and <ROS_DISTRO_RELEASE_DATE> to the comment line added to files/<ROS_DISTRO>/cache.* so that it's possible to regenerate the recipes.
  • Add update the specification for layer version 3.
  • Add connman to the images when DISTRO = "ros1" or DISTRO = "ros2" so that networking will start automatically when booting. Drop issuing /etc/init.d/networking start in the ROS sanity tests.
  • Drop all support for bouncy as its EOL has passed.
  • Indicate that Milestone 6 has been completed.
  • Add a milestone for the initial support of ROS 2 eloquent.
  • Move various items into later milestones.
  • Note the decisions to a number of items under Issues.
  • Correct the versions supplied by bionic-updates and thud for linux-yocto.
  • Fix typos.

Revision 41

  • Drop Milestone 6; it's a duplicate of Milestone 5.
  • Add milestones for supporting melodic on the TurtleBot 3 Waffle Pi.
  • Layer version 1 installs packages built with catkin under /opt/ros/<ROS_DISTRO>. Continue this in layer version 2 for melodic even though they're installed under /usr for ROS 2 distros until a decision is made as to which prefix will be used for both ROS versions.
  • Change the baseline for the version differences tables to be bionic-updates because those versions are most likely to be the ones actually in use.
  • Convert the second version differences table to be one that lists the packages whose versions are different in all of OpenEmbedded thud, webOS OSE, and Ubuntu bionic-updates.
  • Update the version differences tables for more of the platform packages used by melodic.
  • Move all metadata out of ros?_component.bbclass. Everything is either actually dependent on ROS_BUILD_TYPE or on the ROS distro version.
  • Drop all of the morty-related commits from [thud-draft] in Milestone 6 Milestone 7.
  • Change ros-implicit-workspace to be a value for IMAGE_FEATURES instead of for DISTRO_FEATURES.
  • Always start networking before source-ing *setup.sh.
  • Group ROS_SUPERFLORE_GENERATION_SCHEME 2 items that relate to restructuring the layout of meta-ros into a definition for a third version of the layer. Move the others into ROS_SUPERFLORE_GENERATION_SCHEME 3. Add a milestone for the implementation of layer version 3.
  • Add a milestone for initial support of OpenEmbedded warrior and add warrior to what subsequent milestones support.
  • Create meta-ros GitHub issues for two of the items under Issues and state the decisions made for 3 others.
  • Fix typos.

Revision 40

  • Change how platform packages without a mapping in rosdep/base.yaml or rosdep/python.yaml are handled so that it's possible to specify all of the mappings that bitbake requires in order to parse the metadata for a particular recipe without having to rerun ros-generate-recipes multiple times. Do this by having superflore use ${ROS_UNRESOLVED_PLATFORM_PKG_<PLATFORM_PKG>} as the mapped value and adding assignments of the form ROS_UNRESOLVED_PLATFORM_PKG_<PLATFORM_PKG> = "UNRESOLVED-<PLATFORM_PKG>" to conf/ros-distro/include/<ROS_DISTRO>/generated-ros-distro.inc. These assignments are then overridden in conf/ros-distro/include/<ROS_DISTRO>/ros-distro.inc until rosdep/base.yaml and rosdep/python.yaml are updated and ros-generate-recipes rerun. superflore no longer uses the OpenEmbedded Layer Index to guess the mappings; drop an item from ROS_SUPERFLORE_GENERATION_SCHEME 2 related to this.
  • Have the value for ROS_DISTRO_RELEASE_DATE be an argument to ros-generate-recipes => it's a generated setting and therefore appears in conf/ros-distro/include/<ROS_DISTRO>/generated-ros-distro.inc instead of it being a manually set one in conf/ros-distro/include/<ROS_DISTRO>/ros-distro.inc.
  • Add grpc, nanomsg, and all Lisp-related packages to the skip list for ROS 1 distros.
  • + is a legal character in LICENSE => don't map it to -. Map
  • Give catkin_ros_prefix.bbclass a better name: ros_opt_prefix.bbclass.
  • Update the *.bbclass file contents shown to match the actual source.
  • Set the default PYTHON_PN directly in classes/ros_component.bbclass instead of inherit-ing python*.
  • Add an item to ROS_SUPERFLORE_GENERATION_SCHEME 2 to output the entire value of <license>.
  • Drop Qt 5.9 from various milestones => the migration of webOS OSE to use it has been delayed.
  • Drop support for bouncy as it is now past its EOL.
  • Update the version differences tables for some of the platform packages used by melodic.
  • Make various clarifications and corrections.

Revision 39

  • Add instructions on how to update the platform package mappings used by superflore.
  • Add refactoring into sublayers as items for ROS_SUPERFLORE_GENERATION_SCHEME 2.
  • Remove v1-recipes-{ros1,ros2} and unused-recipes-ros1 as part of Milestone 5.

Revision 38

  • Add instructions on how to install and use superflore to generate recipes.
  • Specify the skip list for kinetic.

Revision 37

  • Retain the original layer version 1 implementation in a branch named [legacy].
  • OpenEmbedded has release series. Convert to using this term instead of just release.
  • Clarify that this repo will follow the OE convention of not having release version tags.
  • Clarify that superflore will only be used to generate recipes for ROS distros that have not reached their EOLs.
  • Clarity that the<DATETIME> in the commit message summary and pull request title are same as ROS_SUPERFLORE_GENERATION_DATETIME.
  • Introduce ROS_DISTRO_CODENAME, which is the two word name of the ROS distro, with both words capitalized.
  • Without a network connection, there's no output from ros2 topic echo /chatter => startup networking before testing the ROS basic actions (and make other tweaks).
  • Indicate that Milestones 4 and 5 have been completed.
  • Move the creation of the How to ... wiki pages to Milestones 6 and 9.
  • Slightly change the criterion for considering webOS OSE "done".
  • Add an issue concerning adding the checksums to <ROS_DISTRO>-cache.yaml.
  • Add rosidl-default-generators to the list of development packages to be removed from images.
  • Fix typos.

Revision 36

  • Clarify what the inputs to superflore are.
  • Specify the details of the pull request that superflore creates and have it enforce generating recipes for a single ROS_DISTRO at a time.
  • Specify the convention for the message summary lines of ROS_DISTRO-specific and OpenEmbedded release series-specific commits.
  • Clarify that the ROS_DISTRO-dependent skip lists are not built-in, but are specified to superflore each time it is run. Drop the item from the list for ROS_SUPERFLORE_GENERATION_SCHEME 2 that assumed they were built-in.
  • The "current" superflore has changed from when this document was first written => replace "current" with that version.
  • Make ROS_VERSION the first field of ROS_DISTRO_METADATA_VERSION and shift the existing fields to the right. The final ROS_DISTRO_METADATA_VERSION_REVISION field is now formatted as a Debian-style revision, ie, -<N>, absent when N is 0.
  • Tweak how files/<ROS_DISTRO>/superflore-change-summary.txt is generated to make it more useful.
  • DISTRO must now be set in conf/bblayers.conf instead of in the usual conf/local.conf. Modify mcf to do this when only generating conf/bblayers.conf. This way there's only a single file to modify when changing DISTRO-s.
  • Now that DISTRO is set early enough, use it to form the path to an optional .inc file of fixups for foreign DISTROS-s.
  • Ignore changes to files/<ROS_DISTRO>/newer-platform-components.list when determining if the superflore run is to be considered a NOP.
  • Ensure that ROS_BUILD_TYPE always has a value -- it's only explicitly set in generated recipes.
  • Fix typos.

Revision 35

  • Add a new item to the list for ROS_SUPERFLORE_GENERATION_SCHEME 2 concerning adding ROS 1 packages to a ROS 2 distro build.
  • Tweak the descriptons of several of the issues.

Revision 34

  • Reformulate the milestones for the release of dashing and the change to add support for using webOS OSE as an OE DISTRO. Indicate that Milestones 2 and 3 have been completed. Drop the instructions for fetching the repos as OpenEmbedded Build Instructions is their authoritative source.
  • Split off the setting of ROS_SUPERFLORE_GENERATION_DATETIME into a separate generated file: conf/ros-distro/include/<ROS_DISTRO>/generated-superflore-datetime.inc and consider a superflore run where only it changes as a NOP.
  • Introduce a variable ROS_SUPERFLORE_PROGRAM_VERSION in conf/ros-distro/include/<ROS_DISTRO>/generated-ros-distro.inc that contains the version of superflore that was used to generate the recipes.
  • Ensure that ROS_COMPONENT_TYPE always has a value -- it's only explicitly set in generated recipes.
  • Append ROS_DISTRO to the image basenames which will allow the images for different ROS_DISTRO-s to be distinguished.
  • Have classes/ros_component.bbclass prepend directories uniquely populated by ROS packages to FILES_${PN} and FILES_R{PN}-dev.
  • Introduce ros*_image.bbclass to be inherited by image recipes.
  • Renamed catkin_opt_ros.bbclass to catkin_ros_prefix.bbclass.
  • Update the contents shown of the new files to match what's actually in the commits.
  • For ROS 1 distros, ROS1_COMPONENTS_BASELINE_DISTRO can not be ROS_DISTRO (because its value is needed before we know if ROS_DISTRO is ROS 1 or ROS 2).
  • Don't use overrides to have different values for ROS_DISTRO_COMPAT depending on the ROS_DISTRO_TYPE.
  • Move include/<ROS_DISTRO_BASELINE_PLATFORM>/ros-distro-platform-*.inc back to being under conf/ros-distro. Even though the platform packages are provided by the OE DISTRO, their versions expected by ROS are independent of it.
  • Add classes/ros_<ROS_BUILD_TYPE>.bbclass to the files listed in the New Files section.
  • Tweaked the "ROS basic actions".
  • Undo the split of Issues into two sections.
  • Update the version differences tables for addition of support for using webOS OSE as an OE DISTRO.
  • Fix typos.

Revision 33

  • This document is no longer a proposal => drop "Proposed" from its title.

Revision 32

  • Redefine baseline platform to be a value of <ROS_DISTRO>-cache.distribution_file.release_platforms, ie, it now contains the name of the Linux distro as well as its codename.
  • Because the DISTRO effectively determines which platform packages are provided, move the preferred provider, preferred version, and blacklist include files for the platform packages to be under conf/distro instead under conf/ros-distro.
  • Convert the versioning scheme to be for the meta-ros metadata layer instead of for the DISTRO. Append it to the DISTRO_VERSION when meta-ros is used by a foreign DISTRO, eg, webOS OSE.
  • Prepend to DISTRO_NAME when meta-ros is used by a foreign DISTRO.
  • Add showing the DISTRO_NAME and the ROS build environment variable settings in the build configuration displayed by bitbake.
  • Generate entries in files/<ROS_DISTRO>/rosdep-resolve.yaml when the package needed to be looked up in the OpenEmbedded Layer Index.
  • Ignore the branch name when generating files/<ROS_DISTRO>/superflore-change-summary.txt.
  • There is no longer a default ROS_DISTRO => abort the build unless one is specified in conf/bblayers.conf.
  • Revert dropping the second argument to d.getVar() when it's the default True so as to be compatible with morty.
  • Clarify that the tables of packages with version differences between OpenEmbedded thud and Ubuntu bionic only contain direct dependencies of ROS packages, including some from melodic.
  • OpenEmbedded thud provides a later version of python-future than Ubuntu bionic does; add an entry to the table. opencv v3.2.0 has been backported into meta-ros. Drop its entry (and the bogus one for libxaw) from the table.
  • Add an issue concerning dropping of runs of superflore when the only change is the modified date of <ROS_DISTRO>-cache.yaml.gz.
  • Add an exception to the inclusion of a copyright notice in new files.
  • Resolve the issue concerning having separate branches in meta-ros for each OpenEmbedded release series.
  • Resolve the issue concerning the removal of ament-* from core-* images.
  • Fix typos.

Revision 31

  • Drop the Connext and OpenSplice packages from the skip lists for ROS 2 distros because their recipes will eventually be needed. Instead, temporarily add metadata to prevent them from being built until support for them is added (in Milestone 6).
  • Rename newer-system-components.txt to newer-platform-components.list. (This should have been done in Revision 29.) Change the script that generates it to remove the extraneous spaces from the field containing the list of dependencies.
  • Resolve an issue: stop generating the "Distributed under the terms of the BSD license" lines.
  • Prepend the component subdirectory to the items in ROS_SUPERFLORE_GENERATED_RECIPE_BASENAMES; rename the variable to ROS_SUPERFLORE_GENERATED_RECIPE_BASENAMES_WITH_COMPONENT.

Revision 30

  • Specify a ros-implicit-workspace value for DISTRO_FEATURES that causes login shells to implicitly source ros_setup.
  • Have the fourth field of DISTRO_VERSION be reset to 0 with every new checkin of superflore output by having superflore generate conf/ros-distro/include/<ROS_DISTRO>/generated-ros-distro.inc with ROS_META_REVISION set to 0 and then "allowing" the variable to be manually incremented by editing the file when releasing meta-ros with manually created changes.
  • Switch back to using ROS_DISTRO_RELEASE_DATE for the second field of DISTRO_VERSION unless it is prior to the first release of a ROS_DISTRO, when ROS_DISTRO_RELEASE_DATE is "".
  • Switch to use the same version of the kernel (4.14) for both qemux86 and raspberrypi.
  • Add new items to the list for ROS_SUPERFLORE_GENERATION_SCHEME 2 concerning fetching source "correctly" from GitHub and using BBFILES_DYNAMIC.
  • Add a new issue concerning requiring commits to have a Signed-off-by: line.
  • Resolve the issue concerning OE PR variables.

Revision 29

  • Rename files/<ROS_DISTRO>/cache-diffable.yaml to files/<ROS_DISTRO>/cache.diffme and improve its generation to retain {WORD} instead of converting to <WORD>.
  • Add a warning to be careful when using the standard OE _<DISTRO> override.
  • Specify the skip lists for all of the ROS_DISTRO-s in the same place.
  • Introduce a new variable ROS_DISTRO_BASELINE_PLATFORM set in conf/ros-distro/include/<ROS_DISTRO>/ros-distro.inc. It specifies the codename of the release of Ubuntu (eg, bionic) that <ROS_DISTRO> targets. This determines the versions of the platform packages that OE must provide. Add ROS_DISTRO_BASELINE_PLATFORM to DISTROOVERRIDES.
  • Change "system package" references to "platform package" to match the term used by REP 2000, including renaming ROS_SUPERFLORE_GENERATED_SYSTEM_PACKAGE_DEPENDENCIES to ROS_SUPERFLORE_GENERATED_SYSTEM_PACKAGE_DEPENDENCIES.
  • Change "Ubuntu 18.04" references to "Ubuntu bionic".
  • Use the last-modified timestamp of the <ROS_DISTRO>-cache.yaml from which superflore generated the recipes as the value of ROS_SUPERFLORE_GENERATION_DATETIME. Use its date portion as the second version field of DISTRO_VERSION instead of ROS_DISTRO_RELEASE_DATE, which is dropped.
  • Create separate preferred provider, preferred version, and blacklist include files for platform packages, ie, those dependent on ROS_DISTRO_BASELINE_PLATFORM: conf/ros-distro/include/<ROS_DISTRO_BASELINE_PLATFORM>/ros-distro-platform-{preferred-providers,preferred-version,recipe-blacklist}.inc.
  • Prefix the setup scripts provided by ros-workspace with ros_ and install them under /usr/bin.
  • Update for the creation of the https://github.com/ros/meta-ros fork of https://github.com/bmwcarit/meta-ros. In particular, drop the creation of new branches and the creation of PR-s for them at the end of the milestones. Instead, issue them for [master] as development proceeds.
  • Rearrange the milestones to prioritize complete support for dashing over crystal, including porting Beanbird Bot to it. Also prioritize melodic support over bouncy.
  • Change the milestones to build for qemux86 instead of qemuarm64.
  • Use the [rocko] branch of meta-qt5 so as to get Qt 5.9, which is the version provided by Ubuntu bionic.
  • Indicate that Milestone 1 has been completed.
  • Add a new issue concerning having separate includes-{ros1,ros2} trees.
  • Fix typos.

Revision 28

  • Clarify that mapping <ROS-PKG> to [] to eliminate a platform package dependency that could be needed in another ROS_DISTRO should not be done.
  • Generate a new file files/<ROS_DISTRO>/cache-diffable.yaml, which is files/<ROS_DISTRO>/cache.yaml filtered to make it easier to discover the differences between versions of the file.
  • Clarify that files/<ROS_DISTRO>/rosdep-resolve.yaml is to contain entries where the mapping is to [].
  • For ROS 1 distros, change ROS1_COMPONENTS_BASELINE_DISTRO to be itself.
  • Add more attributes found in package.xml to those to be handled by ROS_SUPERFLORE_GENERATION_SCHEME 2.
  • Add more items to Issues.
  • Ubuntu bionic has a later version of python3 than thud; move its entry to the correct table.
  • Fix typos.

Revision 27

  • Move recipes-{ros1,ros2}/packagegroups/* to recipes-core/packagegroups/. Now there are no longer any recipes under recipes-{ros1,ros2} => rename them to includes-{ros1,ros2}. Make corresponding changes to variable and function names.
  • Since everything under includes-{ros1,ros2} is included by a generated recipe, drop adding the comment stating so to the files.
  • Split up ros_common.bbclass into common metadata for all ROS distro types (ros_distro.bbclass) and common metadata for all ROS component types (ros_component.bbclass). Rename ros_common__get_component_type() to ros_distro__get_component_type() and have each ROS distro type provide its own implementation.
  • Move the inherit ros_distro_${ROS_DISTRO} in the generated recipes to the top of the file so that ros_distro__get_component_type() has been defined before it's used in the assignment to ROS_COMPONENT_TYPE.
  • Bitbake complains about require-ing /dev/null => specify a new empty file conf/ros-distro/include/empty.inc instead.
  • Supply a default for ROS_CN in classes/ros_component.bbclass.
  • Indicate that the first line of files/<ROS_DISTRO>/newer-system-components.txt is a "comment" by beginning it with #.
  • Add an item to the list for ROS_SUPERFLORE_GENERATION_SCHEME 2. Correct the URL for Sources.gz for ROS 1.
  • Specify the resolution to the issue concerning copyright notices.
  • Add several more packages to the table of platform packages provided by thud that might need to be downgraded to the versions provided by Ubuntu bionic .
  • Add a new table of platform packages provided by thud that might need to be upgraded to the versions provided by Ubuntu bionic but are likely to be difficult to do so.
  • Add a new issues concerning OE PR.

Revision 26

  • Create separate bbclasses for component types ({ros1,ros2}_component.bbclass) distinct from those for ROS 1 and ROS 2 distros ({ros1,ros2}.bbclass renamed to {ros1,ros2}_distro.bbclass). Define variables for the distro and component types: ROS_DISTRO_TYPE and ROS_COMPONENT_TYPE. Add overrides for the distro and component types: _<ROS_DISTRO_TYPE>-distro and _<ROS_COMPONENT_TYPE>-component. Add assigning to ROS_COMPONENT_TYPE and inherit ${ROS_COMPONENT_TYPE}_component to the generated recipes.
  • Allow meta-ros to be used with other OE DISTRO-s, eg, webOS, by having conf/ros-distro/include/<ROS_DISTRO>/generated-ros-distro.inc set ROS_DISTRO_TYPE and replacing inherit ${DISTRO} with inherit ${ROS_DISTRO_TYPE}_distro. To have the layer build the OE DISTRO-s that it does define, add DISTRO = ${ROS_DISTRO_TYPE} to conf/local.conf. (As of Revision 36, it must be added to conf/bblayers.conf.)
  • Drop support from meta-ros for ROS distros without generated recipes. Move manually created recipes under recipes-ros1 for which there is not a generated equivalent to unused-recipes-ros1 (except for those under packagegroups); there are no such recipes under recipes-ros2.
  • Drop support for the optional use of Python 3 with ROS 1.
  • Require that all ROS recipes inherit from ros_distro_${ROS_DISTRO} => move the export of variables expected to be in the environment when building to there from the .conf files.
  • Don't retain all *-<PV>*.inc files; only keep the current ones for each ROS_DISTRO.
  • Add a note that the entries in the openembedded stanzas of rosdep/base.yaml (and rosdep/python.yaml) should never have a -native suffix.
  • Add setting the variable ROS_CN to the ROS component name and the variable ROS_AUTHOR to <author> to the generated recipes.
  • To prevent confusion when viewing, have the first line of the files under the include/<ROS_DISTRO> subdirectories of conf and files identify the ROS_DISTRO subdirectory in which the file resides.
  • Update the table for Milestone 2 of the ROS components whose recipes are not found under recipes-{ros1,ros2}.
  • Add a package from bouncy (yaml-cpp) to the table of platform packages whose versions provided by OE are later than those provided by Ubuntu bionic (in this case, the version is provided by a manually created recipe in meta-ros).
  • Rename ROS_ONE_COMPONENTS to ROS1_COMPONENTS. Drop conf/ros-distro/include/ros-one-components.inc; instead set ROS1_COMPONENTS to the value of ROS_SUPERFLORE_GENERATED_RECIPES_FOR_COMPONENTS in conf/ros-distro/include/<ROS1_COMPONENTS_BASELINE_DISTRO>/generated-ros-distro.inc.
  • Include the @<OE-LAYER> in the mappings in files/<ROS_DISTRO>/rosdep-resolve.yaml.
  • Drop the second argument to d.getVar() when it's the default True.
  • Use d.getVar('VAR') instead of ${VAR} in inline Python in variable assignments.
  • Place the file layout into its own section.
  • Add an issue concerning ros-core.
  • Fix typos.

Revision 25

  • Drop the generation of packagegroups/packagegroup-ros-world.bb by superflore. Instead, have it place the list of the "world" packages in a new variable ROS_SUPERFLORE_GENERATED_WORLD_PACKAGES and replace the existing file with new contents that use this new variable.
  • Move recipes-ros1/packagegroups/packagegroup-ros-world.bb to recipes-core/packagegroups/packagegroup-ros-world.bb. Done move the other packagegroup-ros*.bb to recipes-{ros1,ros2}. Rename the remaining recipes-ros1/packagegroups/packagegroup-ros-*.bb to recipes-ros1/packagegroups/packagegroup-ros1-*.bb.
  • Have superflore place the list of packages for which it couldn't generate recipes in a new variable ROS_SUPERFLORE_GENERATION_NOT_POSSIBLE.
  • When generating the value for LICENSES, convert sequences of spaces and punctuation other than -, _, . to a single -.
  • Add an issue concerning packagegroup-ros2-world.
  • Fix typos.

Revision 24

  • Drop the use of the term edition and switch to using the distribution type as the value for the OE DISTRO setting, ie, ros1 or ros2. This eliminates the ambiguity as to whether the term ROS refers to only what's now known as "ROS 1" or to something that's common to both "ROS 1" and "ROS 2". To be consistent, rename the existing recipes-ros directory to recipes-ros1.
  • Mention generated-recipes-<ROS_DISTRO>/packagegroups/packagegroup-ros-world.bb explicitly in the layout.
  • Replace ROS_SUPERFLORE_GENERATED_RECIPES_WITH_VERSIONS with ROS_SUPERFLORE_GENERATED_RECIPE_BASENAMES.
  • Add catkin-virtualenv and iirob_filters to the skip list for melodic.
  • Fix typos.

Revision 23

  • Implement a different scheme for settings ROS_RECIPES_TREE: instead of setting it in <COMPONENT>_common.inc, place a list of all of the ROS components in ROS_ONE_COMPONENTS and use the presence of <COMPONENT> in that list to select recipes-ros. Define ROS_ONE_COMPONENTS in a new file conf/ros-distro/include/ros-one-components.inc (it is not under conf/ros-distro/include/<ROS_DISTRO> because which tree the metadata for the packages of a component appears is independent of ROS_DISTRO).
  • Add new lists ROS_SUPERFLORE_GENERATED_RECIPES_WITH_VERSIONS to conf/ros-distro/include/<ROS_DISTRO>/generated-ros-distro.conf.
  • Move setting of ROS_BPN, ROS_SPN, and ROS_SP into classes/ros_common.bbclass.
  • Explicitly mention appending to ROS_SUPERFLORE_GENERATED_BUILDTOOLS in conf/ros-distro/include/<ROS_DISTRO>/ros-distro.inc.
  • Replace the unused ros_superflore_generated__prefix_all() in superflore_generated.bbclass with ros_superflore_generated__get_recipes_tree().
  • Add two items to the list for ROS_SUPERFLORE_GENERATION_SCHEME 2.
  • Specific the skip list for melodic.
  • List an alternative name for recipes-ros-infrastructure: recipes-extra.
  • Add an issue concerning the license notice that appears in generated files.
  • Fix typos.

Revision 22

  • Place the files under files in a <ROS_DISTRO> subdirectory. Rename <ROS_DISTRO>-cache.yaml to <ROS_DISTRO>/cache.yaml.
  • Have superflore generate a new file files/<ROS_DISTRO>/newer-system-components.txt from http://packages.ros.org/{ros,ros2}/ubuntu/dists/bionic/main/source/Sources.gz. Use it to determine the versions of the ROS infrastructure packages to use.
  • Use the component version (<ROS_DISTRO>-cache.repositories.<COMPONENT>.release.version) instead of the package version (<ROS_DISTRO>-cache.release_package_xmls.<ROS-PKG>.version) for <PV>.
  • Specify that <COMPONENT>, <BPN>, and <PV> are lower case and have hyphens instead of underscores.
  • The openembedded mappings in rosdep/python.yaml for packages that are not prefixed with python- or python3- should be prefixed with ${PYTHON_PN}-.
  • Have superflore append --distro-renamed to any packages named ros or ros2 (because OE does not allow a packages to have the same name as DISTRO).
  • Add ROS_BPN = "<BPN>" (with underscores) to the generated recipes.
  • Drop ROS_DISTRO_EDITION since it's the same as DISTRO. Set DISTRO in conf/ros-distro/include/<ROS_DISTRO>/generated-ros-distro.inc.
  • Clarify when ROS_META_REVISION is to be incremented.
  • Use ----- instead of ===== to separate the sections in files/<ROS_DISTRO>/superflore-change-summary.txt.
  • superflore now handles all forms of the condition attribute.
  • Remove keystroke from the skip list because it has been fixed not to require a version of python3-setuptools later than that which was shipped in Ubuntu bionic .
  • Add several more packages to the table of platform packages provided by thud that might need to be downgraded to the versions provided by Ubuntu bionic .
  • Add another issue to decide whether the recipes for the components found in files/<ROS_DISTRO>/newer-system-components.txt should be moved to a separate recipes-ros-infrastructure tree.

Revision 21

  • Have superflore map ROS package names that end with _dev to recipe <PN>-s that end with -rosdev.
  • Revert: if the same package appears in both <buildtool_depend> and other <*depends>, it is only added to ROS_BUILDTOOL_DEPENDS (with a -native suffix). There are packages (eg, fastrtps-cmake-module in rmw-fastrtps-*.bb) that must appear in ROS_BUILD_DEPENDS to satisfy indirect dependencies even though they also appear in ROS_BUILDTOOL_DEPENDS.
  • Remove processing of the <member_of_group> item in package.xml. Its purpose was misunderstood. A new implementation will be done in ROS_SUPERFLORE_GENERATION_SCHEME 2.
  • Add ROS_PYTHON_VERSION to conf/ros-distro/include/<ROS_DISTRO>/generated-ros-distro.inc. Drop use of ROS_USE_PYTHON3.
  • Have superflore output a files/rosdep-resolve.yaml instead of placing the files it used to resolve platform package names under files/rosdep.
  • Have superflore output a files/superflore-change-summary.txt to be used to determine if there will be manual changes needed before the pull request is submitted.
  • Consider the generated recipes for bouncy and melodic when determining what's moved into v1-recipes-{ros,ros2} in Milestone 2.
  • Add a value for DISTRO_FEATURES that includes BehaviorTree.CPP v3 in packagegroup-ros-world instead of BehaviorTree.CPP v2. Drop the value for Navigation2; its packages will be includes in packagegroup-ros-world unless BehaviorTree.CPP v3 is selected (because it uses BehaviorTree.CPP v2).
  • Add an issue to decide if the versions of the platform packages provided by thud need to be downgraded to the versions provided by Ubuntu bionic .
  • Fix typos.

Revision 20

  • Correct the definition of <PV> to what's actually being used.
  • Update the explanation of how most of the native packages are identified.
  • Update the commentary describing several of the variables in conf/ros-distro/include/<ROS_DISTRO>/generated-ros-distro.inc.
  • superflore must handle <*depends> attributes of the form condition="$ROS_VERSION == <N>" because the entry for ml_classifiers in crystal now uses it. Drop ml_classifiers from the skip list as the lack of condition processing by superflore is why it appeared to be a ROS package that had been added by mistake.
  • Since superflore now needs to know the ROS_VERSION of the ROS_DISTRO it's generating recipes for, have it generate the value for export ROS_VERSION in conf/ros-distro/include/<ROS_DISTRO>/generated-ros-distro.inc and remove it from classes/ros_distro_<ROS_DISTRO>.bbclass.
  • Add keystroke to the skip list because it requires a version of python3-setuptools later than that which was shipped in Ubuntu bionic .
  • Add enhancements to how supreflore will classify packages in ROS_SUPERFLORE_GENERATION_SCHEME 2.
  • Split Issues up those that must be resolved before creating the first meta-ros pull request and those that can be resolved later. Add some new ones.
  • Clean up miscellaneous typos.

Revision 19

  • Make meta-ros an OE distro layer by adding a conf/distro tree. Set the OE DISTRO value to the ROS edition, ie, ros or ros2. Define a versioning convention for the OE distro .
  • Change the layout and names of ROS_DISTRO-dependent files to be analogous to those for an OE distro, in particular, rename the conf/<ROS_DISTRO>/*.conf files to conf/ros-distro/include/<ROS_DISTRO>/*.inc. Move various settings into new files that are a more appropriate place for them to reside.
  • Add ml_classifiers to the skip list for crystal.
  • Generate values for new variables in conf/ros-distro/include/<ROS_DISTRO>/generated-ros-distro.inc:
    • Set ROS_NUM_CACHE_YAML_COMMITS to the number of commits that will be returned by git log files/<ROS_DISTRO>-cache.yaml when the generated files are committed.
    • Set ROS_DISTRO_RELEASE_PLATFORMS to the iterated values of distribution_file.release_platforms.<LINUX-DISTRO>.<NAME> from <ROS_DISTRO>-cache.yaml.
    • Set ROS_SUPERFLORE_GENERATED_SYSTEM_PACKAGE_DEPENDENCIES to the list of platform packages found in the <*depend> items (except for <test_depends>).
  • If the same package appears in both <buildtool_depend> and other <*depends>, it is only added to ROS_BUILDTOOL_DEPENDS (with a -native suffix).
  • Change dgrp_ prefix to depgrp_.
  • Separate function names from the bbclass in which they're defined with __ instead of _.
  • Specify how to handle packages that previosly had manually created recipes now having generated ones.
  • Use most recent ROS release (medlodic) instead of indigo as the reference for determining whether a package is ROS or ROS 2.
  • Have superflore include the other files it uses to translate platform package names to OE recipes and layers under files/rosdep.
  • Create a new Milestone 6 to add support for warrior.
  • Update the Issues list.
  • Clean up miscellaneous typos.

Revision 18

  • Handle there being more than one <member_of_group> items for a package.
  • Specify how to handle multiple <build_type> items: use the last one.
  • Prevent the possibility of collisions between distro names, dependency group names, and build type names by prefixing the first two with distro_ and dgrp_ (respectively) when forming .bbclass basenames.
  • Add new lists ROS_SUPERFLORE_GENERATED_RECIPES andROS_SUPERFLORE_GENERATED_TESTS to conf/<ROS_DISTRO>/generated-ros-distro.conf.
  • Mention a special case that superflore must be modified to handle.
  • Can't generate the list of layer dependencies automatically because some are conditional based on what DISTRO_FEATURES are enabled.

Revision 17

  • Remove libyaml_vendor from the skip list. It builds libyaml from a seemingly arbitrary (untagged) commit (10c907871f1ccd779c7fccf6b81a62762b5c4e7b), but there must be a good reason that this was done...
  • Fold oe-opkg.yaml into rosdep/base.yaml.
  • Process the <member_of_group> item in package.xml:
    • Have the generated recipe inherit ros_<member_of_group>.bbclass. If <member_of_group> is absent, the inherit statement will not specify a bbclass.
    • Add ROS_SUPERFLORE_GENERATED_DEPENDENCY_GROUPS, the list of all of the dependency groups encountered, to conf/<ROS_DISTRO>/generated-ros-distro.conf.
  • Remove the export LANG statement accidentally left in conf/layer.conf.
  • Move items that aren't expected to require changes to superflore out of the list for ROS_SUPERFLORE_GENERATION_SCHEME 2 and into the lists for the milestones.
  • Defer expanding XHTML in <description> until ROS_SUPERFLORE_GENERATION_SCHEME 2.
  • Add the environment variables expected by ROS 2 executables to the default environment of images.
  • Change Milestones 2 and 3 to build for qemuarm64 instead of qemuarm.
  • Specify what needs to run in order for Milestone 2 to be considered done.
  • Classify several groups of packages as optional and control their presence in packagegroup-ros-world via various values for DISTRO_FEATURES. Remove them from the skip list for crystal.
  • Split off getting the optional packages to build for crystal into a separate Milestone 4.
  • Create a new Milestone 5 to add support for dashing.
  • Further refine the milestone instructions.

Revision 16

  • Generate an assignment of the packages found in the <buildtool_depend> and <buildtool_export_depend> items to a new variable ROS_SUPERFLORE_GENERATED_BUILDTOOLS in conf/<ROS_DISTRO>/generated-ros-distro.conf. These packages do not appear in packagegroup-ros-world.
  • Add more packages to the crystal skip list.
  • Add more tasks to ROS_SUPERFLORE_GENERATION_SCHEME 2.
  • Change Milestones 2 and 3 to build for qemuarm instead of Raspberry Pi 3.
  • Add an Issues section.

Revision 15

  • Drop the packages found only in the <buildtool_depend>and <buildtool_export_depend> items from those built by packagegroup-ros-world.
  • Rename generated-superflore.conf to generated-ros-distro.conf as it better describes its contents.
  • Require a new conf/<ROS_DISTRO>/ros-distro.conf and move the setting of LAYERSERIES_COMPAT_ros-layer into it.
  • Tweak the milestone instructions.

Revision 14

  • Don't do anything special for components that build a single package, as one that only build one package in one distro can build multiple packages in a later distro.
  • Drop urdfdom and urdfdom_headers from and add back libyaml_vendor to the skip list for both distros. Drop console_bridge from the skip list for crystal.
  • Introduce recipes-ros/packagegroups/packagegroup-ros-world-${ROS_DISTRO}.inc. It allows overriding the generated RDEPENDS_packagegroup-ros-world.
  • Introduce per-distro blacklist.conf and preferred-versions.conf files. Place them in a conf/<ROS_DISTRO> subdirectory and have superflore generate conf/<ROS_DISTRO>/generated-superflore.conf instead of conf/generated-<ROS_DISTRO>.conf.
  • Add ROS_USE_PYTHON3 = "yes" to generated-superflore.conf when for a ROS 2 distro.
  • Export LANG in ros2.bbclass.
  • Further refine the milestone instructions.

Revision 13

  • Add ROS_SUPERFLORE_GENERATED_RECIPES_FOR_COMPONENTS to conf/generated-<ROS_DISTRO>.conf.

  • Tweak the contents of generated-recipes-<ROS_DISTRO>/packagegroups/packagegroup-ros-world.bb.

  • Requires that all manually generated .inc files include a common comment so that it's easy to find them all.

  • Modify recipes-core/images/core-image-ros-roscore.bb to build a suitable image for ROS 2.

  • Defer support for nativesdk until ROS_SUPERFLORE_GENERATION_SCHEME 2.

  • Rename the [superflore-*] branches to be prefixed by the OpenEmbedded release series they target, ie,

      [superflore-bouncy-beanbirdbot] -> [morty-superflore-bouncy-beanbirdbot]
  [superflore-crystal]            -> [thud-superflore-crystal]

  • Further refine the milestone instructions.

  • Fix typos.

Revision 12

  • Add a new ros_superflore_generated.bbclass that is inherited by every recipe that superflore generates. It sets ROS_SUPERFLORE_GENERATED to "1" which can be used by a bbclass to do different things when the recipe that is inheriting it was generated. Have layer.conf default it to "0".
  • Fix typos.

Revision 11

  • Revert the addition of the new variable ROS_DEPENDS because that libraries used by superflore implicitly fold the <depends> values into other items. Add a task for ROS_SUPERFLORE_GENERATION_SCHEME 2 to enhance the libraries to optionally not do this.
  • Add a oe-opkg.yaml to https://github.com/ros/rosdistro.
  • Enhance the --upstream-repo option to superflore-gen-oe-recipes to allow the specification of the branch from which to fetch.
  • Have superflore map ROS package names that end with _native to recipe <PN>-s that end with -rosnative.
  • Generate generated-recipes-<ROS_DISTRO>/packagegroups/packagegroup-ros-world.bb.
  • Add the input file used to generate the recipes to the layer as files/<ROS_DISTRO>-cache.yaml.
  • Add ROS_SUPERFLORE_GENERATION_DATETIME and ROS_SUPERFLORE_GENERATION_SKIP_LIST to conf/generated-<ROS_DISTRO>.conf.
  • Add a task for ROS_SUPERFLORE_GENERATION_SCHEME 2 to implement --show-skip-list and --override-skip-list="<PACKAGES-TO-SKIP>" options to superflore-gen-oe-recipes.
  • Rename [superflore-bouncy] branches to [superflore-bouncy-beanbirdbot] so that it's clear the not all of the generated recipes for bouncy will be tested.
  • Github only allows an organization to fork an upstream repo once, so revert back to creating a [superflore-crystal] branch in https://github.com/lgsvl/meta-ros2.

Revision 10

  • Add libyaml_vendor and urdfdom to the skip list for bouncy.
  • Misunderstood how ros.bbclass was used by the original code: it's not inherited by all of the recipes, only by those that inherit catkin.bbclass. Rename the original ros.bbclass to catkin_opt_ros.bbclass instead of ros_common.bbclass. (RTFC!)
  • Assign the <depend> values to a new variable ROS_DEPENDS that's added to DEPENDS and RDEPENDS_${PN} instead of adding the <depend> values to ROS_BUILD_DEPENDS, ROS_EXPORT_DEPENDS, and ROS_EXEC_DEPENDS. The prevents needless duplication of the dependencies.
  • Make it clear that the values of ROS_*DEPENDS contain hyphens not underscores.
  • Rename ROS_SF_GENERATION_SCHEME -> ROS_SUPERFLORE_GENERATION_SCHEME as not everyone will associate SF with superflore.

Revision 9

  • Rename ROSDISTRO to ROS_DISTRO, add ROS_VERSION, and export them to the build tasks so as to match REP 149: Environment Variables.
  • Finalize the skip list for bouncy.
  • Support <depend> -- see REP 149: <depend> (how could this have been missed? :-) ).
  • Reorder the statements in the generated recipe to be easier to read.
  • Always place patches in a <BPN> subdirectory.
  • Create a [superflore-bouncy] branch in https://github.com/lgsvl/meta-ros2-lgsvl.
  • Clarify how the build tree is to be set up for Milestone 2 and 3. Use the latest commits along each layer's [thud] branch instead of the one that was current when Yocto 2.6.1 was released (except for openembedded-core).

Revision 8

  • Add generation of <buildtool_export_depend> values found in <ROS_DISTRO>-cache.yaml (how could this have been missed? :-) ).
  • Generate a value for ROS_EXEC_DEPENDS and append it to RDEPENDS_${PN} instead of appending to the latter directly.
  • Clarify the comment about the "export" items found in <ROS_DISTRO>-cache.yaml.
  • Add rmw-connext-cpp and rmw-opensplice-cpp to the skip list for bouncy.
  • Add gazebo to the skip list for crystal.

Revision 7

  • Have the generated recipes always append to RDEPENDS instead of setting it. This is stricly only necessary for ${PN}-dev packages but it's simpler just to always do it.
  • Make it clear that <COMPONENT> and <BPN> contain hyphens not underscores.
  • Drop fastrtps and fastcdr from the skip list as they appear to be needed.

Revision 6

  • Don't use <BPN>_<PV>.bbappend files because bitbake complains when there isn't a matching <BPN>_<PV>.bb. Instead, place everything that will need to be checked for validity with each new <PV> in <BPN>-<PV>.inc.
  • Instead of having mcf recognize an optional 6th element to the layer defintion tuple in weboslayers.py, have it recognize an optional BblayersConfExtraLines variable.

Revision 5

  • Change the names of the .inc files common to a component to have a _common suffix instead of -common. This must be done because there are some components that build packages named <COMPONENT>-common => their <BPN>.inc and <BPN>-<PV>.inc filenames duplicate their <COMPONENT>-common.inc and <COMPONENT>-common-<PV>.inc filenames.
  • superflore needs to ignore a default list of components and packages.
  • Use a newly forked repo https://github.com/lgsvl/meta-ros for milestones 2 and 3.

Revision 4

  • Because it will be possible for this layer to support ROS releases as well as ROS 2 releases, assume that it will be eventually be named meta-ros.
  • Drop overrides from the basename of the .inc files because more than just variable overrides will be placed in them.
  • Add a -common suffix to the basenames of the <COMPONENT>*.inc files because many components that generate multiple packages have one with the same name as the component.
  • Add further clarification as to what's intended to be placed in each of the files under recipes-{ros,ros2}/<COMPONENT>.
  • Prefix all files added to classes with ros_ (exception: ros2.bbclass), since we don't control the namespace.
  • Generate conf/generated-<ROS_DISTRO>.conf instead of generated_<ROS_DISTRO>.bbclass. Have the user add the setting for ROS_DISTRO to conf/bblayers.conf (which is read before conf/layer.conf for the layers). Add an task to Milestone 1 to enhance mcf to support adding lines to the conf/bblayers.conf that it generates.
  • Require that a classes/ros_<ROS_DISTRO>.bbclass be manually added for each new ROS_DISTRO. It is where ROS_DISTRO_EDITION is set and <ROS_DISTRO_EDITION>.bbclass is inherited. ros_<ROS_DISTRO>.bbclass is inherited by the generated <BPN>_<PV>.bb.
  • Move common metadata for ROS and ROS 2 distros into classes/ros_common.bbclass, including the append to DISTROOVERRIDES.
  • Specify from what layer commits the builds for the milestones will be done.
  • Add a Milestone 3 that has the remainder of the packages in meta-ros2 cross-building successfully for Raspberry Pi.

Revision 3

  • Follow the convention use by OpenEmbedded and precede <PV> with - instead of _ in .inc filenames.
  • Use _ instead of - in .bbclass filenames so that any functions def-ed in them can be prefixed with the bbclass basename.
  • Add <ROS_DISTRO_EDITION> and <ROS_DISTRO> to DISTROOVERRIDES.`
  • Eventually support reading from a locally generated <ROS_DISTRO>-cache.yaml.

Revision 2

  • Updates from initial reviews and prototyping.
  • Superflore no longer needs to try to determine which components are build tools.
  • Added milestones.

Revision 1

  • Initial publication.
Clone this wiki locally