Update Devfile Samples to reference v2.2.0 Parent stacks

[kim-tsao]

Which area this feature is related to?

/area registry

Which functionality do you think we should add?

The current samples have a parent reference to the default stack version in the registry. e.g. Devfile sample go basic points to the default go stack version in the community registry. With this set up, parsing the samples will be successful because there are no conflicts. If the sample were changed to point to the latest version (v2.2.0 schema), parsing will fail because the parent and sample devfiles both contain the same command ids. We should create a new version of the devfile sample that is compatible with the parent 2.2.0 schema.

Why is this needed? Is your feature request related to a problem?

Detailed description:

Describe the solution you'd like

Describe alternatives you've considered

Additional context

[thepetk]

@kim-tsao I'm not 100% about the workaround for this issue. In my mind I have two options:

1. Create a dir inside the sample repo and have different versions of devfiles (like we do in the registry)
2. Create a new branch that will host the new devfile which will point to the latest version of go stack)

[kim-tsao]

@kim-tsao I'm not 100% about the workaround for this issue. In my mind I have two options:

1. Create a dir inside the sample repo and have different versions of devfiles (like we do in the registry)

2. Create a new branch that will host the new devfile which will point to the latest version of go stack)

@thepetk I had to verify if the extraDevfileEntries.yaml has multi-version support and it looks like it does: https://github.com/devfile/registry-support/blob/main/index/generator/tests/registry/extraDevfileEntries.yaml#L11-L23

We can set up the samples similar to the test registry nodejs sample , so option 1 is preferred but we need to see if the viewer can process samples like stacks where default versions can be specified like this example here: https://github.com/devfile/registry/blob/main/stacks/nodejs/stack.yaml

[thepetk]

Moving to next sprint (239) as we need to put it on hold for now.

[thepetk]

Resumed working on the issue.

[thepetk]

Working locally in order to check if the registry viewer supports sample versioning

[thepetk]

I've tried to check if the viewer supports multiple versions for samples:

• Created a new image locally for registry having the nodejs sample with two versions:

versions:
  - version: 1.0.0
    schemaVersion: 2.0.0
    description: nodejs with devfile v2.0.0
    git:
      remotes:
        origin: https://github.com/redhat-developer/devfile-sample
  - version: 1.0.1
    schemaVersion: 2.2.1
    default: true
    description: nodejs with devfile v2.2.0
    git:
      remotes:
        origin: https://github.com/nodeshift-starters/devfile-sample

• I've built the image and deploy it on local minikube cluster but I wasn't able to see the 2 different versions of the sample in the UI.

Should we consider adding a feature in the viewer or am I missing something?

[thepetk]

We have confirmed that the viewer does show up an updated image of the registry with multi-version samples.
Additionally it is confirmed that the index generator is able to handle multi-version samples. The investigation will continue on index server side (in order to be sure that the API is ok before started investigating on viewer side)

[thepetk]

After an investigation on the registry-support I have found that the multiple versions of a sample are supported. I think most probably we should investigate the viewer.

More detailed looking into v2index/sample I can see all updates from the image and both versions look ok.

{
  "name": "nodejs-basic",
  "displayName": "Basic Node.js",
  "description": "Node.js 16 application using Express 4.18.x",
  "type": "sample",
  "tags": [
    "Node.js",
    "Express"
  ],
  "icon": "https://nodejs.org/static/images/logos/nodejs-new-pantone-black.svg",
  "projectType": "Node.js",
  "language": "JavaScript",
  "provider": "Red Hat",
  "versions": [
    {
      "version": "1.0.0",
      "schemaVersion": "2.0.0",
      "git": {
        "remotes": {
          "origin": "https://github.com/redhat-developer/devfile-sample"
        }
      },
      "description": "nodejs with devfile v2.0.0"
    },
    {
      "version": "1.0.1",
      "schemaVersion": "2.2.1",
      "default": true,
      "git": {
        "remotes": {
          "origin": "https://github.com/nodeshift-starters/devfile-sample"
        }
      },
      "description": "nodejs with devfile v2.2.0"
    }
  ]
}

[thepetk]

@michael-valdron I was investigating on the viewer side and found this https://github.com/devfile/devfile-web/blob/main/libs/core/src/functions/fetch-devfiles/fetch-devfiles.tsx#L38 which I think means that by default on the viewer we are not accepting samples having multiple versions. WDYT?

[michael-valdron]

@michael-valdron I was investigating on the viewer side and found this https://github.com/devfile/devfile-web/blob/main/libs/core/src/functions/fetch-devfiles/fetch-devfiles.tsx#L38 which I think means that by default on the viewer we are not accepting samples having multiple versions. WDYT?

@thepetk Yeah that looks to be the culprit. I checked the /v2index again and I can see both versions for code-with-quarkus so it looks like a fix we need for the registry viewer.

[thepetk]

@michael-valdron do you think we should create a separate issue for the viewer or we should included it as part of this item?

[michael-valdron]

@michael-valdron do you think we should create a separate issue for the viewer or we should included it as part of this item?

@thepetk I think this should be its own item, as a user story under the area/registry-viewer tag with a reference to this thread.

[thepetk]

Issue created here: #1184 , not sure if we should assign it the blocker label.

Temporarily, I'm removing the target date for this issue.

[thepetk]

I'm resuming working on the issue. As discussed, we can introduce references to v.2.2.0 Parent stacks even if the viewer is not supporting them.

New target date: [EDIT] Removed

[thepetk]

I've tested different things while looking for the best workaround. The approach I'm most interested in is:

Multiple sample versions inside extraDevfileEntries

As suggested above a good way to start is to include multiple sample versions inside the extraDevfileEntries.yaml. For example, for go-basic we can have two different versions, one pointing to the default 1.0.2 and one pointing to 2.1.0 which supports schemaVersion >= 2.2.x.

- name: go-basic
  displayName: Basic Go
  description: Go 1.16 application
  icon: https://go.dev/blog/go-brand/Go-Logo/SVG/Go-Logo_Blue.svg
  tags:
    - Go
  projectType: Go
  language: Go
  provider: Red Hat
  versions:
    - version: 1.0.2
      schemaVersion: 2.1.0
      git:
        remotes:
          origin: https://github.com/devfile-samples/devfile-sample-go-basic.git
    - version: 2.1.0
      schemaVersion: 2.2.0
      default: true
      git:
        remotes:
          origin: https://github.com/thepetk/devfile-sample-go-basic.git

Separate Branch, Separate Sample Repo OR Directory with devfiles?

If we follow the above format we will need to support different of devfile samples versions for devfile.yaml. I can see 3 approaches:

[Not Supported ATM] Separate branch per devfile sample repo

This way we can introduce a new branch for every sample repo (e.g. 2.2.x default name) and upon extraDevfileEntries.yaml we can have:

versions:
  - version: 2.1.0
    schemaVersion: 2.2.0
    git:
      checkoutFrom:
        revision: 2.2.x
      remotes:
        origin: https://github.com/devfile-samples/devfile-sample-go-basic.git

If I'm not missing something, right now we don't support this in build-tools. If so, we will have to create a new issue to support this and then we can unblock this approach.

[Not Supported ATM] Directory inside each samples that will introduce different devfile versions

The idea is to have exactly the same structure as the stacks. So for every devfile sample we can have:

sample_root_dir/
...
---   devfiles/
       --- version1/
       --- version2/
       ...
       --- versionX/

Use different repo per devfile.yaml version

Following this solution we will have to use a different repo per devfile version. For example for the go-basic:

  versions:
    - version: 1.0.2
      schemaVersion: 2.1.0
      git:
        remotes:
          origin: <repo1>
    - version: 2.1.0
      schemaVersion: 2.2.0
      default: true
      git:
        remotes:
          origin: <repo2>

[thepetk]

I can confirm that all branches are protected, as only members of the devfile-samples org can merge PRs in any branch.

Other than that, in order to be able to use consistent port numbers I've created a PR (devfile/registry-support#188) to update the library version, which will let us accept two components declaring the same port.

[thepetk]

As seen during testing, the ODC uses index/sample so it always fetches the default branch of each sample.

That said, in order to have always the latest version of every sample on the default branch we should first complete #1136. This update will allow us to have consistent ports (8080) on every latest sample version.

As a result, I'm putting it back to todo and putting the blocked label.

[thepetk]

The library version of the registry support has been updated (here) and now we can support building a registry with a sample having multiple components using the same port

[thepetk]

Updating this item due to an issue found for parent overrides inside samples.

New scope

This issue will focus on updating the extraDevfileEntries.yaml inside the registry by introducing a new sample version to every sample that has a parent with multiple stack versions. For example, the go-basic parent, go stack, has more than one versions so we will need to add a new go-basic version that will point to the latest parent version (atm 2.1.0).

Default Version = Default Branch

For each sample we will keep the convention of maintaining the default sample version to the default sample branch. For example the default version of the go-basic sample (v1.0.2) will be inside the main branch (default branch of the repo).

Latest Version = v2.x.x Branch

For every sample that needs to have a new version we will create a new branch starting with v2.x.x. format. This will be the latest version of the sample.

Updated extraDevfileEntries.yaml

After the completion of the above steps the new extraDevfileEntries.yaml will look like:

schemaVersion: 1.0.0
samples:
  - name: nodejs-basic
    displayName: Basic Node.js
    description: Node.js 16 application using Express 4.18.x
    icon: https://nodejs.org/static/images/logos/nodejs-new-pantone-black.svg
    tags:
      - Node.js
      - Express
    projectType: Node.js
    language: JavaScript
    provider: Red Hat
    versions:
      - version: 1.0.0
        default: true
        schemaVersion: 2.0.0
        git:
          revision: main           
          remotes:
            origin: https://github.com/redhat-developer/devfile-sample.git
  - name: go-basic
    displayName: Basic Go
    description: Go 1.16 application
    icon: https://go.dev/blog/go-brand/Go-Logo/SVG/Go-Logo_Blue.svg
    tags:
      - Go
    projectType: Go
    language: Go
    provider: Red Hat
    versions:
      - version: 1.1.0
        default: true
        schemaVersion: 2.2.0
        git:
          revision: main
          remotes:
            origin: https://github.com/devfile-samples/devfile-sample-go-basic.git
      - version: 2.0.0
        schemaVersion: 2.2.0
        git:
          revision: v2.0.0
          remotes:
            origin: https://github.com/devfile-samples/devfile-sample-go-basic.git

[thepetk]

Removing the blocked label as the updated scope is not affected anymore by the blocker.

Also I've created a new version for each sample having a parent stack with multiple versions:

• Create springboot sample 2.0.0 version devfile-samples/devfile-sample-java-springboot-basic#37
• Create version 2.0.0 for python sample devfile-samples/devfile-sample-python-basic#35
• Create version 2.0.0 for go-basic sample devfile-samples/devfile-sample-go-basic#29

After those PRs are reviewed we can go ahead and create a PR for the updated extraDevfileEntries.yaml

[thepetk]

Removing the blocked label as the updated scope is not affected anymore by the blocker.

Also I've created a new version for each sample having a parent stack with multiple versions:

• Create springboot sample 2.0.0 version devfile-samples/devfile-sample-java-springboot-basic#37
• Create version 2.0.0 for python sample devfile-samples/devfile-sample-python-basic#35
• Create version 2.0.0 for go-basic sample devfile-samples/devfile-sample-go-basic#29

After those PRs are reviewed we can go ahead and create a PR for the updated extraDevfileEntries.yaml

I'm moving this item to review status as the 3 sample related PRs are under review :)

[thepetk]

I've splited the current issue into two parts:

• Create new sample versions for sample with parent outerloop support #1360 (which is the new versions implementation).
• The current issue which will focus on the update of the extraDevfileEntries.yaml. I'm putting it back to in progress and in the next sprint as it will have to wait until the merge of the PRs related to Create new sample versions for sample with parent outerloop support #1360

I've updated the story points of this issue to 5 and the new issue has 3 points (total 8, exactly like the previous story points)

[thepetk]

#1360 is now completed we can focus on the update of the extraDevfileEntries.yaml