feat(bulk-import): add the Bulk Import from GitHub Backend [RHIDP-955]

[rm3l]

What type of PR is this

/kind feature
/area bulk-import

What does this PR do / why we need it

This adds the backend plugin for the "Bulk Import from GitHub" feature. It will be used as backend for the Bulk Import UI (see some parts already in https://github.com/janus-idp/backstage-plugins/tree/main/plugins/bulk-import, but using mock data).

This is a continuation of the work started with #1216

The API exposed by this Backend is driven by the OpenAPI schema in plugins/bulk-import-backend/src/schema/openapi.yaml. Both type definitions and API documentation are generated from this OpenAPI spec file.

In a nutshell, this backend exposes API endpoints allowing to:

1. list all available GitHub repositories from all GitHub integrations listed in the app-config files
2. for selected repositories, create import PRs in the corresponding repos. Those PRs will add a catalog-info.yaml. Once the PR is merged, the entity will be ingested and made available in the Backstage catalog
3. return statuses for the import PRs
4. delete selected import PRs and start over if needed

Which issue(s) this PR fixes

• Fixes https://issues.redhat.com/browse/RHIDP-1207
• Fixes https://issues.redhat.com/browse/RHIDP-1128
• https://issues.redhat.com/browse/RHIDP-955

PR acceptance criteria

• Unit test

• Integration test

• Documentation

How to test changes / Special notes to the reviewer

As a prerequisite, you need to add at least one GitHub Integration (using either a GitHub token or a GitHub App or both) in your app-config file.
See https://backstage.io/docs/integrations/github/locations/#configuration and https://backstage.io/docs/integrations/github/github-apps/#including-in-integrations-config
You can add that to a local app-config.local.yaml.

Starting the backend

• For local dev, see the instructions in the CONTRIBUTING.md file. Start the Backend with:

yarn workspace @janus-idp/backstage-plugin-bulk-import-backend run start

• For integration in a Backstage App, see the plugin README.md file for more details.

REST API

The API is exposed at http://localhost:7007/api/bulk-import-backend. See the API Docs for more details, but below are a few examples.

Examples (with httpie)

`GET /ping`: anonymous endpoint to just get a response from the Backend

$ http GET http://localhost:7007/api/bulk-import-backend/ping
HTTP/1.1 200 OK

{
    "status": "ok"
}

`GET /repositories`: list repositories accessible from all GitHub Integrations

This is a paginated API endpoint, for performance purposes. The following query parameters allow to control the amount of data returned:

• pagePerIntegration: default is 1
• sizePerIntegration: default is 20

$ http GET http://localhost:7007/api/bulk-import-backend/repositories
HTTP/1.1 200 OK

{
    "errors": [],
     "repositories": [
        {
            "defaultBranch": "main",
            "errors": [],
            "id": "lemra-org-ent-2/animated-happiness",
            "importStatus": "WAIT_PR_APPROVAL",
            "name": "animated-happiness",
            "organization": "lemra-org-ent-2",
            "url": "https://github.com/lemra-org-ent-2/animated-happiness"
        },
        {
            "defaultBranch": "master",
            "errors": [],
            "id": "lemra-org-ent-2/flutter",
            "importStatus": null,
            "name": "flutter",
            "organization": "lemra-org-ent-2",
            "url": "https://github.com/lemra-org-ent-2/flutter"
        }
    ],
    "size": 20
}

`GET /imports`: list import jobs along with their status

This is a paginated API endpoint, for performance purposes. The following query parameters allow to control the amount of data returned:

• pagePerIntegration: default is 1
• sizePerIntegration: default is 20

$ http GET http://localhost:7007/api/bulk-import-backend/imports
HTTP/1.1 200 OK

[
    {
        "approvalTool": "GIT",
        "id": "lemra-org-ent-1/java-quarkus-starter",
        "repository": {
            "defaultBranch": "main",
            "errors": [],
            "id": "lemra-org-ent-1/java-quarkus-starter",
            "name": "java-quarkus-starter",
            "organization": "lemra-org-ent-1",
            "url": "https://github.com/lemra-org-ent-1/java-quarkus-starter"
        },
        "status": "ADDED"
    },
    {
        "approvalTool": "GIT",
        "github": {
            "pullRequest": {
                "number": 14,
                "url": "https://github.com/lemra-org-ent-2/animated-happiness/pull/14"
            }
        },
        "id": "lemra-org-ent-2/animated-happiness",
        "repository": {
            "defaultBranch": "main",
            "errors": [],
            "id": "lemra-org-ent-2/animated-happiness",
            "name": "animated-happiness",
            "organization": "lemra-org-ent-2",
            "url": "https://github.com/lemra-org-ent-2/animated-happiness"
        },
        "status": "WAIT_PR_APPROVAL"
    }
]

`POST /imports`: submit import requests - will create PRs and register Location entity in the Catalog

$ echo -n '[{"repository": {"url": "https://github.com/lemra-org-ent-2/animated-happiness", "defaultBranch": "main"}, "catalogInfoContent": "---\\napiVersion: backstage.io/v1alpha1\\nkind: Component\\nmetadata:\\n  name: animated-happiness\\n  annotations:\\n    github.com/project-slug: lemra-org-ent-2/animated-happiness\\nspec:\\n  type: other\\n  lifecycle: unknown\\n  owner: lemta-org-ent-2\\n---\\n", "github":{"pullRequest":{"title": "Custom PR title: catalog-info.yaml"}}}, {"repository": {"url": "https://github.com/lemra-org-ent-1/java-quarkus-starter", "defaultBranch": "main"}, "catalogInfoContent": "---\\napiVersion: backstage.io/v1alpha1\\nkind: Component\\nmetadata:\\n  name: java-quarkus-starter\\n  annotations:\\n    github.com/project-slug: lemra-org-ent-1/java-quarkus-starter\\nspec:\\n  type: other\\n  lifecycle: unknown\\n  owner: lemra-org-ent-1\\n---\\n"}]' | http POST http://localhost:7007/api/bulk-import-backend/imports
HTTP/1.1 202 Accepted

[
    {
        "github": {
            "pullRequest": {
                "number": 14,
                "url": "https://github.com/lemra-org-ent-2/animated-happiness/pull/14"
            }
        },
        "repository": {
            "name": "animated-happiness",
            "organization": "lemra-org-ent-2",
            "url": "https://github.com/lemra-org-ent-2/animated-happiness"
        },
        "status": "WAIT_PR_APPROVAL"
    },
    {
        "repository": {
            "name": "java-quarkus-starter",
            "organization": "lemra-org-ent-1",
            "url": "https://github.com/lemra-org-ent-1/java-quarkus-starter"
        },
        "status": "ADDED"
    }
]

[sonarcloud]

Quality Gate failed

Failed conditions
C Reliability Rating on New Code (required ≥ A)

See analysis details on SonarCloud

Catch issues before they fail your Quality Gate with our IDE extension SonarLint

[sonarcloud]

Quality Gate failed

Failed conditions
C Reliability Rating on New Code (required ≥ A)

See analysis details on SonarCloud

Catch issues before they fail your Quality Gate with our IDE extension SonarLint