feat(orchestrator): add orchestrator plugin (janus-idp#783)

* feat(orchestrator): add orchestrator plugins Squashed and rebased. Credits to - Guilherme Caponetto - Tiago Dolphine - Michael Anstis who did the work before squash and rebase. * feat(orchestrator): enable dynamic plugin * fix(orchestrator): load HostDirectory from backend-app-api (janus-idp#28) Signed-off-by: Moti Asayag <[email protected]> (cherry picked from commit 3ba02c6) * fix(orchestrator): make the port config optional (janus-idp#38) * feat(orchestrator): handle assessment workflow and make optional the workflowsSource config (janus-idp#35) * feat: issue FLPATH-591 - Add assessment workflow type and display outputs (janus-idp#21) * Configure for assessment swf * filter assessment on workflow definition page * workflow columns * workflow columns * workflow columns * workflow filter * feat: Render assessment results supporting dynamic categories * issue FLPATH-657: workflow label * Revert "Configure for assessment swf" This reverts commit b4048e1. * Revert sonata service port * Add key for workflow options category * Fix workflow execution from choose btn * Fix review comments * address comments * Fix review comment to switch to useRouteRef * fix review comments * fix review comments --------- Co-authored-by: richardwang98 <[email protected]> * feat: make optional the workflowsSource config (janus-idp#25) --------- Co-authored-by: anludke <[email protected]> * feat(orchestrator): add backend endpoint for getting workflows from data index service (janus-idp#39) * feat: add backend for getting work flows from data index service * feat: cherrypick - add backend for getting work flows from data index service * feat: cherrypick - add backend for getting work flows from data index service * feat: cherrypick - add backend for getting work flows from data index service * chore(orchestrator): update dependencies * feat(orchestrator): orchestrator plugin entry page & workflows table (janus-idp#31) https://issues.redhat.com/browse/FLPATH-686 https://issues.redhat.com/browse/FLPATH-682 * feat: introduced the workflows/overview endpoint (janus-idp#36) * FLPATH-702 : New endpoint to fetch workflow overview * FLPATH-702 : iterate through all elements instead of picking the first element * modified the map method * calculate avg execution time change * Add log message when http calls failing * Fix lint issues * rename variable * paginated graphql object fetch * Include workflowId in the result object * Renamed to description * to epoc timestamp * suppress eslint warning * epoc number to string * make all string fields optional * use a simple for loop * extracted to a new method * rename to lastTriggeredMs * include uri in the WorkflowOverview type * mark avgDurationMs as optional * feat(orchestrator): introduced `workflows/:workflowId/overview` endpoint (janus-idp#40) * FLPATH-702 : New endpoint to fetch workflow overview * FLPATH-702 : iterate through all elements instead of picking the first element * modified the map method * calculate avg execution time change * Add log message when http calls failing * Fix lint issues * rename variable * paginated graphql object fetch * Include workflowId in the result object * Renamed to description * to epoc timestamp * suppress eslint warning * epoc number to string * make all string fields optional * use a simple for loop * extracted to a new method * rename to lastTriggeredMs * include uri in the WorkflowOverview type * mark avgDurationMs as optional * fetch only one overview obj * fix(orchestrator): make the port config optional (janus-idp#37) * feat(orchestrator): orchestrator plugin entry page & workflows table (janus-idp#31) https://issues.redhat.com/browse/FLPATH-686 https://issues.redhat.com/browse/FLPATH-682 * removed unnecessary method --------- Co-authored-by: Guilherme Caponetto <[email protected]> Co-authored-by: Jonathan Kilzi <[email protected]> * feat(orchestrator): add Workflow Run List (janus-idp#30) * feat(orchestrator): add Workflow Run List FLPATH-693 A component listing running workflows is added. * chore: move loading logic out of the OrchestratorPage * chore(orchestrator): implement the WorkflowViewerFormatter (janus-idp#41) * chore: implement the WorkflowViewerFormatter: utility for converting WorkflowOverview backend data to data UI can display * removed redundant export and removed redundant 'Interface' suffix from DataFormatter interface name * Update plugins/orchestrator/src/dataFormatters/DataFormatter.ts Co-authored-by: Guilherme Caponetto <[email protected]> --------- Co-authored-by: Guilherme Caponetto <[email protected]> * feat(orchestrator): fetch data input schema from `/management/processes` (janus-idp#45) * Fetch data input schema from /management/processes * Fix some code smells * Rename WorkflowProcess -> WorkflowInfo * feat(orchestrator): implement the workflow viewer new UX (janus-idp#32) * feat(orechestrator): implement the workflow viewer new UX * code review fixes * visual fixes including skeleton for loading state * chore(orchestrator): Renames the components displaying the tabs content * chore(orchestrator): updates the OrchestratorClient Lazy loads the baseUrl Adds a method for calling GET /workflows/overview * feat(orchestrator): orchestrator workflow execution page (janus-idp#46) Co-authored-by: Jonathan Kilzi <[email protected]> * chore(orchestrator): updates the stories grouping (janus-idp#50) * chore(orchestrator): aligns the workflows table with the design (janus-idp#51) * chore(orchestrator): add Category to the procesInstance result (janus-idp#52) * fix(orchestrator): add links to workflows in the workflow list (janus-idp#53) * feat: added color icon to workflow details page last run status field (janus-idp#55) * feat(orchestrator): enable usage of local envelope for the workflow editor (janus-idp#56) * feat(orchestrator): add business key to assessment and pass on to workflow options (janus-idp#42) * feat(orchestrator): add page listing details of a workflow run (janus-idp#49) * feat(orchestrator): add feature flag for developer mode and config to enable/disable the integration with catalog (janus-idp#58) * feat(orchestrator): workflow editor modal (janus-idp#59) * feat(orchestrator): fix missing workflow type in overview (janus-idp#60) Signed-off-by: Gloria Ciavarrini <[email protected]> * fix(orchestrator): fix some code smells (janus-idp#63) * fix(orchestrator): addresses sonarcloud issues (janus-idp#65) * fix(orchestrator): addressed a couple of issues (janus-idp#64) * fix(orchestrator): minor fixes (janus-idp#67) * feat(orchestrator): refactoring to use data-index to fetch workflow definitions (janus-idp#57) Co-authored-by: Guilherme Caponetto <[email protected]> * fix(orchestrator): fix sonar issue (janus-idp#69) * feat(orchestrator): use assessment process id as bk for next workflows (janus-idp#70) * feat(orchestrator): execute workflow page new UX (janus-idp#68) * fix(orchestrator): theme for `monaco-editor` in `JsonTextAreaForm` component and mandatory `dataIndexService.url` (janus-idp#71) * fix(orchestrator): remove dependency on devmode in the entity provider (janus-idp#72) * fix(orchestrator): skip it if hardcoded specs do not exist (janus-idp#74) * feat(orchestrator): execute workflow page polishing (janus-idp#75) * feat(orchestrator): workflow instance result page (janus-idp#73) * chore(orchestrator): add codeowners to orchestrator * fix(FLPATH-852): show WF description on the execution result page (janus-idp#77) * chore(orchestrator): add OWNERS file to each orchestrator package * feat(orchestrator): migrates to the new UI (janus-idp#78) --------- Signed-off-by: Gloria Ciavarrini <[email protected]> Co-authored-by: Moti Asayag <[email protected]> Co-authored-by: richard wang <[email protected]> Co-authored-by: anludke <[email protected]> Co-authored-by: rhkp <[email protected]> Co-authored-by: Jonathan Kilzi <[email protected]> Co-authored-by: Jude Niroshan <[email protected]> Co-authored-by: Marek Libra <[email protected]> Co-authored-by: Bat-Zion Rotman <[email protected]> Co-authored-by: yu zhao <[email protected]> Co-authored-by: Gloria Ciavarrini <[email protected]> Co-authored-by: Tiago Dolphine <[email protected]>
aljesusg · Jan 17, 2024 · cf5fe74 · cf5fe74
1 parent 0b67a0d
commit cf5fe74
Show file tree

Hide file tree

Showing 141 changed files with 20,789 additions and 64 deletions.
diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS
@@ -29,4 +29,6 @@ yarn.lock                    @janus-idp/maintainers-plugins
 /plugins/rbac-common/        @janus-idp/maintainers-plugins @gorkem @AndrienkoAleksandr @PatAKnight
 /plugins/notifications       @janus-idp/maintainers-plugins @mareklibra
 /plugins/notifications-backend  @janus-idp/maintainers-plugins @mareklibra @ydayagi
-
+/plugins/orchestrator          @janus-idp/maintainers-plugins @caponetto @jkilzi
+/plugins/orchestrator-backend  @janus-idp/maintainers-plugins @caponetto @jkilzi
+/plugins/orchestrator-common   @janus-idp/maintainers-plugins @caponetto @jkilzi
diff --git a/.gitignore b/.gitignore
@@ -52,10 +52,16 @@ site
 # vscode database functionality support files
 *.session.sql
 
+.vscode
+
 # turbo
 .turbo
 
+# idea
+.idea
+*.iml
+
 # build cache
 .webpack-cache
 
-.idea/
+.tmp
diff --git a/package.json b/package.json
@@ -31,6 +31,10 @@
     "packages": [
       "packages/*",
       "plugins/*"
+    ],
+    "nohoist": [
+      "@janus-idp/backstage-plugin-orchestrator",
+      "@janus-idp/backstage-plugin-orchestrator/@kie-tools/**/!(react|react-dom)"
     ]
   },
   "devDependencies": {
@@ -54,7 +58,8 @@
   },
   "resolutions": {
     "@types/react": "^17.0.68",
-    "@types/react-dom": "^17.0.21"
+    "@types/react-dom": "^17.0.21",
+    "vscode-languageserver-types": "3.17.1"
   },
   "lint-staged": {
     "*": "turbo run prettier:fix --",

diff --git a/packages/backend/package.json b/packages/backend/package.json
@@ -27,6 +27,8 @@
     "@backstage/plugin-auth-backend": "^0.19.3",
     "@backstage/plugin-auth-node": "^0.4.0",
     "@backstage/plugin-catalog-backend": "^1.14.0",
+    "@backstage/plugin-events-backend": "^0.2.8",
+    "@backstage/plugin-events-node": "^0.2.8",
     "@backstage/plugin-permission-backend": "^0.5.29",
     "@backstage/plugin-permission-common": "^0.7.9",
     "@backstage/plugin-permission-node": "^0.7.17",

diff --git a/packages/backend/src/index.ts b/packages/backend/src/index.ts
@@ -11,6 +11,7 @@ import {
   createServiceBuilder,
   DatabaseManager,
   getRootLogger,
+  HostDiscovery,
   loadBackendConfig,
   notFoundHandler,
   ServerTokenManager,
@@ -19,8 +20,10 @@ import {
   useHotMemoize,
 } from '@backstage/backend-common';
 import { TaskScheduler } from '@backstage/backend-tasks';
+import { CatalogClient } from '@backstage/catalog-client';
 import { Config } from '@backstage/config';
 import { DefaultIdentityClient } from '@backstage/plugin-auth-node';
+import { DefaultEventBroker } from '@backstage/plugin-events-backend';
 import { ServerPermissionClient } from '@backstage/plugin-permission-node';
 
 import Router from 'express-promise-router';
@@ -43,6 +46,9 @@ function makeCreateEnv(config: Config) {
   const databaseManager = DatabaseManager.fromConfig(config, { logger: root });
   const tokenManager = ServerTokenManager.fromConfig(config, { logger: root });
   const taskScheduler = TaskScheduler.fromConfig(config);
+  const catalogApi = new CatalogClient({
+    discoveryApi: HostDiscovery.fromConfig(config),
+  });
 
   const identity = DefaultIdentityClient.create({
     discovery,
@@ -52,6 +58,8 @@ function makeCreateEnv(config: Config) {
     tokenManager,
   });
 
+  const eventBroker = new DefaultEventBroker(root.child({ type: 'plugin' }));
+
   root.info(`Created UrlReader ${reader}`);
 
   return (plugin: string): PluginEnvironment => {
@@ -70,6 +78,8 @@ function makeCreateEnv(config: Config) {
       scheduler,
       permissions,
       identity,
+      eventBroker,
+      catalogApi,
     };
   };
 }

diff --git a/packages/backend/src/types.ts b/packages/backend/src/types.ts
@@ -6,8 +6,10 @@ import {
   UrlReader,
 } from '@backstage/backend-common';
 import { PluginTaskScheduler } from '@backstage/backend-tasks';
+import { CatalogApi } from '@backstage/catalog-client';
 import { Config } from '@backstage/config';
 import { IdentityApi } from '@backstage/plugin-auth-node';
+import { EventBroker } from '@backstage/plugin-events-node';
 import { PermissionEvaluator } from '@backstage/plugin-permission-common';
 
 import { Logger } from 'winston';
@@ -23,4 +25,6 @@ export type PluginEnvironment = {
   scheduler: PluginTaskScheduler;
   permissions: PermissionEvaluator;
   identity: IdentityApi;
+  eventBroker: EventBroker;
+  catalogApi: CatalogApi;
 };
diff --git a/plugins/orchestrator-backend/.eslintrc.js b/plugins/orchestrator-backend/.eslintrc.js
@@ -0,0 +1 @@
+module.exports = require('@backstage/cli/config/eslint-factory')(__dirname);
diff --git a/plugins/orchestrator-backend/CHANGELOG.md b/plugins/orchestrator-backend/CHANGELOG.md
diff --git a/plugins/orchestrator-backend/OWNERS b/plugins/orchestrator-backend/OWNERS
@@ -0,0 +1,6 @@
+approvers:
+  - caponetto
+  - jkilzi
+reviewers:
+  - caponetto
+  - jkilzi
diff --git a/plugins/orchestrator-backend/README.md b/plugins/orchestrator-backend/README.md
@@ -0,0 +1,5 @@
+# Orchestrator Backend Plugin for Backstage
+
+Welcome to the backend package for the Orchestrator plugin!
+
+For more information about the Orchestrator plugin, see the [Orchestrator Plugin documentation](https://github.com/janus-idp/backstage-plugins/tree/main/plugins/orchestrator) on GitHub.
diff --git a/plugins/orchestrator-backend/app-config.janus-idp.yaml b/plugins/orchestrator-backend/app-config.janus-idp.yaml
@@ -0,0 +1,4 @@
+orchestrator:
+  sonataFlowService:
+    baseUrl: http://localhost
+    port: 8080
diff --git a/plugins/orchestrator-backend/dev/index.ts b/plugins/orchestrator-backend/dev/index.ts
@@ -0,0 +1,51 @@
+import { createServiceBuilder, UrlReader } from '@backstage/backend-common';
+import { CatalogApi } from '@backstage/catalog-client';
+import { Config } from '@backstage/config';
+import { DiscoveryApi } from '@backstage/core-plugin-api';
+import { EventBroker } from '@backstage/plugin-events-node';
+
+import { Logger } from 'winston';
+
+import { Server } from 'http';
+
+import { createRouter } from '../src/routerWrapper';
+
+export interface ServerOptions {
+  port: number;
+  enableCors: boolean;
+  logger: Logger;
+  eventBroker: EventBroker;
+  config: Config;
+  discovery: DiscoveryApi;
+  catalogApi: CatalogApi;
+  urlReader: UrlReader;
+}
+
+export async function startStandaloneServer(
+  options: ServerOptions,
+): Promise<Server> {
+  const logger = options.logger.child({ service: 'orchestrator-backend' });
+  logger.debug('Starting application server...');
+  const router = await createRouter({
+    logger: logger,
+    eventBroker: options.eventBroker,
+    config: options.config,
+    discovery: options.discovery,
+    catalogApi: options.catalogApi,
+    urlReader: options.urlReader,
+  });
+
+  let service = createServiceBuilder(module)
+    .setPort(options.port)
+    .addRouter('/orchestrator', router);
+  if (options.enableCors) {
+    service = service.enableCors({ origin: 'http://localhost:3000' });
+  }
+
+  return await service.start().catch(err => {
+    logger.error(err);
+    process.exit(1);
+  });
+}
+
+module.hot?.accept();
diff --git a/plugins/orchestrator-backend/dist-dynamic/package.json b/plugins/orchestrator-backend/dist-dynamic/package.json
@@ -0,0 +1,94 @@
+{
+  "name": "@janus-idp/backstage-plugin-orchestrator-backend-dynamic",
+  "version": "0.0.1",
+  "license": "Apache-2.0",
+  "main": "dist/index.cjs.js",
+  "types": "dist/index.d.ts",
+  "publishConfig": {
+    "access": "public",
+    "main": "dist/index.cjs.js",
+    "types": "dist/index.d.ts"
+  },
+  "backstage": {
+    "role": "backend-plugin"
+  },
+  "exports": {
+    ".": {
+      "require": "./dist/index.cjs.js",
+      "default": "./dist/index.cjs.js"
+    },
+    "./alpha": {
+      "require": "./dist/alpha.cjs.js",
+      "default": "./dist/alpha.cjs.js"
+    },
+    "./package.json": "./package.json"
+  },
+  "homepage": "https://janus-idp.io/",
+  "repository": "github:janus-idp/backstage-plugins",
+  "bugs": "https://github.com/janus-idp/backstage-plugins/issues",
+  "keywords": [
+    "backstage",
+    "plugin",
+    "orchestrator",
+    "workflows"
+  ],
+  "files": [
+    "app-config.janus-idp.yaml",
+    "dist",
+    "alpha"
+  ],
+  "scripts": {
+    "start": "backstage-cli package start",
+    "build": "backstage-cli package build",
+    "tsc": "tsc",
+    "lint": "backstage-cli package lint",
+    "test": "backstage-cli package test --passWithNoTests --coverage",
+    "clean": "backstage-cli package clean",
+    "prepack": "backstage-cli package prepack",
+    "postpack": "backstage-cli package postpack",
+    "export-dynamic": "janus-cli package export-dynamic-plugin"
+  },
+  "dependencies": {
+    "@octokit/rest": "^19.0.3",
+    "@severlessworkflow/sdk-typescript": "^3.0.3",
+    "@urql/core": "^4.1.4",
+    "cloudevents": "^8.0.0",
+    "express": "^4.18.2",
+    "express-promise-router": "^4.1.1",
+    "fs-extra": "^10.1.0",
+    "json-schema": "^0.4.0",
+    "openapi-types": "^12.1.3",
+    "winston": "^3.11.0",
+    "yn": "^5.0.0",
+    "js-yaml": "^4.1.0"
+  },
+  "devDependencies": {},
+  "bundleDependencies": true,
+  "peerDependencies": {
+    "@backstage/backend-app-api": "^0.5.8",
+    "@backstage/backend-common": "^0.19.8",
+    "@backstage/backend-plugin-api": "^0.6.6",
+    "@backstage/backend-plugin-manager": "npm:@janus-idp/[email protected]",
+    "@backstage/backend-tasks": "^0.5.11",
+    "@backstage/catalog-client": "^1.4.5",
+    "@backstage/catalog-model": "^1.4.3",
+    "@backstage/config": "^1.1.1",
+    "@backstage/core-plugin-api": "^1.7.0",
+    "@backstage/integration": "^1.7.1",
+    "@backstage/plugin-catalog-node": "^1.4.7",
+    "@backstage/plugin-events-backend": "^0.2.8",
+    "@backstage/plugin-events-node": "^0.2.8",
+    "@backstage/plugin-scaffolder-backend": "^1.18.0",
+    "@backstage/plugin-scaffolder-common": "^1.4.2",
+    "@backstage/plugin-scaffolder-node": "^0.2.6",
+    "@backstage/types": "^1.1.1"
+  },
+  "overrides": {
+    "@aws-sdk/util-utf8-browser": {
+      "@smithy/util-utf8": "^2.0.0"
+    }
+  },
+  "resolutions": {
+    "@aws-sdk/util-utf8-browser": "npm:@smithy/util-utf8@~2"
+  }
+}
Original file line number	Diff line number	Diff line change
		@@ -0,0 +1 @@
		module.exports = require('@backstage/cli/config/eslint-factory')(__dirname);