From 8c9c5c093148792d5ea68292ff112f2f97187e14 Mon Sep 17 00:00:00 2001 From: Nico Koprowski Date: Tue, 21 May 2024 14:55:32 +0200 Subject: [PATCH] docs: add INSTALL.md containing the latest installation instructions - Remove outdated admin README - Adapt references --- INSTALL.md | 247 +++++++++++++++++++++++++++++++++++++++++++ README.md | 8 +- docs/README.md | 7 +- docs/admin/README.md | 149 -------------------------- 4 files changed, 256 insertions(+), 155 deletions(-) create mode 100644 INSTALL.md delete mode 100644 docs/admin/README.md diff --git a/INSTALL.md b/INSTALL.md new file mode 100644 index 000000000..bcc9b330c --- /dev/null +++ b/INSTALL.md @@ -0,0 +1,247 @@ +# INSTALL + +BPDM offers Helm Charts, Dockerfiles and configuration files to support the installation process of the BPDM applications. +The following chapters show how to install the applications in different scenarios. + +## Local Installation + +### Requirements + +* JAVA 21 +* Maven (3.9 supported) +* Docker Engine (tested on 26.1.2) +* Docker Compose (tested on 2.27.0) + +### Default Installation + +BPDM services require a PostgreSQL database and Keycloak server to run. +Navigate to the root folder of the BPDM repository. +Then set up the necessary dependencies by using the provided Docker Compose file: + +```console +docker compose -f docker/compose/dependencies/docker-compose.yml up -d +``` + +This will run a Postgres and Keycloak with an initial realm already configured to be used by the BPDM services. +You will need to pass the necessary client secrets when starting the BPDM services. +Those client secrets are generated by Keycloak after the first time setup. +Therefore, we will log in to the Keycloak server, look-up the secrets and write them into environment variables that are read by the BPDM services. + +1. Log in to Keycloaks admin console with the admin credentials (default in the Docker Compose file): http://localhost:8180/admin/master/console +2. Navigate to realm "Cx-Central" +3. Navigate to the following clients and write the client secret to the environment variable: + +| Client | Environment Variable | +|-----------------------------------|-------------------------------| +| DUMMY-ORCHESTRATOR-TASK_PROCESSOR | BPDM_DUMMY_ORCH_CLIENT_SECRET | +| GATE-ORCHESTRATOR-TASK_CREATOR | BPDM_GATE_ORCH_CLIENT_SECRET | +| GATE-POOL-SHARING_MEMBER | BPDM_GATE_POOL_CLIENT_SECRET | +| POOL-ORCHESTRATOR-TASK-PROCESSOR | BPDM_POOL_ORCH_CLIENT_SECRET | + +This information on what client expects which environment variable is also available in the application properties files of the respective BPDM applications (like for the [gate](bpdm-gate/src/main/resources/application.yml)) +Now, make sure that client secrets are available in the expected environment variables: +```console +export BPDM_DUMMY_ORCH_CLIENT_SECRET=... +export BPDM_GATE_ORCH_CLIENT_SECRET=... +export BPDM_GATE_POOL_CLIENT_SECRET=... +export BPDM_POOL_ORCH_CLIENT_SECRET=... +``` + +After this step, we can finally build, install the BPDM applications. + +```console +mvn clean install +``` + +Each BPDM application can now be run separately. +We will run each application after another. +For this, we navigate into the application's subfolder and run the application with the spring-boot run goal. + +```console +cd bpdm-orchestrator +mvn spring-boot:run +cd ../bpdm-cleaning-service-dummy +mvn spring-boot:run +cd ../bpdm-pool +mvn spring-boot:run +cd ../bpdm-gate +mvn spring-boot:run +``` + +After this you have full-fledged running BPDM system. +As a next step on how to pass business partner data and create golden records you can have a look at the [api documentation](docs/api/README.md) of this repository. + +### Insecure Installation + +You may want to perform a quick installation in which security is not necessary. +In this case, you don't need to configure any environment variables. +This means you can directly run the BPDM applications after you set up the necessary dependencies. +Make sure to disable authentication requirements by using the provided `no-auth` profile when running the applications: + +```console +mvn spring-boot:run -Dspring.profiles.active=no-auth +``` + + +## Helm Charts + +Installation of BPDM applications with the Helm Charts has the most software requirements but is the qickest way to set up a running system. + +### Requirements + +* kubectl (1.30 supported) +* Docker Engine (tested on 26.1.2) +* Minikube (tested on 1.33.0) +* Helm (tested on 3.14.4) + + +### Default Installation + +Navigate to the projects root folder. +Then install a new release of BPDM on your default namespace via helm: + +```console +helm install bpdm ./charts/bpdm +``` +This will install the BPDM applications with its own Postgres and Keycloak in default values. +Mind that this will also install the applications with default passwords. + +### Override Default Secrets + +It is good practice to overwrite the default secrets and passwords that are used in the BPDM Charts. +For this, you can first define a bunch of environment variables holding new secret values and use them later during deployment: + +```console +helm install bpdm \ + --set-value postgres.auth.password=$BPDM_POSTGRES \ + --set-value keycloak.auth.adminPassword=$BPDM_KEYCLOAK_ADMIN \ + --set-value keycloak.bpdm.realm.clientSecrets.cleaningDummyOrchestrator=$BPDM_DUMMY_ORCH_CLIENT_SECRET \ + --set-value keycloak.bpdm.realm.clientSecrets.poolOrchestrator=$BPDM_POOL_ORCH_CLIENT_SECRET \ + --set-value keycloak.bpdm.realm.clientSecrets.gateOrchestrator=$BPDM_GATE_ORCH_CLIENT_SECRET \ + --set-value keycloak.bpdm.realm.clientSecrets.gatePool=$BPDM_GATE_POOL_CLIENT_SECRET \ + --set-value bpdm-gate.applicationSecrets.bpdm.client.orchestrator.registration=$BPDM_GATE_ORCH_CLIENT_SECRET \ + --set-value bpdm-gate.applicationSecrets.bpdm.client.pool.registration=$BPDM_GATE_POOL_CLIENT_SECRET \ + --set-value bpdm-pool.applicationSecrets.bpdm.client.orchestrator.registration=$BPDM_POOL_ORCH_CLIENT_SECRET \ + --set-value bpdm-cleaning-service-dummy.applicationSecrets.bpdm.client.orchestrator.registration=$BPDM_DUMMY_ORCH_CLIENT_SECRET\ + .charts/bpdm/bpdm +``` + +### Insecure Installation + +For non-production purposes you may want to install BPDM applications that are not authenticated. +All BPDM applications offer a Spring profile to quickly remove all authentication configuration for their APIs and client connections. +In this case you can also disable the Keycloak dependency from being deployed. + +```console +helm install bpdm \ + --set-value keycloak.enabled=false + --set-value bpdm-gate.profiles=["no-auth"] \ + --set-value bpdm-orchestrator.profiles=["no-auth"] \ + --set-value bpdm-pool.profiles=["no-auth"] \ + --set-value bpdm-cleaning-service-dummy.profiles=["no-auth"] + .charts/bpdm/bpdm +``` + +You can also more fine-granulary remove authentication on APIs and BPDM client connections. +You can refer to the no-auth profile configurations (for example that of the [Gate](bpdm-gate/src/main/resources/application-no-auth.yml)) as a documentation. + +### Use External Dependencies + +The BPDM Charts deploy their own PostgreSQL and Keycloak dependencies. +However, for production it is recommended to host dedicated Postgres and Keycloak instances with which the BPDM applications should connect to. + +#### Additional Requirements + + * Postgres (15.4.0 supported) + * Keycloak (22.0.3 supported) + +#### Installation + +In this case, you can disable the dependencies and configure the connection to external systems in the application configuration. + +```console +helm install bpdm \ + --set-value keycloak.enabled=false + --set-value postgres.enabled=false + --set-value bpdm-gate.applicationConfig.bpdm.datasource.host=external-db \ + --set-value bpdm-gate.applicationConfig.bpdm.security.auth-server-url=http://external-keycloak \ + --set-value bpdm-pool.applicationConfig.bpdm.datasource.host=external-db \ + --set-value bpdm-pool.applicationConfig.bpdm.security.auth-server-url=http://external-keycloak \ + --set-value bpdm-orchestrator.applicationConfig.bpdm.security.auth-server-url=http://external-keycloak \ + --set-value bpdm-cleaning-service-dummy.applicationConfig.bpdm.client.orchestrator.provider.issuer-uri= http://external-keycloak/realms/CX-Central \ + .charts/bpdm/bpdm +``` + +### Fine-granular Configuration + +You can configure all BPDM applications over Helm values more fine-granulary via the `applicationConfig` and `applicationSecrets`. +Values under these groups are directly injected as application properties in the deployed containers. + +As a reference of what can be changed have a look at the respective application properties files of each application: +- [BPDM Gate](bpdm-gate/src/main/resources/application.yml) +- [BPDM Pool](bpdm-pool/src/main/resources/application.yml) +- [BPDM Orchestrator](bpdm-orchestrator/src/main/resources/application.yml) +- [BPDM Cleaning Service Dummy](bpdm-cleaning-service-dummy/src/main/resources/application.yml) + + +## EDC Installation + +This section shows how to make your BPDM Gate and Pool APIs available over an EDC. +This documentation assumes that you already have running BPDM and EDC deployments. +For deploying an EDC please consult the documentation on the [EDC repository](https://github.com/eclipse-tractusx/tractusx-edc). + +### Requirements + +* Running BPDM applications +* Running EDC (0.7.0 supported) + +### Installation + +The general idea of configuring data offers for BPDM is to assets which grant access to a portion of the BPDM APIs. +Which API resources are accessible over an asset is determined by the purposes defined in the BPDM framework agreement. +For some purposes you may need to access business partner output data from the BPDM Gate for example but won't have access to the input data. +Blueprints for such assets are documented in this [POSTMAN collection](../postman/EDC Provider Setup.postman_collection.json). +Accompanying the asset definitions are Policy and Contract Definition blueprints. +Except for a general Access Policy those blueprints are grouped by purpose. + +After all assets, policies and contract definitions are configured a sharing company's EDC now can query its available assets and the contract under which they +are exposed. + +## Post-Run Configuration + +For the most part BPDM applications don't need to be further configured after they started. +The BPDM Pool is an exception to that rule. +The Pool offers endpoints for operators to regulate available metadata. +Metadata are supporting business partner information like legal forms, identifier types or administrative level 1 areas. +For example, by adding a range of legal forms operators are able to determine the available legal forms business partners in the Pool can have. + +Typically, business partner data references metadata via technical keys. +If a technical key does not exist in the respective metadata the Pool rejects the record. + +Administrative level 1 areas follows the ISO 3166-2 norm and is filled by default. +Such metadata does not need to be added by the operator. + +The [use case Postman collection](../postman/BPDM%20Tests.postman_collection.json) shows which metadata can be added and how that is done. +Refer to use cases for Operators(O): + +- CL: Shows how to add available legal forms +- CIL: Shows how to add available identifier types used for legal entities +- CIA: Shows how to add available identifier types used for addresses + +Please note that in the current implementation there are no endpoints to delete metadata. +Deletions would need to be done directly in the database. + + +## NOTICE + +This work is licensed under the [Apache-2.0](https://www.apache.org/licenses/LICENSE-2.0). + +- SPDX-License-Identifier: Apache-2.0 +- SPDX-FileCopyrightText: 2023,2024 ZF Friedrichshafen AG +- SPDX-FileCopyrightText: 2023,2024 SAP SE +- SPDX-FileCopyrightText: 2023,2024 Bayerische Motoren Werke Aktiengesellschaft (BMW AG) +- SPDX-FileCopyrightText: 2023,2024 Mercedes Benz Group +- SPDX-FileCopyrightText: 2023,2024 Robert Bosch GmbH +- SPDX-FileCopyrightText: 2023,2024 Schaeffler AG +- SPDX-FileCopyrightText: 2023,2024 Contributors to the Eclipse Foundation +- Source URL: https://github.com/eclipse-tractusx/bpdm \ No newline at end of file diff --git a/README.md b/README.md index 89c7d8e63..bdcfabd7e 100644 --- a/README.md +++ b/README.md @@ -29,10 +29,12 @@ Subfolders for BPDM applications are easily recognizable by the `bpdm` prefix. ## Installation -Installation instructions for the BPDM services can be found in the following places: +Please consult the [INSTALL](INSTALL.md) documentation file for in-depth installation instructions. -1. [Local Installation](docs/admin/README.md): Details how to install and configure the BPDM services on a host machine. -2. [Helm Installation](charts/bpdm/README.md): Explains how to use given Helm Charts to install the BPDM services on a kubernetes environment. +## Usage + +BPDM is an application environment designed to be interacted with over APIs. +Therefore, please consult the [api](docs/api/README.md) documentation for getting to know how to use BPDM. ## GitHub Workflows diff --git a/docs/README.md b/docs/README.md index 629b458fe..9b19365bd 100644 --- a/docs/README.md +++ b/docs/README.md @@ -3,6 +3,7 @@ This folder contains supporting documentation for the BPDM applications. A good entrypoint are the `views` providing guidance for different interest groups: -- [Adoption view](api/README.md): You want to integrate BPDM APIs -- [Operator view](admin/README.md): You want to operate BPDM applications -- [Developer view](developer/README.md): You want to contribute to the development of BPDM \ No newline at end of file +- [API](api/README.md): You want to integrate BPDM APIs +- [Installation](../INSTALL.md): You want to install and operate BPDM applications +- [Development](developer/README.md): You want to contribute to the development of BPDM +- [Architecture](arc42/arc42-bpdm.md): You are interested in the general architecture of BPDM \ No newline at end of file diff --git a/docs/admin/README.md b/docs/admin/README.md deleted file mode 100644 index 6dd349ca9..000000000 --- a/docs/admin/README.md +++ /dev/null @@ -1,149 +0,0 @@ -# Operator View - -Here you can find documentation about running, configuring and installing the BPDM services. - -## Running a BPDM application - -### Prerequisites - -* Maven -* JDK17 -* Docker and Docker Compose -* Keycloak (with enabled `auth` profile) - -Each BPDM application is a SpringBoot Kotlin software project managed by Maven. -To run the application first you need to install it from the parent pom. For that go to the root folder and then there are two options to run the project: - -1. `mvn clean install -DskipTests` - This will install all the dependencies without need for test execution. -2. Install missing docker images used by the bridge-dummy tests `docker compose -f docker-compose.build.yml build`. After that run `mvn clean install`. - -After installation, go to the subfolder of the application you would like to run and use the following command: `mvn spring-boot:run` - -When running, the application may require a PostgreSQL database instance to be available to connect to. You can set up these dependencies manually, or you can -use the -provided [docker-compose file](../docker-compose.yml) which will start the necessary dependencies for you. - -To use Docker Compose, navigate to the project's root directory and run the following command: - -```bash -docker-compose up -d -``` - -Per default configuration the application expects postgres to run on `localhost` on port `5432`. -After the application runs you can use it's generated swagger documentation to access the API. -Caution: The default port of each application is different, thus making it possible to run them next to each other. -You can find and edit the default configuration application properties files in the `resources` folder. -For example, the REST API documentation for the Pool can be accessed at . - -## Configuring a BPDM application - -The default configuration of the application is determined by the `application.yml` file in the `resources` folder. -Here you can find core application configuration such as Swagger documentation, server port, Actuator and Spring datasource (currently, developed against -PostgreSQL). - -One notable configuration for applications are the BPDM clients configuration. -BPDM applications can typically connect to other BPDM applications to exchange business partner data information. -The default configuration for those clients to connect to other BPDM applications connects against the default ports of those applications. -BPDM applications connect to other BPDM applications on schedule by polling. -For that, the scheduling time needs to be configured since on default scheduling is turned off. - -You can also run the project with Spring profiles that set property bundles which enable additional features of the application. -These are the available profiles: - -1. auth: Each application with API has this profile. It enables authentication and authorization of the API. -2. client-auth: An application with a BPDM client has a profile to activate authentication and authorization for the access with that client. - -In order to run the application with a specific profile you can use the appropriate maven flag `Dspring.profiles.active`. - -For example, the command `mvn clean spring-boot:run -Dspring.profiles.active=auth` starts the application with additional `auth` configuration enabled. - -The following sections detail the configuration properties for each profile. - -### Auth - -`application-auth.yml` enables authorization of endpoints and configures the connection to a Keycloak instance on which the authorization relies on. -The application expects the Keycloak to run on `localhost` on port `8180`. -However, as with the Spring datasource connection, the connection to the Keycloak can be freely configured. -The application uses the configured auth server URL to validate incoming tokens. - -For authorization purposes the application checks incoming token's permissions with names as defined per application. -Note that the token needs to have these permissions in the client/resource level and not on the realm level. -Each application has its own set of available permissions. Please refer to the application's properties file for further details. -Each permission's default name can be overwritten if so needed. - -This profile also enables/disables the oauth2 login form in the auto-generated Swagger documentation. -The Swaggger-UI offers several oauth2 authentication flows, including providing a Bearer token that will be passed to the BPDM API in the header. - -### Client Auth - -We already established before that BPDM applications have client configuration to connect to other BPDM applications. -If those other BPDM applications are authenticated (auth profile activated) you would need to configure an authentication logic for that connection as well. -Therefore, each BPDM application with a BPDM client also comes with a matching client auth profile. -Activating such a profile configures authentication that matches the default auth configuration of the BPDM application to connect to. - -The actual name of the profile has the pattern `client-auth` where `client` is the name of the BPDM application to connect to. -For example, the BPDM pool has a profile called `orchestrator-auth`. - -### Specific Application Configuration - -For specific configuration options of a BPDM application please refer to the application's property files in the resource folder. -These property files count to the operation views documentation. - -## Installation via Docker - -Each BPDM application has an exemplary Dockerfile that can be used to install the BPDM application in a container. -Please refer to the [README](../../README.md#container-images) for more information. - -## Installation via Helm - -This repository contains Helm files for deploying BPDM Applications to a Kubernetes environment. -See the [BPDM Chart](../../charts/bpdm) for details. - -## Post-Run Configuration - -For the most part BPDM applications don't need to be further configured after they started. -The BPDM Pool is an exception to that rule. -The Pool offers endpoints for operators to regulate available metadata. -Metadata are supporting business partner information like legal forms, identifier types or administrative level 1 areas. -For example, by adding a range of legal forms operators are able to determine the available legal forms business partners in the Pool can have. - -Typically, business partner data references metadata via technical keys. -If a technical key does not exist in the respective metadata the Pool rejects the record. - -Administrative level 1 areas follows the ISO 3166-2 norm and is filled by default. -Such metadata does not need to be added by the operator. - -The [use case Postman collection](../postman/BPDM%20Tests.postman_collection.json) shows which metadata can be added and how that is done. -Refer to use cases for Operators(O): - -- CL: Shows how to add available legal forms -- CIL: Shows how to add available identifier types used for legal entities -- CIA: Shows how to add available identifier types used for addresses - -Please note that in the current implementation there are no endpoints to delete metadata. -Deletions would need to be done directly in the database. - -## EDC Setup - -While the BPDM applications currently do not connect to an EDC in order to access other APIs you may want to expose BPDM assets over an EDC. - -This section details how to configure an EDC that provides access to the BPDM API for a company to share and query business partner data. -This documentation assumes that you already have running BPDM and EDC deployments. -For deploying an EDC please consult the documentation on the [EDC repository](https://github.com/eclipse-tractusx/tractusx-edc). - -The general idea of configuring data offers for BPDM is to assets which grant access to a portion of the BPDM APIs. -Which API resources are accessible over an asset is determined by the purposes defined in the BPDM framework agreement. -For some purposes you may need to access business partner output data from the BPDM Gate for example but won't have access to the input data. -Blueprints for such assets are documented in this [POSTMAN collection](../postman/EDC Provider Setup.postman_collection.json). -Accompanying the asset definitions are Policy and Contract Definition blueprints. -Except for a general Access Policy those blueprints are grouped by purpose. - -After all assets, policies and contract definitions are configured a sharing company's EDC now can query its available assets and the contract under which they -are exposed. - -## NOTICE - -This work is licensed under the [Apache-2.0](https://www.apache.org/licenses/LICENSE-2.0). - -- SPDX-License-Identifier: Apache-2.0 -- Source URL: https://github.com/eclipse-tractusx/bpdm \ No newline at end of file