Catena-X Knowledge Agents KIT (Hey Catena!) Source Repository

This is a MonoRepo hosting or linking all the module and infrastructure codes related to the Hey Catena! KIT.

• See this copyright notice
• See the authors file
• See the license file
• See the code of conduct
• See the contribution guidelines
• See the dependencies and their licenses

Repository Linking and Initialisation

The individual sources may be maintained in separate repositories (currently: none).

If they are (currently: not), they would be linked as git submodules so you should be sure to run the following command after cloning this repo:

git submodule update --init

You may open this repository in a Github Codespace. Be sure to use a "4-core" or bigger machine type since some of the docker images need a certain amount of memory and horsepower.

Modules

These are the sub-modules of the Hey Catena! product (and their respective sub-folders)

• Ontology hosts the CX domain ontologies (including the fully merged CX ontology and tooling) describing the semantics of the Catena-X Dataspace.
• Dataspace hosts reference implementations of the Gaia-X/IDS Dataspace extensions (Agents) for both Providers and Consumers which the semantics of Catena-X.
• UX hosts User Experience components and a sample portal/development environment for developing and executing semantically-driven logic and apps.
• Infrastructure hosts "Infrastructure as Code" descriptions for easy deployment of above artifacts including a sample dataspace consisting of three tenants (Consumer, OEM and supplier).

Besides the markdown documentation including this file, we have some helper folders

• Github contains all workflows and CI/CD processes.
  • Github CI/CD Workflow builds and unit-tests all artifacts, checks source and binary code quality using CodeQL and publishes the results (only main branch).
  • Github KICS Workflow checks Docker Buildfiles and Helm Charts for the most common vulnerabilities.
• Maven contains bootstrap code for the main build system.

And some related scripts and settings

• Git Settings
  • Attributes has large-file settings for binary artifacts.
  • Ignore excludes certain build artifacts from versioning.
• Maven Scripts
  • Maven Wrapper(For Windows) for bootstrapping the build system.
  • Maven Pom describing the KA root module and common build steps.
  • Maven Settings configuring the associated artifact repository credentials.
• Postman Collections and Environments
  • Knowledge Agents contains API interactions and samples for all REST-based interfaces of the KA architecture. Includes integration test steps. Excludes conformity asessment methods. Should be combined with one of the following environments
  • Localhost contains API endpoints and secrets for interacting with the local deployment of the sample dataspace.
  • Development contains API endpoints and (emptied) secrets for interacting with the development deployment of the sample dataspace. Please contact the CX association to get the actual secrets.
  • Localhost contains API endpoints and (emptied) secrets for interacting with the integration deployment of the sample dataspace. Please contact the CX association to get the actual secrets.
• Conda Environment for setting up python.

Build

Environment Variables

To interact with the required package and container registries, the following environment variables should be set

export GITHUB_ACTOR=
export GITHUB_TOKEN=

Prepare

A suitable conda environment named knowledgeagents can be created and activated with:

conda env create -f environment.yaml
conda activate knowledgeagents

Ontology Create

Creating a new ontology excel source can be done by invoking

python 
>>> import ontology.ontology_tools.create_ontology as co
>>>  co.create_ontology_table('test','Schorsch','1.0.0')

Ontology Merge

Creating a merged ontology out of several domain ontologies may be done by invoking

python -m ontology.ontology_tools.merge_ontology vehicle_ontology.ttl load_spectrum_ontology.ttl vehicle_information_ontology.ttl part_ontology.json vehicle_component.ttl

Compile

To build all compilation artifacts (without tests), you can invoke

./mvnw -s settings.xml install -DskipTests

Test

To build all compilation artifacts (including tests), you can invoke

./mvnw -s settings.xml install

Package and Deploy

To bundle all deployment artifacts for a particular platform (currently supported: linux/amd64 and linux/arm64), you can invoke

./mvnw -s settings.xml package -Pwith-docker-image -Dplatform=linux/amd64

To deploy the artifacts, choose

./mvnw -s settings.xml deploy -Pwith-docker-image -Dplatform=linux/amd64

Containerizing, Registering and Deployment

Knowledge Agents builds all containers using docker technology. The docker buildfiles are part of the respective source code repositories.

Containers

To build all artifacts (including compilation artifacts, tests and container images with the default target platform linux/amd64), you can invoke

./mvnw -s settings.xml install -Pwith-docker-image

To build all artifacts especially for target platform linux/arm64, use this command

./mvnw -s settings.xml install -Dplatform=linux/arm64 -Pwith-docker-image

Registry

To register all artifacts (including compilation artifacts, tests and container images) into their respective registries, you can invoke

./mvnw -s settings.xml deploy -Pwith-docker-image

Deployment

Knowledge Agents containers will be deployed very individually.

We provide a sample environment (dataspace consisting of three business partners) using docker-compose (for local deployment) and helm (for cloud/cluster deployment) technology.

The docker compose files and helm charts can be found in the infrastructure folder.

Running Against the Services and APIs / Integration Tests

You may use/export/fork this online Postman Workspace/Collecion a copy of which is embedded here.

It contains collection of sample interactions with the various sub-products in several environments (e.g. local, development and integration) and tailored to the sample dataspace.

Also integrated there is a folder with the integrations tests which are scripted and consective Postman actions which test features and state changes within the target environment. This is used in the Github Integration Test Workflow.