diff --git a/README.md b/README.md index b367fe66..aeb64ac4 100644 --- a/README.md +++ b/README.md @@ -57,6 +57,15 @@ custom code for their enforcement. All policy samples are located in the `policy` directory. +### [Federated Catalog](./federated-catalog/README.md) + +These samples focus on the implementation of federated catalogs, covering scenarios such as deploying it as +a standalone runtime or as part of a connector. This also includes demonstration of how to implement a target +node resolver, which resolves the participant connectors in the dataspace and crawls these connectors to compile +a set of all offered catalogs. + +All federated catalog samples are located in the [`federated-catalog`](./federated-catalog/README.md) directory. + ## Contributing See [how to contribute](https://github.com/eclipse-edc/docs/blob/main/CONTRIBUTING.md). diff --git a/federated-catalog/README.md b/federated-catalog/README.md index 0e9ad868..ff7a0bdc 100644 --- a/federated-catalog/README.md +++ b/federated-catalog/README.md @@ -8,14 +8,16 @@ participants in the dataspace. To accomplish this, the FC utilizes crawlers that periodically crawl the catalogs from each participant and store this list of catalogs in a local cache. By maintaining this locally cached version of catalogs, it eliminates the need to query -each participant individually, resulting in faster and more reliable queries. +each participant individually, resulting in faster and more reliable queries. Refer to the +[eclipse-edc/FederatedCatalog](https://github.com/eclipse-edc/FederatedCatalog) for further details. The following samples shows how to * implement, build and run different versions of FC e.g. * standalone, * embedded. - +* implement TargetNodeDirectory and resolve Target Nodes, + * from a static file containing all participants' DSP endpoints, ## Samples @@ -38,3 +40,8 @@ a standalone federated catalog will not have the added functionalities of a conn exposes a catalog API that serves the list of catalogs. --- +### Different Implementations of Node Resolver + +### [FC sample 03](./fc-03-static-node-directory/README.md): Resolve Target Catalog Nodes from static participant file +This sample demonstrates a Catalog Node resolver, that implements TargetNodeDirectory. It resolves the Target Catalog +Nodes from a static participant file containing the DSP endpoints of the participants. \ No newline at end of file diff --git a/federated-catalog/fc-00-basic/README.md b/federated-catalog/fc-00-basic/README.md index 13cd65f1..ffbd7b14 100644 --- a/federated-catalog/fc-00-basic/README.md +++ b/federated-catalog/fc-00-basic/README.md @@ -29,8 +29,8 @@ Any further dependencies will be added in the later samples based on their use c ### fixed-node-resolver The Federated Catalog requires a list of Target Catalog Nodes, which are essentially the DSP endpoints of the dataspace participants. The catalog crawler then crawls these listed endpoints to collect their offered catalogs. -This list of Target Nodes is resolved by a Catalog Node Resolver which implements the [TargetNodeDirectory](https://github.com/eclipse-edc/FederatedCatalog/blob/main/spi/crawler-spi/src/main/java/org/eclipse/edc/crawler/spi/TargetNodeDirectory.java). -Check out [eclipse-edc/FederatedCatalog](https://github.com/eclipse-edc/FederatedCatalog/tree/main) for further information on this topic. +This list of Target Nodes is resolved by a Catalog Node Resolver which implements the `TargetNodeDirectory`. +Check out [eclipse-edc/FederatedCatalog](https://github.com/eclipse-edc/FederatedCatalog) for further information on this topic. In this module, we've included a fixed Node Resolver, [fixed-node-resolver](./fixed-node-resolver) @@ -47,7 +47,8 @@ of the federated catalogs that we are going to build in sample When the federated catalog boots up, the crawler begins periodically invoking the Target Nodes returned by the Node Resolver and collecting the catalogs offered by these nodes. To test whether our federated catalogs -(which we will build in later samples: [fc-01-embedded](../fc-01-embedded) and [fc-02-standalone](../fc-02-standalone)) can successfully request and retrieve these catalogs, we need at least one connector with a contract offer. +(which we will build in later samples: [fc-01-embedded](../fc-01-embedded) and [fc-02-standalone](../fc-02-standalone)) +can successfully request and retrieve these catalogs, we need at least one connector with a contract offer. Therefore, in this section, we will start a connector and then create a contract for this connector. In the future samples, we will refer to it as `participant-connector`. diff --git a/federated-catalog/fc-00-basic/federated-catalog-base/build.gradle.kts b/federated-catalog/fc-00-basic/federated-catalog-base/build.gradle.kts index bbbffbda..87a67ea3 100644 --- a/federated-catalog/fc-00-basic/federated-catalog-base/build.gradle.kts +++ b/federated-catalog/fc-00-basic/federated-catalog-base/build.gradle.kts @@ -18,7 +18,6 @@ plugins { } dependencies { - implementation(libs.edc.fc.spi.crawler) runtimeOnly(libs.edc.fc.core) runtimeOnly(libs.edc.fc.ext.api) } diff --git a/federated-catalog/fc-03-static-node-directory/README.md b/federated-catalog/fc-03-static-node-directory/README.md new file mode 100644 index 00000000..ad02ad19 --- /dev/null +++ b/federated-catalog/fc-03-static-node-directory/README.md @@ -0,0 +1,139 @@ +# Target Node Resolver - Static Node Directory +The Federated Catalog requires a list of Target Catalog Nodes (TCN), which are essentially the participant connectors in the dataspace. +The catalog crawler then crawls the DSP endpoints of these listed nodes, and stores the consolidated set of catalogs in a Federated Catalog Cache (FCC). + + +This list of Target Catalog Nodes, represented by `TargetNodes`, +is provided by the `TargetNodeDirectory`. +This `TargetNodeDirectory` serves as a 'phone book', maintaining specific information about the +dataspace participants. It accepts an initial list of participants (e.g. list of participants' +IDs), and resolves this input to a list of TargetNodes. + +The initial participant list may vary in its source and format depending on specific use cases. +To accommodate these variations, different implementations of the TargetNodeDirectory can be +adapted to customize the resolution process of Target Nodes from the provided participant list. +In this sample, we will build a Catalog Node Resolver that reads the participants' data from a +static file, [participants.json](./target-node-resolver/src/main/resources/participants.json) +and resolves it into TargetNodes. + + +The code in this sample has been organized into several Java modules: +- `target-node-resolver`: contains `CatalogNodeDirectory`, an implementation of +`TargetNodeDirectory`, which accepts the [`participants.json`](./target-node-resolver/src/main/resources/participants.json) +and returns a list of TargetNodes. +- `embedded|standalone-fc-with-node-resolver`: the embedded/ standalone federated catalog that will be using the `catalog-node-resolver`. + + +## Implement the Catalog Node Resolver + +### Participant file +To keep things straightforward, in this sample we will store our participant list in a static +json file, [participant.json](./target-node-resolver/src/main/resources/participants.json), that contains the `TargetNode` +properties of the dataspace participants. +In this case, the file contains the properties of the `participant-connector` from [fc-00-basic](../fc-00-basic). + +```json +{ + "name": "https://w3id.org/edc/v0.0.1/ns/", + "id": "provider", + "url": "http://localhost:19194/protocol", + "supportedProtocols": ["dataspace-protocol-http"] +} +``` +However, this solution is intended for use only within the sample scope; in production, it must be managed in different way. + +### Target Node Resolver + +The [CatalogNodeDirectory](./target-node-resolver/src/main/java/org/eclipse/edc/sample/extension/fc/CatalogNodeDirectory.java) +implements TargetNodeDirectory and overrides its `getAll()` method. +In our implementation, this method maps the file content of [`participant.json`](./target-node-resolver/src/main/resources/participants.json) +to a list of TargetNodes. + +```java +public class CatalogNodeDirectory implements TargetNodeDirectory { + //... + @Override + public List getAll() { + try { + return objectMapper.readValue(participantFileContent, new TypeReference<>() {}); + } catch (IOException e) { + throw new RuntimeException(e); + } + } + //... +} +``` +During the preparation phase of a crawler run, the FC ExecutionManager invokes this method +to obtain the list of TargetNodes. +The crawler requests the DSP endpoints of the participants and stores the +aggregated catalogs in a Federated Catalog Cache (FCC). +In this example we are using the in-memory implementation of an FCC. + +## Run Federated Catalog with Node Resolver + +Previously, we discussed the implementation of standalone and embedded FCs. +In this example, we introduce two separate modules,`standalone-fc-with-node-resolver` +and `embedded-fc-with-node-resolver`, which demonstrate the implementation of each type +of federated catalogs that uses the node resolver`. + +Before requesting each of the federated catalog APIs, make sure the `partcipant-connector` that we have set up in the +[fc-00-basic](../fc-00-basic/README.md) is running, and it has a contract offer. + +### Run standalone-fc with Node Resolver +Apply the following steps to run a standalone federated catalog that uses the implemented static `target-node-resolver`. + +#### Build the standalone-fc JAR +Execute this command in project root: +```bash +./gradlew federated-catalog:fc-03-static-node-directory:standalone-fc-with-node-resolver:build +``` + +#### Run the standalone-fc + +To run the federated catalog, execute the following command + +```shell +java -Dedc.fs.config=federated-catalog/fc-02-standalone/standalone-fc/config.properties -jar federated-catalog/fc-03-static-node-directory/standalone-fc-with-node-resolver/build/libs/standalone-fc-with-node-resolver.jar +``` + +If the execution is successful, then the Catalog API of our standalone FC will listen on port `39195`. + +#### Test catalog query API + +To get the combined set of catalogs, use the following request: + +```http request +curl -d @federated-catalog/fc-01-embedded/resources/empty-query.json \ + -H 'content-type: application/json' http://localhost:39195/api/catalog/v1alpha/catalog/query \ + -s | jq +``` + +### Run embedded-FC with Node Resolver +Apply the following steps to run an embedded federated catalog connector that uses the implemented static `target-node-resolver`. + +#### Build the fc-connector JAR +Execute this command in project root: + +```bash +./gradlew federated-catalog:fc-03-static-node-directory:embedded-fc-with-node-resolver:build +``` + +#### Run the fc-connector + +To run the federated catalog, execute the following command + +```shell +java -Dedc.fs.config=federated-catalog/fc-01-embedded/fc-connector/config.properties -jar federated-catalog/fc-03-static-node-directory/embedded-fc-with-node-resolver/build/libs/fc-connector-with-node-resolver.jar +``` + +If the execution is successful, then the Catalog API of our standalone FC will listen on port `29195`. + +#### Test catalog query API + +To get the combined set of catalogs, use the following request: + +```http request +curl -d @federated-catalog/fc-01-embedded/resources/empty-query.json \ + -H 'content-type: application/json' http://localhost:29195/api/catalog/v1alpha/catalog/query \ + -s | jq +``` \ No newline at end of file diff --git a/federated-catalog/fc-03-static-node-directory/embedded-fc-with-node-resolver/build.gradle.kts b/federated-catalog/fc-03-static-node-directory/embedded-fc-with-node-resolver/build.gradle.kts new file mode 100644 index 00000000..c6302151 --- /dev/null +++ b/federated-catalog/fc-03-static-node-directory/embedded-fc-with-node-resolver/build.gradle.kts @@ -0,0 +1,47 @@ +/* + * Copyright (c) 2024 Fraunhofer-Gesellschaft + * + * This program and the accompanying materials are made available under the + * terms of the Apache License, Version 2.0 which is available at + * https://www.apache.org/licenses/LICENSE-2.0 + * + * SPDX-License-Identifier: Apache-2.0 + * + * Contributors: + * Fraunhofer-Gesellschaft - initial API and implementation + * + */ + +plugins { + `java-library` + id("application") + alias(libs.plugins.shadow) +} + +dependencies { + runtimeOnly(project(":federated-catalog:fc-00-basic:federated-catalog-base")) + runtimeOnly(project(":federated-catalog:fc-03-static-node-directory:target-node-resolver")) + + implementation(libs.edc.connector.core) + implementation(libs.edc.control.plane.core) + implementation(libs.edc.configuration.filesystem) + implementation(libs.edc.management.api) + implementation(libs.edc.dsp) + implementation(libs.edc.iam.mock) + implementation(libs.edc.http) + implementation(libs.edc.edr.store.core) + +} + +application { + mainClass.set("org.eclipse.edc.boot.system.runtime.BaseRuntime") +} + +var distTar = tasks.getByName("distTar") +var distZip = tasks.getByName("distZip") + +tasks.withType { + mergeServiceFiles() + archiveFileName.set("fc-connector-with-node-resolver.jar") + dependsOn(distTar, distZip) +} diff --git a/federated-catalog/fc-03-static-node-directory/standalone-fc-with-node-resolver/build.gradle.kts b/federated-catalog/fc-03-static-node-directory/standalone-fc-with-node-resolver/build.gradle.kts new file mode 100644 index 00000000..dcac9036 --- /dev/null +++ b/federated-catalog/fc-03-static-node-directory/standalone-fc-with-node-resolver/build.gradle.kts @@ -0,0 +1,48 @@ +/* + * Copyright (c) 2024 Fraunhofer-Gesellschaft + * + * This program and the accompanying materials are made available under the + * terms of the Apache License, Version 2.0 which is available at + * https://www.apache.org/licenses/LICENSE-2.0 + * + * SPDX-License-Identifier: Apache-2.0 + * + * Contributors: + * Fraunhofer-Gesellschaft - initial API and implementation + * + */ + +plugins { + `java-library` + id("application") + alias(libs.plugins.shadow) +} + +dependencies { + runtimeOnly(project(":federated-catalog:fc-00-basic:federated-catalog-base")) + runtimeOnly(project(":federated-catalog:fc-03-static-node-directory:target-node-resolver")) + + implementation(libs.edc.connector.core) + runtimeOnly(libs.edc.boot) + runtimeOnly(libs.edc.control.plane.core) + implementation(libs.edc.configuration.filesystem) + runtimeOnly(libs.edc.token.core) + implementation(libs.edc.http) + runtimeOnly(libs.edc.dsp) + implementation(libs.edc.iam.mock) + +} + + +application { + mainClass.set("org.eclipse.edc.boot.system.runtime.BaseRuntime") +} + +var distTar = tasks.getByName("distTar") +var distZip = tasks.getByName("distZip") + +tasks.withType { + mergeServiceFiles() + archiveFileName.set("standalone-fc-with-node-resolver.jar") + dependsOn(distTar, distZip) +} diff --git a/federated-catalog/fc-03-static-node-directory/target-node-resolver/build.gradle.kts b/federated-catalog/fc-03-static-node-directory/target-node-resolver/build.gradle.kts new file mode 100644 index 00000000..766f3d27 --- /dev/null +++ b/federated-catalog/fc-03-static-node-directory/target-node-resolver/build.gradle.kts @@ -0,0 +1,22 @@ +/* + * Copyright (c) 2024 Fraunhofer-Gesellschaft + * + * This program and the accompanying materials are made available under the + * terms of the Apache License, Version 2.0 which is available at + * https://www.apache.org/licenses/LICENSE-2.0 + * + * SPDX-License-Identifier: Apache-2.0 + * + * Contributors: + * Fraunhofer-Gesellschaft - initial API and implementation + * + */ + +plugins { + `java-library` + id("application") +} + +dependencies { + implementation(libs.edc.fc.spi.crawler) +} diff --git a/federated-catalog/fc-03-static-node-directory/target-node-resolver/src/main/java/org/eclipse/edc/sample/extension/fc/CatalogNodeDirectory.java b/federated-catalog/fc-03-static-node-directory/target-node-resolver/src/main/java/org/eclipse/edc/sample/extension/fc/CatalogNodeDirectory.java new file mode 100644 index 00000000..bd18e106 --- /dev/null +++ b/federated-catalog/fc-03-static-node-directory/target-node-resolver/src/main/java/org/eclipse/edc/sample/extension/fc/CatalogNodeDirectory.java @@ -0,0 +1,49 @@ +/* + * Copyright (c) 2024 Fraunhofer-Gesellschaft + * + * This program and the accompanying materials are made available under the + * terms of the Apache License, Version 2.0 which is available at + * https://www.apache.org/licenses/LICENSE-2.0 + * + * SPDX-License-Identifier: Apache-2.0 + * + * Contributors: + * Fraunhofer-Gesellschaft - initial API and implementation + * + */ + +package org.eclipse.edc.sample.extension.fc; + +import com.fasterxml.jackson.core.type.TypeReference; +import com.fasterxml.jackson.databind.ObjectMapper; +import org.eclipse.edc.crawler.spi.TargetNode; +import org.eclipse.edc.crawler.spi.TargetNodeDirectory; + +import java.io.IOException; +import java.util.List; + +public class CatalogNodeDirectory implements TargetNodeDirectory { + + private final ObjectMapper objectMapper; + private final String participantFileContent; + + + public CatalogNodeDirectory(ObjectMapper objectMapper, String participantFileContent) { + this.objectMapper = objectMapper; + this.participantFileContent = participantFileContent; + } + + @Override + public List getAll() { + try { + return objectMapper.readValue(participantFileContent, new TypeReference<>() {}); + } catch (IOException e) { + throw new RuntimeException(e); + } + } + + @Override + public void insert(TargetNode targetNode) { + + } +} diff --git a/federated-catalog/fc-03-static-node-directory/target-node-resolver/src/main/java/org/eclipse/edc/sample/extension/fc/CatalogNodeDirectoryExtension.java b/federated-catalog/fc-03-static-node-directory/target-node-resolver/src/main/java/org/eclipse/edc/sample/extension/fc/CatalogNodeDirectoryExtension.java new file mode 100644 index 00000000..d6eec628 --- /dev/null +++ b/federated-catalog/fc-03-static-node-directory/target-node-resolver/src/main/java/org/eclipse/edc/sample/extension/fc/CatalogNodeDirectoryExtension.java @@ -0,0 +1,48 @@ +/* + * Copyright (c) 2024 Fraunhofer-Gesellschaft + * + * This program and the accompanying materials are made available under the + * terms of the Apache License, Version 2.0 which is available at + * https://www.apache.org/licenses/LICENSE-2.0 + * + * SPDX-License-Identifier: Apache-2.0 + * + * Contributors: + * Fraunhofer-Gesellschaft - initial API and implementation + * + */ + +package org.eclipse.edc.sample.extension.fc; + +import org.eclipse.edc.crawler.spi.TargetNodeDirectory; +import org.eclipse.edc.runtime.metamodel.annotation.Inject; +import org.eclipse.edc.runtime.metamodel.annotation.Provider; +import org.eclipse.edc.spi.system.ServiceExtension; +import org.eclipse.edc.spi.types.TypeManager; + +import java.io.IOException; +import java.io.InputStream; +import java.nio.charset.StandardCharsets; + +public class CatalogNodeDirectoryExtension implements ServiceExtension { + @Inject + private TypeManager typeManager; + + @Provider + public TargetNodeDirectory federatedCacheNodeDirectory() { + String participantsFilePath = "participants.json"; + + ClassLoader classLoader = getClass().getClassLoader(); + try (InputStream participantFileInputStream = classLoader.getResourceAsStream(participantsFilePath)) { + if (participantFileInputStream == null) { + throw new RuntimeException("Participant list file does not exist: " + participantsFilePath); + } + + String participantFileContent = new String(participantFileInputStream.readAllBytes(), StandardCharsets.UTF_8); + + return new CatalogNodeDirectory(typeManager.getMapper(), participantFileContent); + } catch (IOException e) { + throw new RuntimeException("Failed to read and map participant list file: " + participantsFilePath, e); + } + } +} diff --git a/federated-catalog/fc-03-static-node-directory/target-node-resolver/src/main/resources/META-INF/services/org.eclipse.edc.spi.system.ServiceExtension b/federated-catalog/fc-03-static-node-directory/target-node-resolver/src/main/resources/META-INF/services/org.eclipse.edc.spi.system.ServiceExtension new file mode 100644 index 00000000..df3d2d3d --- /dev/null +++ b/federated-catalog/fc-03-static-node-directory/target-node-resolver/src/main/resources/META-INF/services/org.eclipse.edc.spi.system.ServiceExtension @@ -0,0 +1 @@ +org.eclipse.edc.sample.extension.fc.CatalogNodeDirectoryExtension \ No newline at end of file diff --git a/federated-catalog/fc-03-static-node-directory/target-node-resolver/src/main/resources/participants.json b/federated-catalog/fc-03-static-node-directory/target-node-resolver/src/main/resources/participants.json new file mode 100644 index 00000000..a8afb3b5 --- /dev/null +++ b/federated-catalog/fc-03-static-node-directory/target-node-resolver/src/main/resources/participants.json @@ -0,0 +1,8 @@ +[ + { + "name": "https://w3id.org/edc/v0.0.1/ns/", + "id": "provider", + "url": "http://localhost:19194/protocol", + "supportedProtocols": ["dataspace-protocol-http"] + } +] \ No newline at end of file diff --git a/settings.gradle.kts b/settings.gradle.kts index 88da6243..aec13b97 100644 --- a/settings.gradle.kts +++ b/settings.gradle.kts @@ -57,6 +57,9 @@ include(":federated-catalog:fc-00-basic:federated-catalog-base") include(":federated-catalog:fc-00-basic:fixed-node-resolver") include(":federated-catalog:fc-01-embedded:fc-connector") include(":federated-catalog:fc-02-standalone:standalone-fc") +include(":federated-catalog:fc-03-static-node-directory:target-node-resolver") +include(":federated-catalog:fc-03-static-node-directory:standalone-fc-with-node-resolver") +include(":federated-catalog:fc-03-static-node-directory:embedded-fc-with-node-resolver") // modules for code samples ------------------------------------------------------------------------ diff --git a/system-tests/build.gradle.kts b/system-tests/build.gradle.kts index 11855353..dfb6652b 100644 --- a/system-tests/build.gradle.kts +++ b/system-tests/build.gradle.kts @@ -10,6 +10,7 @@ * Contributors: * Bayerische Motoren Werke Aktiengesellschaft (BMW AG) - initial API and implementation * Fraunhofer-Gesellschaft - dependencies for Federated Catalog Tests + * Fraunhofer-Gesellschaft - set working directory to project directory * */ @@ -60,6 +61,9 @@ dependencies { testCompileOnly(project(":federated-catalog:fc-00-basic:fixed-node-resolver")) testCompileOnly(project(":federated-catalog:fc-01-embedded:fc-connector")) testCompileOnly(project(":federated-catalog:fc-02-standalone:standalone-fc")) + testCompileOnly(project(":federated-catalog:fc-03-static-node-directory:target-node-resolver")) + testCompileOnly(project(":federated-catalog:fc-03-static-node-directory:standalone-fc-with-node-resolver")) + testCompileOnly(project(":federated-catalog:fc-03-static-node-directory:embedded-fc-with-node-resolver")) } tasks.compileJava { diff --git a/system-tests/src/test/java/org/eclipse/edc/samples/common/FederatedCatalogCommon.java b/system-tests/src/test/java/org/eclipse/edc/samples/common/FederatedCatalogCommon.java index 1705f5dc..2272e9e9 100644 --- a/system-tests/src/test/java/org/eclipse/edc/samples/common/FederatedCatalogCommon.java +++ b/system-tests/src/test/java/org/eclipse/edc/samples/common/FederatedCatalogCommon.java @@ -37,10 +37,10 @@ public class FederatedCatalogCommon { private static final String V3_ASSETS_PATH = "/v3/assets"; private static final String ASSET_ID = "@id"; - private static final String STANDALONE_FC_MODULE_PATH = ":federated-catalog:fc-02-standalone:standalone-fc"; - private static final String FC_CONNECTOR_MODULE_PATH = ":federated-catalog:fc-01-embedded:fc-connector"; private static final String STANDALONE_FC = "standalone-fc"; private static final String EMBEDDED_FC = "fc-connector"; + private static final String STANDALONE_FC_CONFIG_PROPERTIES_FILE_PATH = "federated-catalog/fc-02-standalone/standalone-fc/config.properties"; + private static final String FC_CONNECTOR_CONFIG_PROPERTIES_FILE_PATH = "federated-catalog/fc-01-embedded/fc-connector/config.properties"; private static final String EDC_KEYSTORE = "edc.keystore"; private static final String EDC_KEYSTORE_PASSWORD = "edc.keystore.password"; @@ -48,13 +48,10 @@ public class FederatedCatalogCommon { private static final String CERT_PFX_FILE_PATH = "transfer/transfer-00-prerequisites/resources/certs/cert.pfx"; private static final String KEYSTORE_PASSWORD = "123456"; - private static final String STANDALONE_FC_CONFIG_PROPERTIES_FILE_PATH = "federated-catalog/fc-02-standalone/standalone-fc/config.properties"; - private static final String FC_CONNECTOR_CONFIG_PROPERTIES_FILE_PATH = "federated-catalog/fc-01-embedded/fc-connector/config.properties"; - private static final String CRAWLER_EXECUTION_DELAY = "edc.catalog.cache.execution.delay.seconds"; public static final int CRAWLER_EXECUTION_DELAY_VALUE = 5; private static final String CRAWLER_EXECUTION_PERIOD = "edc.catalog.cache.execution.period.seconds"; - public static final int CRAWLER_EXECUTION_PERIOD_VALUE = 5; + public static final int CRAWLER_EXECUTION_PERIOD_VALUE = 40; public static final int TIMEOUT = 5 * CRAWLER_EXECUTION_PERIOD_VALUE; public static final String EMBEDDED_FC_CATALOG_API_ENDPOINT = "http://localhost:29195/api/catalog/v1alpha/catalog/query"; @@ -64,12 +61,12 @@ public class FederatedCatalogCommon { public static final String CATALOG = "dcat:Catalog"; public static final String DATASET_ASSET_ID = "[0].'dcat:dataset'.@id"; - public static RuntimeExtension getFcEmbeddedConnector() { - return getRuntime(FC_CONNECTOR_MODULE_PATH, EMBEDDED_FC, FC_CONNECTOR_CONFIG_PROPERTIES_FILE_PATH); + public static RuntimeExtension getFcEmbeddedConnector(String modulePath) { + return getRuntime(modulePath, EMBEDDED_FC, FC_CONNECTOR_CONFIG_PROPERTIES_FILE_PATH); } - public static RuntimeExtension getStandaloneFc() { - return getRuntime(STANDALONE_FC_MODULE_PATH, STANDALONE_FC, STANDALONE_FC_CONFIG_PROPERTIES_FILE_PATH); + public static RuntimeExtension getStandaloneFc(String modulePath) { + return getRuntime(modulePath, STANDALONE_FC, STANDALONE_FC_CONFIG_PROPERTIES_FILE_PATH); } private static RuntimeExtension getRuntime( diff --git a/system-tests/src/test/java/org/eclipse/edc/samples/federated/catalog/FederatedCatalog01embeddedTest.java b/system-tests/src/test/java/org/eclipse/edc/samples/federated/catalog/FederatedCatalog01embeddedTest.java index b40e362d..22b48fb8 100644 --- a/system-tests/src/test/java/org/eclipse/edc/samples/federated/catalog/FederatedCatalog01embeddedTest.java +++ b/system-tests/src/test/java/org/eclipse/edc/samples/federated/catalog/FederatedCatalog01embeddedTest.java @@ -25,7 +25,6 @@ import static org.assertj.core.api.Assertions.assertThat; import static org.awaitility.Awaitility.await; import static org.eclipse.edc.samples.common.FederatedCatalogCommon.CRAWLER_EXECUTION_DELAY_VALUE; -import static org.eclipse.edc.samples.common.FederatedCatalogCommon.CRAWLER_EXECUTION_PERIOD_VALUE; import static org.eclipse.edc.samples.common.FederatedCatalogCommon.DATASET_ASSET_ID; import static org.eclipse.edc.samples.common.FederatedCatalogCommon.EMBEDDED_FC_CATALOG_API_ENDPOINT; import static org.eclipse.edc.samples.common.FederatedCatalogCommon.EMPTY_QUERY_FILE_PATH; @@ -45,7 +44,7 @@ public class FederatedCatalog01embeddedTest { static final RuntimeExtension PARTICIPANT_CONNECTOR = getProvider(); @RegisterExtension - static final RuntimeExtension FC_CONNECTOR = getFcEmbeddedConnector(); + static final RuntimeExtension FC_CONNECTOR = getFcEmbeddedConnector(":federated-catalog:fc-01-embedded:fc-connector"); @Test void shouldStartConnector() { @@ -62,7 +61,6 @@ void runSampleSteps() { await() .atMost(Duration.ofSeconds(TIMEOUT)) .pollDelay(Duration.ofSeconds(CRAWLER_EXECUTION_DELAY_VALUE)) - .pollInterval(Duration.ofSeconds(CRAWLER_EXECUTION_PERIOD_VALUE)) .ignoreExceptions() .until(() -> postAndAssertType(EMBEDDED_FC_CATALOG_API_ENDPOINT, getFileContentFromRelativePath(EMPTY_QUERY_FILE_PATH), DATASET_ASSET_ID), id -> id.equals(assetId)); diff --git a/system-tests/src/test/java/org/eclipse/edc/samples/federated/catalog/FederatedCatalog02standaloneTest.java b/system-tests/src/test/java/org/eclipse/edc/samples/federated/catalog/FederatedCatalog02standaloneTest.java index d748d959..072af756 100644 --- a/system-tests/src/test/java/org/eclipse/edc/samples/federated/catalog/FederatedCatalog02standaloneTest.java +++ b/system-tests/src/test/java/org/eclipse/edc/samples/federated/catalog/FederatedCatalog02standaloneTest.java @@ -25,7 +25,6 @@ import static org.assertj.core.api.Assertions.assertThat; import static org.awaitility.Awaitility.await; import static org.eclipse.edc.samples.common.FederatedCatalogCommon.CRAWLER_EXECUTION_DELAY_VALUE; -import static org.eclipse.edc.samples.common.FederatedCatalogCommon.CRAWLER_EXECUTION_PERIOD_VALUE; import static org.eclipse.edc.samples.common.FederatedCatalogCommon.DATASET_ASSET_ID; import static org.eclipse.edc.samples.common.FederatedCatalogCommon.EMPTY_QUERY_FILE_PATH; import static org.eclipse.edc.samples.common.FederatedCatalogCommon.STANDALONE_FC_CATALOG_API_ENDPOINT; @@ -44,7 +43,7 @@ public class FederatedCatalog02standaloneTest { static final RuntimeExtension PARTICIPANT_CONNECTOR = getProvider(); @RegisterExtension - static final RuntimeExtension STANDALONE_FC_RUNTIME = getStandaloneFc(); + static final RuntimeExtension STANDALONE_FC_RUNTIME = getStandaloneFc(":federated-catalog:fc-02-standalone:standalone-fc"); @Test void shouldStartRuntimes() { @@ -61,7 +60,6 @@ void runSampleSteps() { await() .atMost(Duration.ofSeconds(TIMEOUT)) .pollDelay(Duration.ofSeconds(CRAWLER_EXECUTION_DELAY_VALUE)) - .pollInterval(Duration.ofSeconds(CRAWLER_EXECUTION_PERIOD_VALUE)) .ignoreExceptions() .until(() -> postAndAssertType(STANDALONE_FC_CATALOG_API_ENDPOINT, getFileContentFromRelativePath(EMPTY_QUERY_FILE_PATH), DATASET_ASSET_ID), id -> id.equals(assetId)); diff --git a/system-tests/src/test/java/org/eclipse/edc/samples/federated/catalog/FederatedCatalog03staticNodeResolverTest.java b/system-tests/src/test/java/org/eclipse/edc/samples/federated/catalog/FederatedCatalog03staticNodeResolverTest.java new file mode 100644 index 00000000..c64977c1 --- /dev/null +++ b/system-tests/src/test/java/org/eclipse/edc/samples/federated/catalog/FederatedCatalog03staticNodeResolverTest.java @@ -0,0 +1,83 @@ +/* + * Copyright (c) 2024 Fraunhofer-Gesellschaft + * + * This program and the accompanying materials are made available under the + * terms of the Apache License, Version 2.0 which is available at + * https://www.apache.org/licenses/LICENSE-2.0 + * + * SPDX-License-Identifier: Apache-2.0 + * + * Contributors: + * Fraunhofer-Gesellschaft - initial API and implementation + * + */ + +package org.eclipse.edc.samples.federated.catalog; + +import org.eclipse.edc.junit.annotations.EndToEndTest; +import org.eclipse.edc.junit.extensions.RuntimeExtension; +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.extension.RegisterExtension; + +import java.time.Clock; +import java.time.Duration; + +import static org.assertj.core.api.Assertions.assertThat; +import static org.awaitility.Awaitility.await; +import static org.eclipse.edc.samples.common.FederatedCatalogCommon.CRAWLER_EXECUTION_DELAY_VALUE; +import static org.eclipse.edc.samples.common.FederatedCatalogCommon.DATASET_ASSET_ID; +import static org.eclipse.edc.samples.common.FederatedCatalogCommon.EMBEDDED_FC_CATALOG_API_ENDPOINT; +import static org.eclipse.edc.samples.common.FederatedCatalogCommon.EMPTY_QUERY_FILE_PATH; +import static org.eclipse.edc.samples.common.FederatedCatalogCommon.STANDALONE_FC_CATALOG_API_ENDPOINT; +import static org.eclipse.edc.samples.common.FederatedCatalogCommon.TIMEOUT; +import static org.eclipse.edc.samples.common.FederatedCatalogCommon.createAsset; +import static org.eclipse.edc.samples.common.FederatedCatalogCommon.getFcEmbeddedConnector; +import static org.eclipse.edc.samples.common.FederatedCatalogCommon.getStandaloneFc; +import static org.eclipse.edc.samples.common.FederatedCatalogCommon.postAndAssertType; +import static org.eclipse.edc.samples.common.FileTransferCommon.getFileContentFromRelativePath; +import static org.eclipse.edc.samples.common.NegotiationCommon.createContractDefinition; +import static org.eclipse.edc.samples.common.NegotiationCommon.createPolicy; +import static org.eclipse.edc.samples.common.PrerequisitesCommon.getProvider; + +@EndToEndTest +public class FederatedCatalog03staticNodeResolverTest { + + @RegisterExtension + static final RuntimeExtension PARTICIPANT_CONNECTOR = getProvider(); + + @RegisterExtension + static final RuntimeExtension STANDALONE_FC_RUNTIME = getStandaloneFc(":federated-catalog:fc-03-static-node-directory:standalone-fc-with-node-resolver"); + + @RegisterExtension + static final RuntimeExtension FC_CONNECTOR = getFcEmbeddedConnector(":federated-catalog:fc-03-static-node-directory:embedded-fc-with-node-resolver"); + + @Test + void shouldStartRuntimes() { + assertThat(PARTICIPANT_CONNECTOR.getService(Clock.class)).isNotNull(); + assertThat(STANDALONE_FC_RUNTIME.getService(Clock.class)).isNotNull(); + assertThat(FC_CONNECTOR.getService(Clock.class)).isNotNull(); + } + + @Test + void runSampleSteps() { + String assetId = createAsset(); + createPolicy(); + createContractDefinition(); + + // call catalog API from standalone FC + await() + .atMost(Duration.ofSeconds(TIMEOUT)) + .pollDelay(Duration.ofSeconds(CRAWLER_EXECUTION_DELAY_VALUE)) + .ignoreExceptions() + .until(() -> postAndAssertType(STANDALONE_FC_CATALOG_API_ENDPOINT, getFileContentFromRelativePath(EMPTY_QUERY_FILE_PATH), DATASET_ASSET_ID), + id -> id.equals(assetId)); + + // call catalog API from embedded FC + await() + .atMost(Duration.ofSeconds(TIMEOUT)) + .pollDelay(Duration.ofSeconds(CRAWLER_EXECUTION_DELAY_VALUE)) + .ignoreExceptions() + .until(() -> postAndAssertType(EMBEDDED_FC_CATALOG_API_ENDPOINT, getFileContentFromRelativePath(EMPTY_QUERY_FILE_PATH), DATASET_ASSET_ID), + id -> id.equals(assetId)); + } +} diff --git a/system-tests/src/test/java/org/eclipse/edc/samples/policy/Policy01BasicTest.java b/system-tests/src/test/java/org/eclipse/edc/samples/policy/Policy01BasicTest.java index 55c5c65d..9d1f2710 100644 --- a/system-tests/src/test/java/org/eclipse/edc/samples/policy/Policy01BasicTest.java +++ b/system-tests/src/test/java/org/eclipse/edc/samples/policy/Policy01BasicTest.java @@ -23,6 +23,7 @@ import org.junit.jupiter.api.Test; import org.junit.jupiter.api.extension.RegisterExtension; +import java.time.Duration; import java.util.Map; import static org.awaitility.Awaitility.await; @@ -33,11 +34,10 @@ import static org.eclipse.edc.samples.common.PolicyCommon.createContractDefinition; import static org.eclipse.edc.samples.common.PolicyCommon.createPolicy; import static org.eclipse.edc.samples.util.TransferUtil.POLL_INTERVAL; -import static org.eclipse.edc.samples.util.TransferUtil.TIMEOUT; @EndToEndTest class Policy01BasicTest { - + private static final Duration TIMEOUT = Duration.ofSeconds(90); private static final String SAMPLE_FOLDER = "policy/policy-01-policy-enforcement"; private static final String CREATE_ASSET_FILE_PATH = SAMPLE_FOLDER + "/resources/create-asset.json"; private static final String CREATE_POLICY_FILE_PATH = SAMPLE_FOLDER + "/resources/create-policy.json";