Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/#358 update edc 0.7.0 #914

Merged
merged 29 commits into from
May 6, 2024
Merged
Show file tree
Hide file tree
Changes from 23 commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
9d98a84
chore(testing):[#358] Update testdata upload script to Asset v3 API
ds-jhartmann Apr 12, 2024
e077c71
chore(testing):[#358] Update test data policy structure
ds-jhartmann Apr 22, 2024
3fa5933
Merge branch 'refs/heads/main' into feature/#358-update-edc-0.7.0
ds-jhartmann Apr 25, 2024
671a61a
feat(testing): Fix testdata upload
ds-jhartmann May 2, 2024
cd45727
feat(edc-client): Update to EDC 0.7.0. Update edc packages to 0.6.0
ds-jhartmann May 2, 2024
d87d6aa
chore(PMD): Increase comment line length to 120 to match checkstyle
ds-jhartmann May 2, 2024
2e4245b
chore(edc-client): Add formatted callback payload
ds-jhartmann May 2, 2024
06d9676
Merge branch 'refs/heads/main' into feature/#358-update-edc-0.7.0
ds-jhartmann May 2, 2024
b9d6827
chore(edc-client): remove test token
ds-jhartmann May 2, 2024
59e22c8
chore(edc-client): clean up dependencies
ds-jhartmann May 2, 2024
dd80b27
feat(edc-client): add missing catalog request field
ds-jhartmann May 2, 2024
658740a
feat(irs): fix typo
ds-jhartmann May 2, 2024
e636918
fix(charts): add missing authorization-grant-type
ds-jhartmann May 2, 2024
58c7267
feat(edc-client): add custom JsonObjectFromPolicyTransformer with ass…
ds-jhartmann May 3, 2024
a157cc8
feat(edc-client): fix contract negotiation request object building
ds-jhartmann May 3, 2024
5591749
feat(edc-client): fix contract negotiation request object type
ds-jhartmann May 3, 2024
85dbdc0
feat(edc-client): fix contract negotiation request object type
ds-jhartmann May 3, 2024
b25114c
feat(edc-client): fix contract negotiation and transfer process objects
ds-jhartmann May 3, 2024
14deeb2
feat(edc-client): fix contract negotiation and transfer process objects
ds-jhartmann May 3, 2024
1285066
feat(edc-client): fix transfer process request
ds-jhartmann May 3, 2024
56be7a6
feat(edc-client): fix transfer process request
ds-jhartmann May 3, 2024
a45960b
feat(edc-client): fix code smells
ds-jhartmann May 3, 2024
2c00761
feat(edc-client): re-enable test
ds-jhartmann May 3, 2024
ca21662
feat(edc-client): refactoring after review
ds-jhartmann May 6, 2024
9720ff2
feat(edc-client): add null checks
ds-jhartmann May 6, 2024
94e8960
feat(edc-client): add link to TODO issue
ds-jhartmann May 6, 2024
a98cbe1
feat(edc-client): refactoring after review
ds-jhartmann May 6, 2024
787324e
feat(edc-client): refactoring after review
ds-jhartmann May 6, 2024
b1d3920
feat(edc-client): refactoring after review
ds-jhartmann May 6, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .config/pmd-rules.xml
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@
<rule ref="category/java/documentation.xml/CommentSize">
<properties>
<property name="maxLines" value="23"/>
<property name="maxLineLength" value="100"/>
<property name="maxLineLength" value="120"/>
</properties>
</rule>
<rule ref="category/java/documentation.xml/CommentRequired">
Expand Down
9 changes: 2 additions & 7 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,18 +8,13 @@ _**For better traceability add the corresponding GitHub issue number in each cha

## [Unreleased]

### Changed

- BPN lookup feature was removed #568

## [5.1.0] - 2024-04-30


### Changed

- Removed obsolete entries from acceptedPolicies configuration. #530
- Support of building relationships based on SingleLevelUsageAsBuilt v3.0.0 #558
- Support of building relationships based on SingleLevelBomAsPlanned v3.0.0 #558
- BPN lookup feature was removed #568
- Update IRS EDC client to use EDC 0.7.0 #358

### Fixed

Expand Down
218 changes: 81 additions & 137 deletions DEPENDENCIES

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion charts/item-relationship-service/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Fixed

- Fix for incomplete renaming of Helm chart (see #489)

- Fixed missing Spring attribute
ds-jhartmann marked this conversation as resolved.
Show resolved Hide resolved


## [7.0.1] - 2024-04-17
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -57,9 +57,11 @@ data:
client:
registration:
semantics:
authorization-grant-type: client_credentials
client-id: "${SEMANTICS_OAUTH2_CLIENT_ID}" # taken from secret ENV
client-secret: "${SEMANTICS_OAUTH2_CLIENT_SECRET}" # taken from secret ENV
discovery:
authorization-grant-type: client_credentials
client-id: ${DISCOVERY_OAUTH2_CLIENT_ID} # taken from secret ENV
client-secret: ${DISCOVERY_OAUTH2_CLIENT_SECRET} # taken from secret ENV
provider:
Expand Down
2 changes: 1 addition & 1 deletion irs-api/src/main/resources/application.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ spring:
oauth2:
client:
registration:
semantics :
semantics:
authorization-grant-type: client_credentials
client-id: ${SEMANTICS_OAUTH2_CLIENT_ID} # Semantic Hub OAuth2 client ID used to authenticate with the IAM
client-secret: ${SEMANTICS_OAUTH2_CLIENT_SECRET} # Semantic Hub OAuth2 client secret used to authenticate with the IAM
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,6 @@
import org.eclipse.tractusx.irs.component.Jobs;
import org.eclipse.tractusx.irs.component.RegisterJob;
import org.eclipse.tractusx.irs.component.enums.JobState;
import org.eclipse.tractusx.irs.data.StringMapper;
import org.eclipse.tractusx.irs.edc.client.EndpointDataReferenceStorage;
import org.eclipse.tractusx.irs.semanticshub.AspectModels;
import org.eclipse.tractusx.irs.semanticshub.SemanticHubWireMockSupport;
Expand Down Expand Up @@ -111,12 +110,6 @@ static void startContainer() {
WiremockSupport.successfulSemanticModelRequest();
}

@AfterEach
void tearDown() {
cacheManager.getCacheNames()
.forEach(cacheName -> Objects.requireNonNull(cacheManager.getCache(cacheName)).clear());
dsmf marked this conversation as resolved.
Show resolved Hide resolved
}

@AfterAll
static void stopContainer() {
minioContainer.stop();
Expand All @@ -141,6 +134,12 @@ static void configureProperties(DynamicPropertyRegistry registry) {
registry.add("resilience4j.retry.configs.default.waitDuration", () -> "1s");
}

@AfterEach
void tearDown() {
cacheManager.getCacheNames()
.forEach(cacheName -> Objects.requireNonNull(cacheManager.getCache(cacheName)).clear());
}

@Test
void shouldStartApplicationAndCollectSemanticModels() throws SchemaNotFoundException {
// Arrange
Expand Down Expand Up @@ -264,7 +263,6 @@ void shouldStartRecursiveProcesses() {
assertThat(jobHandle.getId()).isNotNull();
waitForCompletion(jobHandle);
final Jobs jobForJobId = irsService.getJobForJobId(jobHandle.getId(), false);
System.out.println(StringMapper.mapToString(jobForJobId));

assertThat(jobForJobId.getJob().getState()).isEqualTo(JobState.COMPLETED);
assertThat(jobForJobId.getShells()).hasSize(3);
Expand All @@ -280,8 +278,7 @@ private void successfulRegistryAndDataRequest(final String globalAssetId, final
final String batchFileName, final String sbomFileName) {

final String edcAssetId = WiremockSupport.randomUUIDwithPrefix();
final String batch = WiremockSupport.submodelRequest(edcAssetId, "Batch",
batchAspectName, batchFileName);
final String batch = WiremockSupport.submodelRequest(edcAssetId, "Batch", batchAspectName, batchFileName);
ds-jhartmann marked this conversation as resolved.
Show resolved Hide resolved

final String singleLevelBomAsBuilt = WiremockSupport.submodelRequest(edcAssetId, "SingleLevelBomAsBuilt",
singleLevelBomAsBuiltAspectName, sbomFileName);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,9 @@ public static EndpointDataReference createEndpointDataReference(final String con
.getBytes(StandardCharsets.UTF_8));
final String jwtToken = "eyJhbGciOiJSUzI1NiJ9." + b64EncodedAuthCode + ".test";
return EndpointDataReference.Builder.newInstance()
.authKey("testkey")
.contractId(contractAgreementId)
.authKey("Authorization")
.id("test")
.authCode(jwtToken)
.properties(
Map.of(JsonLdConfiguration.NAMESPACE_EDC_CID, contractAgreementId))
Expand Down
78 changes: 40 additions & 38 deletions irs-edc-client/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -96,78 +96,80 @@
<artifactId>swagger-annotations</artifactId>
<version>${swagger-annotations.version}</version>
</dependency>

<!-- EDC Client -->
<dependency>
<groupId>org.eclipse.edc</groupId>
<artifactId>connector-core</artifactId>
<artifactId>management-api</artifactId>
<version>${edc.version}</version>
<exclusions>
<exclusion>
<artifactId>runtime-metamodel</artifactId>
<groupId>org.eclipse.edc</groupId>
</exclusion>
<exclusion>
<groupId>org.bouncycastle</groupId>
<artifactId>bcprov-jdk18on</artifactId>
</exclusion>
<exclusion>
<groupId>org.bouncycastle</groupId>
<artifactId>bcpkix-jdk18on</artifactId>
<artifactId>swagger-jaxrs2-jakarta</artifactId>
<groupId>io.swagger.core.v3</groupId>
</exclusion>
</exclusions>
</dependency>
<!-- Below overrides the one provided with above EDC Client -->
<dependency>
<groupId>com.squareup.okio</groupId>
<artifactId>okio-jvm</artifactId>
<version>${okio-jvm.version}</version>
</dependency>
<dependency>
<groupId>org.bouncycastle</groupId>
<artifactId>bcpkix-jdk18on</artifactId>
<version>${bc-jdk18on.version}</version>
</dependency>
<dependency>
<groupId>org.bouncycastle</groupId>
<artifactId>bcprov-jdk18on</artifactId>
<version>${bc-jdk18on.version}</version>
<groupId>org.eclipse.edc</groupId>
<artifactId>policy-engine-lib</artifactId>
<version>${edc.version}</version>
</dependency>
<dependency>
<groupId>org.eclipse.edc</groupId>
<artifactId>dsp</artifactId>
<artifactId>connector-core</artifactId>
<version>${edc.version}</version>
<exclusions>
<exclusion>
<artifactId>runtime-metamodel</artifactId>
<artifactId>bcpkix-jdk18on</artifactId>
<groupId>org.bouncycastle</groupId>
</exclusion>
<exclusion>
<artifactId>policy-engine-lib</artifactId>
<groupId>org.eclipse.edc</groupId>
</exclusion>
<exclusion>
<artifactId>websocket-jakarta-server</artifactId>
<groupId>org.eclipse.jetty.websocket</groupId>
<artifactId>policy-engine-spi</artifactId>
<groupId>org.eclipse.edc</groupId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.eclipse.jetty.websocket</groupId>
<artifactId>websocket-jakarta-server</artifactId>
<version>11.0.17</version>
</dependency>
<dependency>
<groupId>org.eclipse.edc</groupId>
<artifactId>json-ld</artifactId>
<artifactId>dsp</artifactId>
<version>${edc.version}</version>
<exclusions>
<exclusion>
<artifactId>runtime-metamodel</artifactId>
<groupId>org.eclipse.edc</groupId>
<artifactId>swagger-jaxrs2-jakarta</artifactId>
<groupId>io.swagger.core.v3</groupId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<artifactId>runtime-metamodel</artifactId>
<groupId>org.eclipse.edc</groupId>
<artifactId>json-ld-lib</artifactId>
<version>${edc.version}</version>
</dependency>
<dependency>
<groupId>org.eclipse.tractusx.edc</groupId>
<artifactId>edr-api</artifactId>
<version>${edc.version}</version>
<exclusions>
<exclusion>
<artifactId>management-api</artifactId>
<groupId>org.eclipse.edc</groupId>
</exclusion>
<exclusion>
<artifactId>validator-core</artifactId>
<groupId>org.eclipse.edc</groupId>
</exclusion>
<exclusion>
<artifactId>swagger-jaxrs2-jakarta</artifactId>
<groupId>io.swagger.core.v3</groupId>
</exclusion>
</exclusions>
</dependency>
<!-- EDC Client -->

<dependency>
<groupId>org.eclipse.tractusx.irs</groupId>
<artifactId>irs-common</artifactId>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -23,8 +23,10 @@
********************************************************************************/
package org.eclipse.tractusx.irs.edc.client;

import java.util.List;
import java.util.Map;
import java.util.Objects;
import java.util.Set;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.TimeUnit;
Expand All @@ -34,12 +36,13 @@
import lombok.extern.slf4j.Slf4j;
import org.apache.commons.lang3.StringUtils;
import org.eclipse.edc.spi.types.domain.DataAddress;
import org.eclipse.edc.spi.types.domain.callback.CallbackAddress;
import org.eclipse.tractusx.irs.edc.client.cache.endpointdatareference.EndpointDataReferenceStatus;
import org.eclipse.tractusx.irs.edc.client.exceptions.ContractNegotiationException;
import org.eclipse.tractusx.irs.edc.client.exceptions.TransferProcessException;
import org.eclipse.tractusx.irs.edc.client.exceptions.UsagePolicyException;
import org.eclipse.tractusx.irs.edc.client.model.CatalogItem;
import org.eclipse.tractusx.irs.edc.client.model.ContractOfferDescription;
import org.eclipse.tractusx.irs.edc.client.model.ContractOffer;
import org.eclipse.tractusx.irs.edc.client.model.EDRAuthCode;
import org.eclipse.tractusx.irs.edc.client.model.NegotiationRequest;
import org.eclipse.tractusx.irs.edc.client.model.NegotiationResponse;
Expand All @@ -60,6 +63,8 @@
public class ContractNegotiationService {

public static final String EDC_PROTOCOL = "dataspace-protocol-http";
public static final String EVENT_TRANSFER_PROCESS_STARTED = "transfer.process.started";
public static final String HTTP_DATA_PULL = "HttpData-PULL";
private final EdcControlPlaneClient edcControlPlaneClient;
private final PolicyCheckerService policyCheckerService;
private final EdcConfiguration config;
Expand Down Expand Up @@ -94,7 +99,7 @@ public NegotiationResponse negotiate(final String providerConnectorUrl, final Ca
if (authCode == null) {
throw new IllegalStateException("Missing information about AuthKey.");
}
log.error("AuthCode to be parsed: " + authCode);
log.error("AuthCode to be parsed: {}", authCode);
contractAgreementId = EDRAuthCode.fromAuthCodeToken(authCode).getCid();
log.info(
"Cached endpoint data reference has expired token. Refreshing token without new contract negotiation for contractAgreementId: {}",
Expand Down Expand Up @@ -148,30 +153,34 @@ private TransferProcessRequest createTransferProcessRequest(final String provide
.managedResources(
TransferProcessRequest.DEFAULT_MANAGED_RESOURCES)
.connectorId(catalogItem.getConnectorId())
.connectorAddress(providerConnectorUrl)
.counterPartyAddress(providerConnectorUrl)
.transferType(HTTP_DATA_PULL)
.contractId(agreementId)
.assetId(catalogItem.getAssetPropId())
.dataDestination(destination);
if (StringUtils.isNotBlank(config.getCallbackUrl())) {
log.info("Setting EDR callback to {}", config.getCallbackUrl());
transferProcessRequestBuilder.privateProperties(Map.of("receiverHttpEndpoint", config.getCallbackUrl()));
final CallbackAddress callbackAddress = CallbackAddress.Builder.newInstance()
.uri(config.getCallbackUrl())
.events(Set.of(
EVENT_TRANSFER_PROCESS_STARTED))
.build();
transferProcessRequestBuilder.callbackAddresses(List.of(callbackAddress));
}
return transferProcessRequestBuilder.build();
}

private NegotiationRequest createNegotiationRequestFromCatalogItem(final String providerConnectorUrl,
final CatalogItem catalogItem) {
final var contractOfferDescription = ContractOfferDescription.builder()
.offerId(catalogItem.getOfferId())
.assetId(catalogItem.getPolicy().getTarget())
.policy(catalogItem.getPolicy())
.build();

return NegotiationRequest.builder()
.connectorId(catalogItem.getConnectorId())
.connectorAddress(providerConnectorUrl)
.counterPartyAddress(providerConnectorUrl)
.counterPartyId(catalogItem.getConnectorId())
.protocol(EDC_PROTOCOL)
.offer(contractOfferDescription)
.contractOffer(
ContractOffer.fromPolicy(catalogItem.getPolicy(), catalogItem.getOfferId(),
catalogItem.getAssetPropId(), catalogItem.getConnectorId()))
.build();
}

Expand Down
Loading