feat(bpdm-gate): Adding a externalSequenceTimestamp attribute to the …

…business partner input request

Adding shouldUpdate criterion to the upsertFromEntitiy process

CHANGELOG.md

@@ -13,6 +13,7 @@ For changes to the BPDM Helm charts please consult the [changelog](charts/bpdm/C
 - BPDM Pool: Post endpoint to fetch the BPNL/S/A based on the requested identifiers.([#1052](https://github.com/eclipse-tractusx/bpdm/issues/1052))
 - BPDM Gate & Orchestrator: Enhance the error handling mechanism for the orchestrator and gate components by extending the list of available error codes.([#1003](https://github.com/eclipse-tractusx/bpdm/pull/1003#pullrequestreview-2477395867))
 - BPDM System Test: Tester module which performs automated end-to-end tests on an existing golden record process.([#1070](https://github.com/eclipse-tractusx/bpdm/pull/1070))
+- BPDM Gate: Requires functionality to handle data updates from the customer side based on timestamps.([#726](https://github.com/eclipse-tractusx/sig-release/issues/726))
 
 ### Changed
 
@@ -65,7 +66,7 @@ For changes to the BPDM Helm charts please consult the [changelog](charts/bpdm/C
 - BPDM Gate: GET endpoint to download the csv file template for business partner upload. ([#700](https://github.com/eclipse-tractusx/sig-release/issues/700))
 - Apps: Tax Jurisdiction Code to the physical address of a business partner ([#955](https://github.com/eclipse-tractusx/bpdm/issues/955))
 - BPDM Orchestrator: Tasks will now be persisted ([#722](https://github.com/eclipse-tractusx/sig-release/issues/722))
-- BPDM Orchestrator: Tasks now come with a gate record identifier. This makes it possible for cleaning services to match tasks for the same Gate record ([#711](https://github.com/eclipse-tractusx/sig-release/issues/711)) 
+- BPDM Orchestrator: Tasks now come with a gate record identifier. This makes it possible for cleaning services to match tasks for the same Gate record ([#711](https://github.com/eclipse-tractusx/sig-release/issues/711))
 
 ### Changed:
 
@@ -108,15 +109,15 @@ For changes to the BPDM Helm charts please consult the [changelog](charts/bpdm/C
 
 ### Added
 
-- BPDM Gate: Configuration to prevent the uploaded business partner input data to immediately enter the golden record process. 
-In this configuration the business partner data needs to be sent to the golden record process manually over the new state/ready API endpoint.
-Default configuration remains automatically sharing.
+- BPDM Gate: Configuration to prevent the uploaded business partner input data to immediately enter the golden record process.
+  In this configuration the business partner data needs to be sent to the golden record process manually over the new state/ready API endpoint.
+  Default configuration remains automatically sharing.
 - BPDM Gate: Limited multi-tenancy support. Business partners are now separated by owner-BPNL.
-The owner is determined from the 'bpn' claim in the token.
-This means users of a Gate can only see and edit their own business partner data.
+  The owner is determined from the 'bpn' claim in the token.
+  This means users of a Gate can only see and edit their own business partner data.
 - BPDM Pool: New API endpoints to query business partner data which belongs to Catena-X members only
 - APIs: Added a major version number to all API endpoint paths indicating the current version of the BPDM APIs.
-In the future we will use version numbers in the URL to differentiate between all currently supported major versions of the API
+  In the future we will use version numbers in the URL to differentiate between all currently supported major versions of the API
 - BPDM Gate Client: Now supports the stats endpoints of the BPDM API
 
 ### Changed
@@ -128,15 +129,15 @@ In the future we will use version numbers in the URL to differentiate between al
 - BPDM Gate: Fix not correctly updating business partner output data from golden record updates in the Pool.
 - JAVA version to 21
 - BPDM API Permissions: Overhaul of the permissions needed to access the BPDM API endpoints.
-Permissions are now more fine-granular and differentiate more clearly between read or write.
-For more details consult the Arc42, API documentation and properties files of the respective applications.
+  Permissions are now more fine-granular and differentiate more clearly between read or write.
+  For more details consult the Arc42, API documentation and properties files of the respective applications.
 - BPDM App Configuration: Now all applications are secured (authenticated and authorized) by default.
-You can still deactivate security in the BPDM apps for testing or development purposes though.
+  You can still deactivate security in the BPDM apps for testing or development purposes though.
 - BPDM Pool: Fix Pool trying to update golden records which the golden record process indicated to have no changes
 - BPDM Orchestrator: The business partner data for golden record process tasks has been completely overhauled.
-Now business partner data is clearly divided into `uncategorized`, `legal entity`, `site` and `additonal address` data.
-This model is less verbose and contains less duplicate data.
-Additionally, both Pool and Gate can write and read from it making it unnecessary for a cleaning service to provide the data in two different models.
+  Now business partner data is clearly divided into `uncategorized`, `legal entity`, `site` and `additonal address` data.
+  This model is less verbose and contains less duplicate data.
+  Additionally, both Pool and Gate can write and read from it making it unnecessary for a cleaning service to provide the data in two different models.
 
 
 
@@ -184,7 +185,7 @@ Additionally, both Pool and Gate can write and read from it making it unnecessar
 - BPDM Gate: New business partner type 'GENERIC' for changelog
 - BPDM Gate: New business partner type 'GENERIC' for sharing state
 - Workflows: Trivy now targets the latest alpha Docker image instead of the latest release version
-- Apps: Increase projectreactor.netty version to fix Trivy vulnerability 
+- Apps: Increase projectreactor.netty version to fix Trivy vulnerability
 
 
 ## [4.0.1] - 2023-08-28
@@ -222,8 +223,8 @@ Please create a back-up of your business partner data before updating.
 - BPDM: Umbrella Chart with BPDM Bridge Dummy.
 
 ### Fixed
-  - BPDM: Deprecated endpoints for retrieving business partners in legacy format.
-  - Endpoint for retrieving changelog entries has now improved filtering (breaking API change)
+- BPDM: Deprecated endpoints for retrieving business partners in legacy format.
+- Endpoint for retrieving changelog entries has now improved filtering (breaking API change)
 
 ## [3.2.2] - 2023-05-12
 
@@ -267,7 +268,7 @@ Please create a back-up of your business partner data before updating.
 
 ### Fixed
 
-- BPDM Gate: For a business partner with a child relation, this relation could be returned as parent relation erroneously. 
+- BPDM Gate: For a business partner with a child relation, this relation could be returned as parent relation erroneously.
 - BPDM Gate: When a business partner with a child relation was updated, this relation was erroneously deleted, rendering the previous child invalid.
 
 ## [3.0.3] - 2023-02-23

bpdm-common-test/src/main/kotlin/org/eclipse/tractusx/bpdm/test/testdata/gate/BusinessPartnerNonVerboseValues.kt

@@ -36,7 +36,8 @@ object BusinessPartnerNonVerboseValues {
 
     val bpInputRequestMinimal = BusinessPartnerInputRequest(
         externalId = BusinessPartnerVerboseValues.externalId2,
-        address = bpPostalAddressInputDtoMinimal
+        address = bpPostalAddressInputDtoMinimal,
+        externalSequenceTimestamp = null
     )
 
     val bpInputRequestFull = BusinessPartnerVerboseValues.bpInputRequestFull

bpdm-common-test/src/main/kotlin/org/eclipse/tractusx/bpdm/test/testdata/gate/BusinessPartnerVerboseValues.kt

@@ -72,6 +72,10 @@ object BusinessPartnerVerboseValues {
     const val businessStatusDescription1 = "Active"
     const val businessStatusDescription2 = "Insolvent"
 
+    val externalSequenceTimestamp1 = Instant.now().minusSeconds(5)
+    val externalSequenceTimestamp2 = Instant.now()
+    val externalSequenceTimestamp3 = Instant.now().plusSeconds(5)
+
     val businessStatusValidFrom1 = LocalDateTime.of(2020, 1, 1, 0, 0)
     val businessStatusValidFrom2 = LocalDateTime.of(2019, 1, 1, 0, 0)
 
@@ -331,7 +335,8 @@ object BusinessPartnerVerboseValues {
             addressType = AddressType.LegalAddress,
             physicalPostalAddress = postalAddress2,
             alternativePostalAddress = alternativeAddressFull
-        )
+        ),
+        externalSequenceTimestamp = null
 
     )
 
@@ -358,7 +363,8 @@ object BusinessPartnerVerboseValues {
             addressType = AddressType.SiteMainAddress,
             physicalPostalAddress = postalAddress2,
             alternativePostalAddress = alternativeAddressFull
-        )
+        ),
+        externalSequenceTimestamp = null
     )
 
     //New Values for Logistic Addresses Tests
@@ -376,7 +382,7 @@ object BusinessPartnerVerboseValues {
         building = "Bauteil A",
         floor = "Etage 1",
         door = "Door One",
-        street = StreetDto(name = "Mercedesstraße", houseNumber = "", direction = "direction1", houseNumberSupplement = "A"),
+        street = StreetDto(name = "Mercedesstraße", houseNumber = "", direction = "direction1", houseNumberSupplement = "A")
     )
 
     val postalAddressLogisticAddress2 = PhysicalPostalAddressDto(
@@ -393,7 +399,7 @@ object BusinessPartnerVerboseValues {
         building = "Building Two",
         floor = "Floor Two",
         door = "Door Two",
-        street = StreetDto(name = "TODO", houseNumber = "", direction = "direction1", houseNumberSupplement = "B"),
+        street = StreetDto(name = "TODO", houseNumber = "", direction = "direction1", houseNumberSupplement = "B")
     )
 
     //New Values for Logistic Address Tests
@@ -657,7 +663,8 @@ object BusinessPartnerVerboseValues {
             addressType = AddressType.LegalAndSiteMainAddress,
             physicalPostalAddress = physicalAddressChina,
             alternativePostalAddress = AlternativePostalAddressDto()
-        )
+        ),
+        externalSequenceTimestamp = null
     )
 
     val bpInputRequestCleaned = BusinessPartnerInputRequest(
@@ -685,7 +692,8 @@ object BusinessPartnerVerboseValues {
             addressType = AddressType.LegalAddress,
             physicalPostalAddress = postalAddress2,
             alternativePostalAddress = alternativeAddressFull
-        )
+        ),
+        externalSequenceTimestamp = null
     )
 
     val bpInputRequestError = BusinessPartnerInputRequest(
@@ -713,7 +721,8 @@ object BusinessPartnerVerboseValues {
             addressType = AddressType.LegalAddress,
             physicalPostalAddress = postalAddress2,
             alternativePostalAddress = alternativeAddressFull
-        )
+        ),
+        externalSequenceTimestamp = null
     )
 
     val bpOutputDtoCleaned = BusinessPartnerOutputDto(
@@ -829,6 +838,60 @@ object BusinessPartnerVerboseValues {
         updatedAt = Instant.now()
     )
 
+    val bpInputRequestWithExternalSequenceTimestamp1 = BusinessPartnerInputRequest(
+        externalId = externalId1,
+        legalEntity = LegalEntityRepresentationInputDto(
+            legalEntityBpn = "BPNL0000000000XY",
+            shortName = "short",
+            legalName = "Limited Liability Company Name",
+            legalForm = "Limited Liability Company"
+        ),
+        address = AddressRepresentationInputDto(
+            addressBpn = "BPNA0000000001XY",
+            name = "Address Name",
+            addressType = null,
+            physicalPostalAddress = physicalAddressMinimal
+        ),
+        externalSequenceTimestamp = externalSequenceTimestamp1
+
+    )
+
+    val bpInputRequestWithExternalSequenceTimestamp2 = BusinessPartnerInputRequest(
+        externalId = externalId1,
+        legalEntity = LegalEntityRepresentationInputDto(
+            legalEntityBpn = "BPNL0000000000XY",
+            shortName = "short1",
+            legalName = "Limited Liability Company Name",
+            legalForm = "Limited Liability Company"
+        ),
+        address = AddressRepresentationInputDto(
+            addressBpn = "BPNA0000000001XY",
+            name = "Address Name",
+            addressType = null,
+            physicalPostalAddress = physicalAddressMinimal
+        ),
+        externalSequenceTimestamp = externalSequenceTimestamp2
+
+    )
+
+    val bpInputRequestWithExternalSequenceTimestamp3 = BusinessPartnerInputRequest(
+        externalId = externalId1,
+        legalEntity = LegalEntityRepresentationInputDto(
+            legalEntityBpn = "BPNL0000000000XY",
+            shortName = "short2",
+            legalName = "Limited Liability Company Name",
+            legalForm = "Limited Liability Company"
+        ),
+        address = AddressRepresentationInputDto(
+            addressBpn = "BPNA0000000001XY",
+            name = "Another address Name",
+            addressType = null,
+            physicalPostalAddress = physicalAddressMinimal
+        ),
+        externalSequenceTimestamp = externalSequenceTimestamp3
+
+    )
+
     val now = Instant.now()
 
 

bpdm-gate-api/src/main/kotlin/org/eclipse/tractusx/bpdm/gate/api/model/IBaseBusinessPartnerGateDto.kt

@@ -22,6 +22,7 @@ package org.eclipse.tractusx.bpdm.gate.api.model
 import io.swagger.v3.oas.annotations.media.Schema
 import org.eclipse.tractusx.bpdm.common.dto.IBaseBusinessPartnerDto
 import org.eclipse.tractusx.bpdm.common.dto.openapidescription.CommonDescription
+import java.time.Instant
 
 interface IBaseBusinessPartnerGateDto : IBaseBusinessPartnerDto {
 
@@ -30,5 +31,8 @@ interface IBaseBusinessPartnerGateDto : IBaseBusinessPartnerDto {
 
     @get:Schema(description = "Indicates whether the sharing member claims (in the initial upload) the business partner to belong to the company data of the sharing member.")
     val isOwnCompanyData: Boolean
+
+    @get:Schema(description = "The timestamp indicates the last time point of change from the user side")
+    val externalSequenceTimestamp: Instant?
 }
 

bpdm-gate-api/src/main/kotlin/org/eclipse/tractusx/bpdm/gate/api/model/request/BusinessPartnerInputRequest.kt

@@ -27,6 +27,7 @@ import org.eclipse.tractusx.bpdm.gate.api.model.IBaseBusinessPartnerGateDto
 import org.eclipse.tractusx.bpdm.gate.api.model.response.AddressRepresentationInputDto
 import org.eclipse.tractusx.bpdm.gate.api.model.response.LegalEntityRepresentationInputDto
 import org.eclipse.tractusx.bpdm.gate.api.model.response.SiteRepresentationInputDto
+import java.time.Instant
 
 @Schema(
     description = "Generic business partner with external id",
@@ -42,6 +43,7 @@ data class BusinessPartnerInputRequest(
     override val isOwnCompanyData: Boolean = false,
     override val legalEntity: LegalEntityRepresentationInputDto = LegalEntityRepresentationInputDto(),
     override val site: SiteRepresentationInputDto = SiteRepresentationInputDto(),
-    override val address: AddressRepresentationInputDto = AddressRepresentationInputDto()
+    override val address: AddressRepresentationInputDto = AddressRepresentationInputDto(),
+    override val externalSequenceTimestamp: Instant? = null
 
 ) : IBaseBusinessPartnerGateDto

bpdm-gate-api/src/main/kotlin/org/eclipse/tractusx/bpdm/gate/api/model/response/BusinessPartnerDto.kt

@@ -41,6 +41,7 @@ data class BusinessPartnerInputDto(
     override val legalEntity: LegalEntityRepresentationInputDto = LegalEntityRepresentationInputDto(),
     override val site: SiteRepresentationInputDto = SiteRepresentationInputDto(),
     override val address: AddressRepresentationInputDto = AddressRepresentationInputDto(),
+    override val externalSequenceTimestamp:Instant? = null,
 
     @get:Schema(description = CommonDescription.createdAt)
     val createdAt: Instant,

bpdm-gate-api/src/main/kotlin/org/eclipse/tractusx/bpdm/gate/api/model/response/BusinessPartnerOutputDto.kt

@@ -40,6 +40,7 @@ data class BusinessPartnerOutputDto(
     override val legalEntity: LegalEntityRepresentationOutputDto,
     override val site: SiteRepresentationOutputDto?,
     override val address: AddressComponentOutputDto,
+    override val externalSequenceTimestamp: Instant? = null,
 
     @get:Schema(description = CommonDescription.createdAt)
     val createdAt: Instant,

bpdm-gate/src/main/kotlin/org/eclipse/tractusx/bpdm/gate/entity/generic/BusinessPartnerDb.kt

@@ -24,6 +24,7 @@ import org.eclipse.tractusx.bpdm.common.dto.BusinessPartnerRole
 import org.eclipse.tractusx.bpdm.common.model.BaseEntity
 import org.eclipse.tractusx.bpdm.common.model.StageType
 import org.eclipse.tractusx.bpdm.gate.entity.SharingStateDb
+import java.time.Instant
 import java.util.*
 
 @Entity
@@ -104,6 +105,9 @@ class BusinessPartnerDb(
     @JoinColumn(name = "address_confidence_id", unique = true)
     var addressConfidence: ConfidenceCriteriaDb?,
 
+    @Column(name = "external_sequence_timestamp")
+    var externalSequenceTimestamp: Instant? = null,
+
     ) : BaseEntity() {
 
     companion object {

bpdm-gate/src/main/kotlin/org/eclipse/tractusx/bpdm/gate/service/BusinessPartnerMappings.kt

@@ -105,6 +105,7 @@ class BusinessPartnerMappings {
             bpnS = dto.site.siteBpn,
             bpnA = dto.address.addressBpn,
             postalAddress = toPostalAddress(dto.address),
+            externalSequenceTimestamp = dto.externalSequenceTimestamp,
             legalEntityConfidence = null,
             siteConfidence = null,
             addressConfidence = null,

bpdm-gate/src/main/kotlin/org/eclipse/tractusx/bpdm/gate/service/BusinessPartnerService.kt

@@ -84,7 +84,7 @@ class BusinessPartnerService(
 
 
             upsertFromEntity(existingInput, updatedData)
-                .takeIf { it.hadChanges || sharingState.sharingStateType == SharingStateType.Error }
+                .takeIf { it.hadChanges || it.shouldUpdate || sharingState.sharingStateType == SharingStateType.Error }
                 ?.also { sharingStateService.setInitial(sharingState) }
                 ?.businessPartner
         }
@@ -128,15 +128,20 @@ class BusinessPartnerService(
         val partnerToUpsert = existingPartner ?: BusinessPartnerDb.createEmpty(upsertData.sharingState, upsertData.stage)
 
         val hasChanges = changeType == ChangelogType.CREATE || compareUtil.hasChanges(upsertData, partnerToUpsert)
+        val shouldUpdate = when {
+            upsertData.externalSequenceTimestamp == null -> true
+            existingPartner?.externalSequenceTimestamp == null -> true
+            else -> upsertData.externalSequenceTimestamp!!.isAfter(existingPartner.externalSequenceTimestamp)
+        }
 
-        if (hasChanges) {
-            changelogRepository.save(ChangelogEntryDb(sharingState.externalId, sharingState.tenantBpnl, changeType, stage))
+        if (hasChanges && shouldUpdate) {
+                changelogRepository.save(ChangelogEntryDb(sharingState.externalId, sharingState.tenantBpnl, changeType, stage))
 
-            copyUtil.copyValues(upsertData, partnerToUpsert)
-            businessPartnerRepository.save(partnerToUpsert)
+                copyUtil.copyValues(upsertData, partnerToUpsert)
+                businessPartnerRepository.save(partnerToUpsert)
         }
 
-        return UpsertResult(hasChanges, changeType, partnerToUpsert)
+        return UpsertResult(hasChanges, shouldUpdate, changeType, partnerToUpsert)
     }
 
     private fun getBusinessPartners(
@@ -156,6 +161,7 @@ class BusinessPartnerService(
 
     data class UpsertResult(
         val hadChanges: Boolean,
+        val shouldUpdate: Boolean,
         val type: ChangelogType,
         val businessPartner: BusinessPartnerDb
     )

bpdm-gate/src/main/kotlin/org/eclipse/tractusx/bpdm/gate/util/BusinessPartnerCopyUtil.kt

@@ -42,6 +42,7 @@ class BusinessPartnerCopyUtil {
             legalEntityConfidence = fromPartner.legalEntityConfidence
             siteConfidence = fromPartner.siteConfidence
             addressConfidence = fromPartner.addressConfidence
+            externalSequenceTimestamp = fromPartner.externalSequenceTimestamp
 
             nameParts.replace(fromPartner.nameParts)
             roles.replace(fromPartner.roles)

bpdm-gate/src/main/kotlin/org/eclipse/tractusx/bpdm/gate/util/Extensions.kt

@@ -21,6 +21,10 @@ package org.eclipse.tractusx.bpdm.gate.util
 
 import org.springframework.security.core.context.SecurityContextHolder
 import org.springframework.security.oauth2.server.resource.authentication.JwtAuthenticationToken
+import java.time.Instant
+import java.time.LocalDateTime
+import java.time.ZoneId
+import java.time.format.DateTimeFormatter
 
 fun <T> List<T>.containsDuplicates(): Boolean = size != distinct().size
 

bpdm-gate/src/main/kotlin/org/eclipse/tractusx/bpdm/gate/util/PartnerFileUtil.kt

@@ -90,7 +90,8 @@ object PartnerFileUtil {
                         legalName = legalName
                     ),
                     site = row.toSiteRepresentationInputDto(formatter, errors, index, row.externalId.orEmpty()),
-                    address = row.toAddressRepresentationInputDto(formatter, errors, index, row.externalId.orEmpty())
+                    address = row.toAddressRepresentationInputDto(formatter, errors, index, row.externalId.orEmpty()),
+                    externalSequenceTimestamp = null
                 )
             } catch (e: Exception) {
                 errors.add("Row - ${index + 2}, External ID - ${row.externalId.orEmpty()} has error: ${e.message}")

bpdm-gate/src/main/resources/db/migration/V6_3_0_1__add_externalSequenceTimestamp_column.sql

@@ -0,0 +1,2 @@
+ALTER TABLE business_partners
+ADD COLUMN external_sequence_timestamp TIMESTAMP WITHOUT TIME ZONE NULL