fix: incorrect behaviors in Batch Entity Upsert endpoint #1290

bobeal · 2024-12-18T17:12:37Z

noOverwrite option is not handled whereas it should be passed when in update mode
replace mode was only replacing the attributes of the entity (and thus was missing all the other needed actions: replace types and scopes, send events...)

github-actions · 2024-12-18T17:19:33Z

Test Results

69 files ±0 69 suites ±0 1m 25s ⏱️ +2s
1 110 tests +1 1 110 ✅ +1 0 💤 ±0 0 ❌ ±0
1 149 runs +1 1 149 ✅ +1 0 💤 ±0 0 ❌ ±0

Results for commit 93d8cb5. ± Comparison against base commit 1b07122.

This pull request removes 189 and adds 34 tests. Note that renamed tests count towards both.

                                    { "id":…, withTemporalValues=true, withAudit=false, expectation={
                      "@id": "https://uri…
                      "@type": "@json",
                      …
                    "@value": "/A/B"
                    "@value": "/C/D"
                    "@value": 20
                    "…
                    {
                  "@type": "https://uri.etsi.org/ngsi-ld/DateTime",
…

com.egm.stellio.search.entity.util.PatchAttributeTests ‑ [1] source={
    "attribute": {
        "type": "Property",
        "value": 12.0,
        "observedAt": "2024-04-14T12:34:56Z"
    }
}, target={
    "attribute": {
        "type": "Property",
        "value": 12.2,
        "unitCode": "GRM"
    }
}, expected={
    "attribute": {
        "type": "Property",
        "value": 12.2,
        "unitCode": "GRM",
        "observedAt": "2024-04-14T12:34:56Z"
    }
}
com.egm.stellio.search.entity.util.PatchAttributeTests ‑ [1] source={
    "attribute": {
        "type": "Property",
        "value": 12.0,
        "observedAt": "2024-04-14T12:34:56Z",
        "subAttribute": {
            "type": "Property",
            "value": "subAttribute"
        }
    }
}, target={
    "attribute": {
        "type": "Property",
        "value": 12.2,
        "unitCode": "GRM",
        "subAttribute": {
            "type": "Property",
            "value": "newSubAttributeValue"
        }
    }
}, expected={
    "attribute": {
        "type": "Property",
        "value": 12.2,
        "unitCode": "GRM",
        "observedAt": "2024-04-14T12:34:56Z",
        "subAttribute": {
            "type": "Property",
            "value": "newSubAttributeValue"
        }
    }
}
com.egm.stellio.search.entity.util.PatchAttributeTests ‑ [2] source={
    "attribute": {
        "type": "Property",
        "value": { "en": "car", "fr": "voiture" }
    }
}, target={
    "attribute": {
        "type": "Property",
        "value": { "fr": "vélo", "es": "bicicleta" }
    }
}, expected={
    "attribute": {
        "type": "Property",
        "value": { "en": "car", "fr": "vélo", "es": "bicicleta" }
    }
}
com.egm.stellio.search.entity.util.PatchAttributeTests ‑ [2] source={
    "incoming": {
        "type": "JsonProperty",
        "json": { "id": 1, "b": null, "c": 12.4 },
        "observedAt": "2022-12-24T14:01:22.066Z",
        "subAttribute": {
            "type": "Property",
            "value": "subAttribute"
        }
    }
}, target={
    "incoming": {
        "type": "JsonProperty",
        "json": { "id": 2, "b": "something" },
        "observedAt": "2023-12-24T14:01:22.066Z"
    }
}, expected={
    "incoming": {
        "type": "JsonProperty",
        "json": { "id": 2, "b": "something" },
        "observedAt": "2023-12-24T14:01:22.066Z",
        "subAttribute": {
            "type": "Property",
            "value": "subAttribute"
        }
    }
}
com.egm.stellio.search.entity.util.PatchAttributeTests ‑ [3] source={
    "attribute": {
        "type": "Property",
        "value": [ "car", "voiture" ]
    }
}, target={
    "attribute": {
        "type": "Property",
        "value": [ "vélo", "bicicleta" ]
    }
}, expected={
    "attribute": {
        "type": "Property",
        "value": [ "vélo", "bicicleta" ]
    }
}
com.egm.stellio.search.entity.util.PatchAttributeTests ‑ [3] source={
    "attribute": {
        "type": "VocabProperty",
        "vocab": "stellio"
    }
}, target={
    "attribute": {
        "type": "VocabProperty",
        "vocab": "egm"
    }
}, expected={
    "attribute": {
        "type": "VocabProperty",
        "vocab": "egm"
    }
}
com.egm.stellio.search.entity.util.PatchAttributeTests ‑ [4] source={
    "attribute": {
        "type": "Relationship",
        "object": "urn:ngsi-ld:Entity:01"
    }
}, target={
    "attribute": {
        "type": "Relationship",
        "object": "urn:ngsi-ld:Entity:02"
    }
}, expected={
    "attribute": {
        "type": "Relationship",
        "object": "urn:ngsi-ld:Entity:02"
    }
}
com.egm.stellio.search.entity.util.PatchAttributeTests ‑ [5] source={
    "attribute": {
        "type": "LanguageProperty",
        "languageMap": { "en": "train", "fr": "train" }
    }
}, target={
    "attribute": {
        "type": "LanguageProperty",
        "languageMap": { "fr": "TGV", "es": "tren" }
    }
}, expected={
    "attribute": {
        "type": "LanguageProperty",
        "languageMap": { "en": "train", "fr": "TGV", "es": "tren" }
    }
}
com.egm.stellio.search.entity.util.PatchAttributeTests ‑ [6] source={
    "incoming": {
        "type": "JsonProperty",
        "json": { "a": 1, "b": "thing" }
    }
}, target={
    "incoming": {
        "type": "JsonProperty",
        "json": { "a": 2, "c": "other thing" }
    }
}, expected={
    "incoming": {
        "type": "JsonProperty",
        "json": { "a": 2, "b": "thing", "c": "other thing" }
    }
}
com.egm.stellio.search.entity.util.PatchAttributeTests ‑ [7] source={
    "attribute": {
        "type": "VocabProperty",
        "vocab": "stellio"
    }
}, target={
    "attribute": {
        "type": "VocabProperty",
        "vocab": "egm"
    }
}, expected={
    "attribute": {
        "type": "VocabProperty",
        "vocab": "egm"
    }
}
…

♻️ This comment has been updated with latest results.

thomasBousselin · 2024-12-19T08:28:55Z

search-service/src/main/kotlin/com/egm/stellio/search/entity/web/EntityOperationHandler.kt

@@ -109,7 +109,9 @@ class EntityOperationHandler(
        )
        @RequestParam queryParams: MultiValueMap<String, String>
    ): ResponseEntity<*> = either {
-        val options = queryParams.getFirst(QP.OPTIONS.key)
+        val options = queryParams.getFirst(QP.OPTIONS.key)?.split(",")
+        val disallowOverwrite = options?.any { it == OptionsValue.NO_OVERWRITE.value } == true


I don't see a no_overwrite option for upsert in the specification. (5.6.8 and 6.15)

see the in-progress doc with misc fixes to contribute to the spec

as discussed, aligned with the figures of operations from 1.6.1. Spec needs clarification, to be discussed with ISG-CIM.

- noOverwrite option is not handled whereas it should be passed when in update mode - replace mode was only replacing the attributes of the entity

sonarqubecloud · 2024-12-19T16:01:49Z

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
81.6% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

bobeal · 2024-12-20T10:39:00Z

@thomasBousselin OK for you?

bobeal requested review from thomasBousselin and ranim-n December 18, 2024 17:12

github-actions bot added the fix Something isn't working label Dec 18, 2024

bobeal self-assigned this Dec 18, 2024

bobeal added the core Relates to core API label Dec 18, 2024

bobeal linked an issue Dec 18, 2024 that may be closed by this pull request

Batch entity upsert does not replace types and scopes of existing entities #1278

Closed

thomasBousselin reviewed Dec 19, 2024

View reviewed changes

ranim-n approved these changes Dec 19, 2024

View reviewed changes

fix: incorrect behaviors in Batch Entity Upsert endpoint

93d8cb5

- noOverwrite option is not handled whereas it should be passed when in update mode - replace mode was only replacing the attributes of the entity

bobeal force-pushed the fix/1278-batch-entity-upsert-types-scopes branch from 82a70b4 to 93d8cb5 Compare December 19, 2024 15:54

thomasBousselin approved these changes Dec 20, 2024

View reviewed changes

bobeal merged commit 336a800 into develop Dec 20, 2024
11 checks passed

bobeal deleted the fix/1278-batch-entity-upsert-types-scopes branch December 20, 2024 13:18

github-actions bot locked and limited conversation to collaborators Dec 20, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix: incorrect behaviors in Batch Entity Upsert endpoint #1290

fix: incorrect behaviors in Batch Entity Upsert endpoint #1290

bobeal commented Dec 18, 2024

github-actions bot commented Dec 18, 2024 •

edited

Loading

thomasBousselin Dec 19, 2024 •

edited

Loading

bobeal Dec 19, 2024

bobeal Dec 19, 2024

sonarqubecloud bot commented Dec 19, 2024

bobeal commented Dec 20, 2024

fix: incorrect behaviors in Batch Entity Upsert endpoint #1290

fix: incorrect behaviors in Batch Entity Upsert endpoint #1290

Conversation

bobeal commented Dec 18, 2024

github-actions bot commented Dec 18, 2024 • edited Loading

Test Results

thomasBousselin Dec 19, 2024 • edited Loading

Choose a reason for hiding this comment

bobeal Dec 19, 2024

Choose a reason for hiding this comment

bobeal Dec 19, 2024

Choose a reason for hiding this comment

sonarqubecloud bot commented Dec 19, 2024

Quality Gate passed

bobeal commented Dec 20, 2024

github-actions bot commented Dec 18, 2024 •

edited

Loading

thomasBousselin Dec 19, 2024 •

edited

Loading