Loading partially overlapping data from multiple bundles results incorrect state

[sirpi]

Short description

When OPA 0.43.0 loads two bundles (via config or discovery) whose paths are partially overlapping (e.g. store/a and store/b)
then final state contains only store/a or store/b, not both
Reproduceable on Windows or Kubernetes, with the latest version: 0.43.0 (while 0.42.2 works fine)

Steps To Reproduce

1. Create two very simple bundles, only containing manifest and data:

a.tar.gz

• .manifest:
  { "revision": "a", "roots": ["store/a"] }
• store/a/data.json:
  { "v1" : {}}

b.tar.gz

• .manifest:
  { "revision": "b", "roots": ["store/b"] }
• store/b/data.json:
  { "v1" : {} }

2. load them by config or by discovery, e.g. with the following config.yml:

services:
- name: storage-service
  ...
bundles:
  a:
    resource: "/bundles/a"
    service: storage-service
  b:
    resource: "/bundles/b"
    service: storage-service

3. validate the data via REST:
   GET localhost:8181/v1/data

the result contains exactly one of the data, depending on the loading order:

{
  "result": {
    "store": {
      "a": {
        "v1": {}
      }
    }
  }
}

when the loading order is b and then a

{"level":"info","msg":"Starting bundle loader.","name":"a","plugin":"bundle","time":"2022-08-10T16:03:01+02:00"}
{"level":"info","msg":"Starting bundle loader.","name":"b","plugin":"bundle","time":"2022-08-10T16:03:01+02:00"}
{"level":"info","msg":"Bundle loaded and activated successfully. Etag updated to \"d7afa6a214d03f32ea740f60e8bc0e03\".","name":"b","plugin":"bundle","time":"2022-08-10T16:03:03+02:00"}
{"level":"info","msg":"Bundle loaded and activated successfully. Etag updated to \"8a5f55ef77291df643eda67b872ef684\".","name":"a","plugin":"bundle","time":"2022-08-10T16:03:04+02:00"}

OR

{
  "result": {
    "store": {
      "b": {
        "v1": {}
      }
    }
  }
}

when the loading order is a and then b

{"level":"info","msg":"Starting bundle loader.","name":"a","plugin":"bundle","time":"2022-08-10T16:04:11+02:00"}
{"level":"info","msg":"Starting bundle loader.","name":"b","plugin":"bundle","time":"2022-08-10T16:04:11+02:00"}
{"level":"info","msg":"Bundle loaded and activated successfully. Etag updated to \"8a5f55ef77291df643eda67b872ef684\".","name":"a","plugin":"bundle","time":"2022-08-10T16:04:13+02:00"}
{"level":"info","msg":"Bundle loaded and activated successfully. Etag updated to \"d7afa6a214d03f32ea740f60e8bc0e03\".","name":"b","plugin":"bundle","time":"2022-08-10T16:04:13+02:00"}

Similarly, if there are multiple (more than two) bundles containing data, only the one which has been loaded last remains and the others are removed from the json tree.

Expected behavior

on 0.42.2 the above REST call works as intended and returns the following structure:

{
  "result": {
    "store": {
      "a": {
        "v1": {}
      },
      "b": {
        "v1": {}
      }
    }
  }
}

The result is also correct even on 0.43.0 if the either

• the bundles are loaded by command line parameters:
  opa run -s --skip-verify -b a.tar.gz -b b.tar.gz
• or the paths are totally independent, e.g. store1/a and store2/b

Additional context

N/A

[ashutosh-narkar]

Thanks for reporting the issue @sirpi. We're able to repro it and will have a fix soon.