[Epic Blob-CSI] Query the OPA instance via controller to get buckets. Add this to controller.

[blairdrummond]

No description provided.

[cboin1996]

Ticket is enveloped in this issue:

• feat(blobcsi): added dev docs with OPA FDI querying aaw-kubeflow-profiles-controller#42

High level summary:

• Modify the instances variable here to contain dynamic values queried from OPA endpoints.
  • minio-gateway-opa.fdi-gateway-unclassified-system:8181
  • minio-gateway-opa.fdi-gateway-protected-b-system:8181
• Basically, I need to do get requests to those above endpoints, parse out any data that is not needed.
  • Example for unclassified through kubectl : kubectl exec minio-gateway-69569fbdc6-4n2x2 -n fdi-gateway-unclassified-system -- curl -s http://minio-gateway-opa:8181/v1/data | jq '.result | del(.httpapi)'
  • Output:

  "2021-12-aaw-unclassified-dev": {
    "readers": [
      "blair-drummond",
      "salwa-mohamedd",
      "charles-belisle"
    ]
  }
}

• The idea is I would need to analyse the json response, parsing and formatting to the instances struct within the blobcsi controller.

Roadmap to success:

• Develop local environment for threaded querying of http endpoints
• Add environment into golang controller:
  • Place thread code into go controller
  • Integrate the thread code with cobra cli
  • Add environment variables where necessary for local testing
  • Update taskfile for somewhat useful instructions/dev tools for future testing
  • Update vscode debugger in order to run locally in the dev cluster
  • Modify the thread code in order to query both unclassified minio-opa-gateway and prot-b minio-opa-gateway
  • Implement schema for the json response, parsing responses and format them to the instances struct within the blobcsi controller.
  • Investigate existing FDI architecture with FDI team, and establish connection from minio bucket to FDI.

Relevant Links:

Existing Blob storage solution: #999