Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(OSSM): introduce disableServiceMesh flag #217

Closed

Conversation

cam-garrison
Copy link
Contributor

Description

This flag is used in ODHDashboardConfig to configure notebooks/projects to be handled by the Service Mesh. Without this change the operator will remove disableServiceMesh from the ODHDashboardConfig resource. With this flag we can extend odh-dashboard to handle service mesh enabled deployments.

How Has This Been Tested?

Manually tested by using operator sdk make bundle and deploying on a CRC cluster. Created a KfDef that includes the Service Mesh changes:

kind: KfDef
metadata:
 name: odh-minimal
spec:
 applications:
 - kustomizeConfig:
     repoRef:
       name: manifests
       path: odh-common
   name: odh-common
 - kustomizeConfig:
     repoRef:
       name: manifests
       path: odh-dashboard
   name: odh-dashboard
 - kustomizeConfig:
     repoRef:
       name: manifests
       path: odh-notebook-controller
   name: odh-notebook-controller
 - kustomizeConfig:
     repoRef:
       name: manifests
       path: notebook-images
   name: notebook-images
 repos:
 - name: manifests
   uri: https://github.com/cam-garrison/odh-manifests/tarball/authorino-with-dashboard
 version: authorino

From here follow instructions here to enable service mesh. Tested by adding the disableServiceMesh flag to an existing ODHDashboardConfig object and was able to use the flag succesfully in our fork of odh-dashboard

Additional findings:

It seems that the content of the generated CRDs in the bundle in this repo is out of sync of the actual definitions stored under config. For example the ODHDashboardConfig has few fields missing already

+                  disableModelServing:
+                    type: boolean
+                  disableProjects:
+                    type: boolean

Merge criteria:

  • The commits are squashed in a cohesive manner and have meaningful messages.
  • Testing instructions have been added in the PR body (for PRs involving changes that are not immediately obvious).
  • The developer has manually tested the changes and verified that the changes work

This flag is used in ODHDashboardConfig to configure notebooks/projects to be handled by the Service Mesh
@openshift-ci
Copy link

openshift-ci bot commented Apr 4, 2023

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign lavlas for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci
Copy link

openshift-ci bot commented Apr 4, 2023

Hi @cam-garrison. Thanks for your PR.

I'm waiting for a opendatahub-io member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

cam-garrison added a commit to maistra/odh-dashboard that referenced this pull request Apr 4, 2023
## Changes in this PR

This change takes care of populating the following annotations to resources created through ODH Dashboard

- `opendatahub.io/service-mesh` can be set to `true` or `false` and will be used to alternate ODH components and make them part of the Service Mesh. This is applied on both `Project` and `Notebook` resources
- `opendatahub.io/hub-url` is added to the `Notebook` resources at this point and is inteded for use with Authorino's `AuthConfig` host 
- The dashboard will now link to notebooks through the mesh if feature flag `disableServiceMesh=false`

In addition, the code hides service-mesh-specific changes to annotations behind `disableServiceMesh` flag which is added in opendatahub-io/opendatahub-operator#217

## How to test

We have tested this manually. If you can offer us some hints on how to test feature flags and resource creation through Jest tests we are happy to extend this PR with it.

Until this [PR](opendatahub-io/opendatahub-operator#217) is merged and a new bundle is released you can use my build of the ODH Operator that includes the `disableServiceMesh` flag as part of the `ODHDashboardConfig`.
Do so by running `operator-sdk run bundle quay.io/cgarriso/opendatahub-operator-bundle:dev-0.0.1 --namespace $OPERATOR_NAMESPACE` (req operator-sdk v1.24.1 )

Either build and push this dashboard image or use `quay.io/maistra-dev/odh-dashboard:enable_ossm_routing` image.

Co-authored-by: bartoszmajsak <[email protected]>
bartoszmajsak added a commit to maistra/odh-dashboard that referenced this pull request Apr 4, 2023
This change takes care of populating the following annotations to resources created through ODH Dashboard

- `opendatahub.io/service-mesh` can be set to `true` or `false` and will be used to alternate ODH components and make them part of the Service Mesh. This is applied on both `Project` and `Notebook` resources
- `opendatahub.io/hub-url` is added to the `Notebook` resources at this point and is inteded for use with Authorino's `AuthConfig` host
- The dashboard will now link to notebooks through the mesh if feature flag `disableServiceMesh=false`

In addition, the code hides service-mesh-specific changes to annotations behind `disableServiceMesh` flag which is added in opendatahub-io/opendatahub-operator#217

We have tested this manually. If you can offer us some hints on how to test feature flags and resource creation through Jest tests we are happy to extend this PR with it.

Until this [PR](opendatahub-io/opendatahub-operator#217) is merged and a new bundle is released you can use my build of the ODH Operator that includes the `disableServiceMesh` flag as part of the `ODHDashboardConfig`.
Do so by running `operator-sdk run bundle quay.io/cgarriso/opendatahub-operator-bundle:dev-0.0.1 --namespace $OPERATOR_NAMESPACE` (req operator-sdk v1.24.1 )

Either build and push this dashboard image or use `quay.io/maistra-dev/odh-dashboard:ossm_annotations` image.

Co-authored-by: bartoszmajsak <[email protected]>
bartoszmajsak added a commit to maistra/odh-dashboard that referenced this pull request Apr 12, 2023
This change takes care of populating the following annotations to resources created through ODH Dashboard

- `opendatahub.io/service-mesh` can be set to `true` or `false` and will be used to alternate ODH components and make them part of the Service Mesh. This is applied on both `Project` and `Notebook` resources
- `opendatahub.io/hub-url` is added to the `Notebook` resources at this point and is inteded for use with Authorino's `AuthConfig` host
- The dashboard will now link to notebooks through the mesh if feature flag `disableServiceMesh=false`

In addition, the code hides service-mesh-specific changes to annotations behind `disableServiceMesh` flag which is added in opendatahub-io/opendatahub-operator#217

We have tested this manually. If you can offer us some hints on how to test feature flags and resource creation through Jest tests we are happy to extend this PR with it.

Until this [PR](opendatahub-io/opendatahub-operator#217) is merged and a new bundle is released you can use my build of the ODH Operator that includes the `disableServiceMesh` flag as part of the `ODHDashboardConfig`.
Do so by running `operator-sdk run bundle quay.io/cgarriso/opendatahub-operator-bundle:dev-0.0.1 --namespace $OPERATOR_NAMESPACE` (req operator-sdk v1.24.1 )

Either build and push this dashboard image or use `quay.io/maistra-dev/odh-dashboard:ossm_annotations` image.

Co-authored-by: bartoszmajsak <[email protected]>
bartoszmajsak added a commit to bartoszmajsak/odh-dashboard that referenced this pull request Apr 18, 2023
This change takes care of populating the following annotations to resources created through ODH Dashboard

- `opendatahub.io/service-mesh` can be set to `true` or `false` and will be used to alternate ODH components and make them part of the Service Mesh. This is applied on both `Project` and `Notebook` resources
- `opendatahub.io/hub-url` is added to the `Notebook` resources at this point and is inteded for use with Authorino's `AuthConfig` host
- The dashboard will now link to notebooks through the mesh if feature flag `disableServiceMesh=false`

In addition, the code hides service-mesh-specific changes to annotations behind `disableServiceMesh` flag which is added in opendatahub-io/opendatahub-operator#217

We have tested this manually. If you can offer us some hints on how to test feature flags and resource creation through Jest tests we are happy to extend this PR with it.

Until this [PR](opendatahub-io/opendatahub-operator#217) is merged and a new bundle is released you can use my build of the ODH Operator that includes the `disableServiceMesh` flag as part of the `ODHDashboardConfig`.
Do so by running `operator-sdk run bundle quay.io/cgarriso/opendatahub-operator-bundle:dev-0.0.1 --namespace $OPERATOR_NAMESPACE` (req operator-sdk v1.24.1 )

Either build and push this dashboard image or use `quay.io/maistra-dev/odh-dashboard:ossm_annotations` image.

Co-authored-by: bartoszmajsak <[email protected]>
@VaishnaviHire
Copy link
Member

@cam-garrison This change needs to be included in the odh-dashboard repo. We have a workflow setup to automatically sync changes from odh-dashboard once they are merged.

@VaishnaviHire
Copy link
Member

/ok-to-test

@openshift-ci openshift-ci bot added ok-to-test Ok to Test and removed needs-ok-to-test labels May 15, 2023
@openshift-ci
Copy link

openshift-ci bot commented May 15, 2023

@cam-garrison: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/opendatahub-operator-e2e 262cb99 link true /test opendatahub-operator-e2e
ci/prow/opendatahub-manifests-e2e 262cb99 link false /test opendatahub-manifests-e2e

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@cam-garrison
Copy link
Contributor Author

Closing as this will be handled by the sync job from odh-dashboard and pull from main or service-mesh-integration branch depending on where our dashboard PR is merged to initially.

bartoszmajsak added a commit to maistra/odh-dashboard that referenced this pull request Jun 28, 2023
This change takes care of populating the following annotations to resources created through ODH Dashboard

- `opendatahub.io/service-mesh` can be set to `true` or `false` and will be used to alternate ODH components and make them part of the Service Mesh. This is applied on both `Project` and `Notebook` resources
- `opendatahub.io/hub-url` is added to the `Notebook` resources at this point and is inteded for use with Authorino's `AuthConfig` host
- The dashboard will now link to notebooks through the mesh if feature flag `disableServiceMesh=false`

In addition, the code hides service-mesh-specific changes to annotations behind `disableServiceMesh` flag which is added in opendatahub-io/opendatahub-operator#217

We have tested this manually. If you can offer us some hints on how to test feature flags and resource creation through Jest tests we are happy to extend this PR with it.

Until this [PR](opendatahub-io/opendatahub-operator#217) is merged and a new bundle is released you can use my build of the ODH Operator that includes the `disableServiceMesh` flag as part of the `ODHDashboardConfig`.
Do so by running `operator-sdk run bundle quay.io/cgarriso/opendatahub-operator-bundle:dev-0.0.1 --namespace $OPERATOR_NAMESPACE` (req operator-sdk v1.24.1 )

Either build and push this dashboard image or use `quay.io/maistra-dev/odh-dashboard:ossm_annotations` image.

Co-authored-by: bartoszmajsak <[email protected]>
VaishnaviHire added a commit to VaishnaviHire/opendatahub-operator that referenced this pull request Mar 19, 2024
* Update bundle

* feat(authz): Authorino for Service Mesh (opendatahub-io#784)

* feat(authz): Authorino for Service Mesh

This first iteration is to cover authentication needs for KServe

* Add templates to install Authorino
* Add templates to configure Service Mesh to use Authorino to delegate Authorization
* Add KServe-specific templates add ability to secure KServe Inference Services
* Add relevant fields to DSCInitialization resource
* Code for proper cleanup, in case of uninstalling

Most (if not all) of this code comes from pull request opendatahub-io#605. Attribution to original authors: @bartoszmajsak, @aslakknutsen, @cam-garrison, et. al.

Related opendatahub-io/kserve#128

Signed-off-by: Edgar Hernández <[email protected]>

* Fix linter issues

Signed-off-by: Edgar Hernández <[email protected]>

* Resolve feedback: Bartosz

Signed-off-by: Edgar Hernández <[email protected]>

* fix: Remove port from the authorization policy

Also, add `/metrics` to the ignored paths for auth.

Signed-off-by: Edgar Hernández <[email protected]>

* Fix feedback: Bartosz

Signed-off-by: Edgar Hernández <[email protected]>

* More feedback: Bartosz

Co-authored-by: Bartosz Majsak <[email protected]>

* Fix feedback: Reto - Adjust AuthorizationPolicy

Signed-off-by: Edgar Hernández <[email protected]>

* Fix more feedback: Bartosz

- Remove Authorino namespace field from DSCI.
- Move around some code in kserve.go to servicemesh_setup.go

Signed-off-by: Edgar Hernández <[email protected]>

* chore: adds sec. prefix to authorino label selector

* fix: adds base dir to manifest sources

* chore: uses security instead of sec as a prefix in authorino label

* fix: /healthz is called by _something_, skipp

* fix: adopt ODH-ADR-0006 for clean up label

* fix: uses correct CRD name for authconfigs

Co-authored-by: Cameron Garrison <[email protected]>

* Remove left-over file

Signed-off-by: Edgar Hernández <[email protected]>

* Feedback: remove auth-refs ConfigMap

Signed-off-by: Edgar Hernández <[email protected]>

* Add missing role.yaml changes

Signed-off-by: Edgar Hernández <[email protected]>

* Go back to installing Authorino on its own namespace

Signed-off-by: Edgar Hernández <[email protected]>

* Feedback: Add clean-up for KServe/OSSM-auth

Signed-off-by: Edgar Hernández <[email protected]>

* Feedback: Simplify namings

Signed-off-by: Edgar Hernández <[email protected]>

* fix: add auth-refs cm

* Feedback: adjust labels and a log message

Signed-off-by: Edgar Hernández <[email protected]>

* Bugfix: Extension provider terminating with error when SMCP is gone

Signed-off-by: Edgar Hernández <[email protected]>

* Fix: add missing RBAC for ConfigMaps func

Signed-off-by: Edgar Hernández <[email protected]>

* Fix: Run `make bundle` and commit resulting changes

Signed-off-by: Edgar Hernández <[email protected]>

* Feedback: Wen - Better feature namings

Signed-off-by: Edgar Hernández <[email protected]>

* Feedback: Bartosz

* Use feature logger
* Don't trim -applications suffix on ResolveAuthNamespace

Signed-off-by: Edgar Hernández <[email protected]>

* Feedback: Wen - revert image placeholder was replaced

Signed-off-by: Edgar Hernández <[email protected]>

---------

Signed-off-by: Edgar Hernández <[email protected]>
Co-authored-by: Bartosz Majsak <[email protected]>
Co-authored-by: Aslak Knutsen <[email protected]>
Co-authored-by: Cameron Garrison <[email protected]>
(cherry picked from commit e32a7c2)

* fix(authz): Fix broken external auth configuration

There are two misconfigurations being fixed:
* In the SMCP, the service hostname of Authorino was coded with `-authorization` suffix, but the right suffix is `-authorino-authorization`.
* In the `kserve-predictor` AuthorizationPolicy, the hardcoded `opendatahub-odh-auth-provider` provider name was used, but it should have been the template `{{ .AppNamespace }}-auth-provider`.

In `pkg/feature/feature.go` the patch manifests (i.e. the ones containing `.patch` in the filename) are always applied. Thus, the first bullet is solved by fixing the patch file that adds the `extensionProvider` to the SMCP.

For the second bullet, the faulty AuthorizationPolicy is created with a regular manifest template which is only applied if the resource does not exist. Thus, a patch manifest is added to properly fix the faulty policy (including operator upgrades).

Signed-off-by: Edgar Hernández <[email protected]>
(cherry picked from commit e4252a0)

* fix: Rework operator precondition checks (opendatahub-io#899)

* init commit

* tmp: switch to subsciption

* tmp

* fix up testing

* linter on import

* minor self nits

* add bracket, make

* use found,err for checking subscription

Co-authored-by: Bartosz Majsak <[email protected]>

* fix import + test error expected outputs

* directly return errs rather than log and ret

Co-authored-by: Bartosz Majsak <[email protected]>

* remove unused log var from condiitons

* move const fixtures to separate package

* move creating op subscription to function

* rename noop features in testing

* remove redundant comments

Co-authored-by: Bartosz Majsak <[email protected]>

* move CreateSubscription to fixtures

---------

Co-authored-by: Bartosz Majsak <[email protected]>
(cherry picked from commit f44528e)

* chore: follow up review comments from previous PR (opendatahub-io#858)

* update: follow up comments

- cleanup commented out code
- rename function
- cleanup unnecessary sleep

Signed-off-by: Wen Zhou <[email protected]>

* update: add check on return err + remove apierrs.IsNotFound check

Signed-off-by: Wen Zhou <[email protected]>

* Update pkg/deploy/deploy.go

Co-authored-by: Bartosz Majsak <[email protected]>

* update(review): create new function DeleteSubscription

Signed-off-by: Wen Zhou <[email protected]>

* update: return for get and delete subscription

- get: return 'sub, nil' or 'nil, err' here error can be real one or
notfound

Signed-off-by: Wen Zhou <[email protected]>

* Update pkg/deploy/deploy.go

Co-authored-by: Bartosz Majsak <[email protected]>

* fix(linter)

Signed-off-by: Wen Zhou <[email protected]>

---------

Signed-off-by: Wen Zhou <[email protected]>
Co-authored-by: Bartosz Majsak <[email protected]>
(cherry picked from commit a81a3da)

* fix(authz): ensures extauthz provider is removed from control plane during cleanup (opendatahub-io#905)

### Renames migration folder
The reason for this is to have a simple naming convention instead of suggesting storing migration patches in dedicated folders named after tickets.

Additionally, the feature explicitly orders files instead of assuming that the underlying fsys implementation fulfills such a contract.

### Ports opendatahub-io#605 test for extension provider

This test ensures the addition of an extension provider for external authorization and that it is removed from the control plane properly using a custom cleanup function.
We have missed it in the original work.

### Fix: aligns provider name between template and cleanup logic

This is short-term fix for the existing codebase. In the long term (which is actively worked on) we need to improve the way of how we are storing config information to limit cases where we rely on pre/suffixes. Cases like this should be kept as its own thing instead, as it represents the concept in the infrastructure/authz setup.

* chore: indentation

Signed-off-by: Wen Zhou <[email protected]>

* fix: use old package path till we cherry-pick refactor commit

Signed-off-by: Wen Zhou <[email protected]>

---------

Signed-off-by: Wen Zhou <[email protected]>
Co-authored-by: Edgar Hernández <[email protected]>
Co-authored-by: Edgar Hernández <[email protected]>
Co-authored-by: Cameron Garrison <[email protected]>
Co-authored-by: Wen Zhou <[email protected]>
Co-authored-by: Bartosz Majsak <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ok-to-test Ok to Test
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants