-
Notifications
You must be signed in to change notification settings - Fork 140
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: introduces OSSM plugin for KfDef #515
Conversation
ff71a25
to
d0b3a2f
Compare
// +optional | ||
// +kubebuilder:pruning:PreserveUnknownFields | ||
Spec *runtime.RawExtension `json:"spec,omitempty"` |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Without this option we are not able to use custom struct
s for configuration.
pkg/kfapp/coordinator/coordinator.go
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As the plugin being defined as the platform is not participating in Apply
and Delete
phases by default, we need to hook into those in order to configure Service Mesh. GCP integration is done similarly.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is a copy of pkg/utils/k8utils.go
, but I wanted to keep it isolated from the main codebase. It has additional func for applying a patch file.
pkg/kfapp/ossm/ossm_installer.go
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is the central place of kf plugin integration.
* add data science migration to init Co-authored-by: Bartosz Majsak <[email protected]>
* chore: removes unused test setup * feat(test): sets up testing for ossm plugin * feat(make): adds test-ossm target
* feat: introduces OssmResourceTracker resource It is a cluster-scoped resource for tracking objects created by Ossm plugin. Primarily used as owner reference for resources created across namespaces so that they can be garbage collected by Kubernetes when they are not needed anymore. * feat: uses OssmResourceTracker to cleanup owned resources - adds cleanup hook for Delete phase of KfDef operator - moves k8s_utils code to ossm package to keep it in one place for future extraction of the logic * chore: renames plugin to OssmInstaller * chore: moves cleanup logic to single file * feat: handles token volume removal in SMCP * chore: adds plugin name as extra key-value to logger * feat: removes extAuthzProvider on deletion * chore: minor naming fixes
embed templates to OSSM plugin Co-authored-by: Bartosz Majsak <[email protected]>
they might have been removed in some merge (?)
* add prereq checks to init, add tests --------- Co-authored-by: Bartosz Majsak <[email protected]>
- easy builder to define what particular feature consists of - enables the use of existing YAML manifests - allows the definition of additional resources programmatically with a simple func registration - cleanup functions are now part of each feature and are called one after the other - each feature has its own resource tracker object - ability to test smaller parts of the whole thing by using `Feature`s in integration tests
* chore: extracts smcp GVR to a var and reuse it in smcp-related funcs * fix: returns err on oauth creation failure instead of nil by mistake * feat: adds ability to enable feature based on predicates in certain cases this information can only be determined at runtime, for example based on user-defined values of the plugin spec. * feat: introduces control plane installation coupled with wait condition * fix: makes feature enabled by default * chore: reworks smcp creation It relies on owner reference so can be cleaned up as other resources * fix: applies cleanups only if feature is enabled Additionally moves OssmResourceTracker creation to .Apply func * feat: reworks plugin integration to use Apply and Delete funcs of KfApp This way we can cleanup the resources in case deploying service mesh manifests because creating those in Generate phase leaves them hanging due to failing Delete hook * fix(reconcile): corrects KfDef status in case of any error Additionally propagates the error through reconcile to keep trying, as it was ignored. * fix: reworks SMCP component readiness check * chore: improves KfApp func docs * chore: reworks logging in verifiers * chore: simplifies CRD existence check * chore: adds tests for feature enablement * fix: sets interval for SMCP polling to 5s * fix: handles types which are derived for built-in ones for example `type InstallationMode string` was previously failing when converting the default value defined in a custom tag. Now there's a check performed and if value can be convereted to a given type it will. In all other cases it will return an error. * fix: sets default control plane installation mode rely on existing installation Proposed installation modes are: - minimal - pre-installed
Without this setting certain environments are not working, as service account issuer might be set to different endpoint than kubernetes.default.svc
… cluster without this fix we could fetch OdhDashboardConfig from different ns and try to perform an update on selected app ns, which leads to error and blocks operator from performing entire OSSM setup. Oh well...
otherwise we create route for ns.domain but expects hosts to be opendatahub.domain which obviously does not work. Doh!
Creates ossm.env in each overlays/service-mesh of defined apps so that it can be used e.g. to overwrite configmap values
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks great - nice fixes in recent commits. Have also tested this E2E in the past with success using the Feature
implementation.
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: cam-garrison The full list of commands accepted by this bot can be found here.
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
Co-authored-by: Cameron Garrison <[email protected]>
New changes are detected. LGTM label has been removed. |
Co-authored-by: Cameron Garrison <[email protected]>
@bartoszmajsak: The following test failed, say
Full PR test history. Your PR dashboard. Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here. |
sorry for late chime in @bartoszmajsak : |
@zdtsw that's a valid concern. However, the context here is broader than just sunsetting v1. For ODH users still on v1 who require service mesh, the suggested approach offers clear advantages over our current Based on the comment from @LaVLaS I understood we need both. It would be great if he could provide more clarity on this matter. |
ok, I see. I was not aware of this issue. but yeah, lets wait for more inputs. btw, for the manifests, we are doing the transition to move away from |
I am aware of this (found out yesterday by accident ;)) and we are already working on the split on our end. But I see not everything is ported yet, e.g. notebook controllers? opendatahub-operator/get_all_manifests.sh Lines 22 to 32 in 0b2bb98
|
good to know that!
it is up to when servicemesh can finish "split" work. |
@zdtsw is this split going to affect v1/KfDef operator as well? |
Withdrawing in favor of Operator V2 support #605 |
The operator bundle has not been officially released yet. If you wish to test it, please use for the time being:
It will need opendatahub-io/odh-manifests#932 to be merged alongside to use
service-mesh-integration
branch.Description
This PR introduces a
KfDef
plugin aimed at enhancing the configuration and deployment functionalities of OpenShift Service Mesh. The development was inspired by analysis of how other platforms like GCP and AWS manage integrations. We've designed these improvements to be easily integrated into the upcoming version of the opendatahub-operator, while minimizing changes to the existing codebase. All new features are contained within dedicated packages.The plugin addresses several limitations we encountered when using the conventional manifest and init-job solution, specifically:
KfDef
file. An example is provided below.This PR includes #501 fix.
Manifests used for the plugin can be found here https://github.com/maistra/odh-manifests/tree/ossm_plugin_templates. PR opendatahub-io/odh-manifests#932
KfDef
example(1)
-installationMode
: currently it's eitherminimal
which deploys our own, very basic control plane, or assumes there's one already which we should use (installationMode: pre-installed
)(2)
- This will generate a self-signed certificate used by Mesh internally. This will need further enhancements.(3)
- notice there's no need to explicitly addservice-mesh
overlay. If an application has one defined it will be detected and enabled automaticallyHow Has This Been Tested?
test-ossm
has been providedMerge criteria: