Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: add a hash suffix based on the spec for pipeline #244

Closed
wants to merge 2 commits into from

Conversation

xdevxy
Copy link
Contributor

@xdevxy xdevxy commented Sep 10, 2024

Fixes #243

Modifications

Since we may have multiple pipelines associate with a pipeline rollout soon for the no downtime upgrade feature. This PR adds the logic to uniquely identify a pipeline that associate with a pipeline rollout based on its spec. To deterministic identify a pipeline, added a hash of the pipeline spec as a suffix to the name of the pipeline.

This PR also fixes a bug that when a pipeline with the same name already exist without a pipeline Rollout manage it. It should update the phase to failed without checking the child status.

Verification

make test
make test-e2e

@juliev0
Copy link
Collaborator

juliev0 commented Sep 11, 2024

hmm, looks like e2e failed?

if pipelineSpec.InterStepBufferServiceName != "" {
labelMapping[common.LabelKeyISBServiceNameForPipeline] = pipelineSpec.InterStepBufferServiceName
} else {
labelMapping[common.LabelKeyISBServiceNameForPipeline] = "default"
Copy link
Collaborator

@juliev0 juliev0 Sep 11, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe doesn't need to be done in this PR, but are we going to add a label for PipelineRollout name? Then we can easily find all Pipelines using that PipelineRollout. I know the Pipeline name will be prefixed with the PipelineRollout name, but I think this would enable easy selection both through kubectl and potentially in the code.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

unless we can do a query on OwnerReference I guess...but just to note that Numaflow includes labels like this

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FYI, I have just created this issue which depends on this

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah, that will be the follow up PR for this.

@xdevxy xdevxy force-pushed the pipeline_name branch 3 times, most recently from b0cb33d to 68d146a Compare September 11, 2024 19:48
Signed-off-by: Hao Hao <[email protected]>
@xdevxy
Copy link
Contributor Author

xdevxy commented Sep 12, 2024

Decided to postponed this PR as it can introduce the issue that multiple pipelines associate with a pipelineRollout after a spec change.

@xdevxy xdevxy closed this Sep 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support multiple pipelines can coexist for a PipelineRollout
2 participants