Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[backend]: Determine cache scope for pipeline-root and namespace #6324

Closed
2 of 3 tasks
zijianjoy opened this issue Aug 13, 2021 · 3 comments
Closed
2 of 3 tasks

[backend]: Determine cache scope for pipeline-root and namespace #6324

zijianjoy opened this issue Aug 13, 2021 · 3 comments
Assignees
Labels
lifecycle/stale The issue / pull request is stale, any activities remove this label.

Comments

@zijianjoy
Copy link
Collaborator

zijianjoy commented Aug 13, 2021

How should cache work when everything is same except the pipeline_root?
For example,
pipeline #1 uses minio://xxxx as the pipeline_root
pipeline #2 uses gs://xxxx as the pipeline_root
other configs of pipeline#1 and pipeline#2 are same.
Now pipeline #2 will hit the cache of pipeline #1 but it won't use the pipeline root under gs://xxxx, it will use the pipeline root under minio://xxxx as what pipeline#1 does.

Discussed with @Bobgy @IronPan, this is the expected behavior.

EDIT by @Bobgy :
However, before implementing support for downloading artifacts not from the pipeline root, the pipeline still breaks.
TODOs:

  • add this as a known caveat
  • when supporting importer, we need to do the same fix

EDIT by @capri-xiyue
For shared pipelines, cross-namespace cache hit is not allowed
TODOs:

@zijianjoy
Copy link
Collaborator Author

/assign @Bobgy
/assign @capri-xiyue

@stale
Copy link

stale bot commented Mar 2, 2022

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@stale stale bot added the lifecycle/stale The issue / pull request is stale, any activities remove this label. label Mar 2, 2022
Copy link

This issue has been automatically closed because it has not had recent activity. Please comment "/reopen" to reopen it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
lifecycle/stale The issue / pull request is stale, any activities remove this label.
Projects
None yet
Development

No branches or pull requests

3 participants