-
Notifications
You must be signed in to change notification settings - Fork 1.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Task with more than one workspace waiting forever to start #2829
Comments
/kind bug |
No - by intention. When the Affinity Assistant was introduced, the docs was also updated so that we recommend at most one writeable workspace per Task. If you really need to use two PVC volumes in a single Task - you should disable the Affinity Assistant.
And this will happen if e.g. a pipeline starts with two parallel tasks - even without the Affinity Assistant. In the worst case - these two volumes are located in two different Availability Zones - and your Pipeline is deadlocked - with or without the Affinity Assistant. For this reason I had to remove an end-to-end-test using two volumes - that sometimes ended up in to different Availability Zones in a Regional cluster: #2546 If you really want to use two volumes in a single Task - you must disable the Affinity Assistant, but you will must likely run into problems in regional clusters - but it depends on how your storage work. There are two alternatives, either disable the Affinity Assistant (but parallelism in pipeline will be harder) or design Tasks so that they use at most one workspace. |
We should probably validate and emit an Event or use the Admission Webhook to validate these situation to inform the user in a better way. |
@r0bj is there any specific reason to why you need two workspaces in the same Task? It is possible to have a single workspace volume but mount different parts of it using Is that a way to solve your use case? |
@jlpettersson thanks for explanation. In my case using |
A TaskRun that mount more than one PVC-backed workspace is incompatible with the Affinity Assistant. But there is no validation if the TaskRun is compatible - so the TaskRun Pod is stuck with little information on why to the user. This commit adds validation of TaskRuns. When a TaskRun is associated with an Affinity Assistant, it is checked that not more than one PVC workspace is used - if so, the TaskRun will fail with a TaskRunValidationFailed condition. Proposed in tektoncd#2829 (comment) Closes tektoncd#2864
A TaskRun that mount more than one PVC-backed workspace is incompatible with the Affinity Assistant. But there is no validation if the TaskRun is compatible - so the TaskRun Pod is stuck with little information on why to the user. This commit adds validation of TaskRuns. When a TaskRun is associated with an Affinity Assistant, it is checked that not more than one PVC workspace is used - if so, the TaskRun will fail with a TaskRunValidationFailed condition. Proposed in tektoncd#2829 (comment) Closes tektoncd#2864
A TaskRun that mount more than one PVC-backed workspace is incompatible with the Affinity Assistant. But there is no validation if the TaskRun is compatible - so the TaskRun Pod is stuck with little information on why to the user. This commit adds validation of TaskRuns. When a TaskRun is associated with an Affinity Assistant, it is checked that not more than one PVC workspace is used - if so, the TaskRun will fail with a TaskRunValidationFailed condition. Proposed in tektoncd#2829 (comment) Closes tektoncd#2864
A TaskRun that mount more than one PVC-backed workspace is incompatible with the Affinity Assistant. But there is no validation if the TaskRun is compatible - so the TaskRun Pod is stuck with little information on why to the user. This commit adds validation of TaskRuns. When a TaskRun is associated with an Affinity Assistant, it is checked that not more than one PVC workspace is used - if so, the TaskRun will fail with a TaskRunValidationFailed condition. Proposed in tektoncd#2829 (comment) Closes tektoncd#2864
A TaskRun that mount more than one PVC-backed workspace is incompatible with the Affinity Assistant. But there is no validation if the TaskRun is compatible - so the TaskRun Pod is stuck with little information on why to the user. This commit adds validation of TaskRuns. When a TaskRun is associated with an Affinity Assistant, it is checked that not more than one PVC workspace is used - if so, the TaskRun will fail with a TaskRunValidationFailed condition. Proposed in tektoncd#2829 (comment) Closes tektoncd#2864
A TaskRun that mount more than one PVC-backed workspace is incompatible with the Affinity Assistant. But there is no validation if the TaskRun is compatible - so the TaskRun Pod is stuck with little information on why to the user. This commit adds validation of TaskRuns. When a TaskRun is associated with an Affinity Assistant, it is checked that not more than one PVC workspace is used - if so, the TaskRun will fail with a TaskRunValidationFailed condition. Proposed in #2829 (comment) Closes #2864
Now that we have #2885 merged, should we close this issue? |
I would think so yes. If anyone feels it should be re-open, please comment 👼 |
@vdemeester: Closing this issue. In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
A TaskRun that mount more than one PVC-backed workspace is incompatible with the Affinity Assistant. But there is no validation if the TaskRun is compatible - so the TaskRun Pod is stuck with little information on why to the user. This commit adds validation of TaskRuns. When a TaskRun is associated with an Affinity Assistant, it is checked that not more than one PVC workspace is used - if so, the TaskRun will fail with a TaskRunValidationFailed condition. Proposed in tektoncd#2829 (comment) Closes tektoncd#2864
Expected Behavior
Task with more than one workspace starts.
Actual Behavior
Task with more than one workspace waiting forever to start because it cannot attach volumes.
Steps to Reproduce the Problem
ReadWriteOnce
volumesIs it possible to handle this situation with affinity-assistant enabled?
Additional Info
The text was updated successfully, but these errors were encountered: