resources: allow job authors to configure size of secrets tmpfs #23696
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
tgross
force-pushed
the
f-jobspec-secrets-size
branch
from
a61da74
to
be27109
Compare
tgross
force-pushed
the
f-jobspec-secrets-size
branch
from
be27109
to
6bd71f2
Compare
tgross
force-pushed
the
f-jobspec-secrets-size
branch
from
6bd71f2
to
e39908b
Compare
tgross
force-pushed
the
f-jobspec-secrets-size
branch
3 times, most recently
from
6941e13
to
9d51553
Compare
tgross
force-pushed
the
f-jobspec-secrets-size
branch
from
9d51553
to
7c82818
Compare
On supported platforms, the secrets directory is a 1MiB tmpfs. But some tasks need larger space for downloading large secrets. This is especially the case for tasks using `templates`, which need extra room to write a temporary file to the secrets directory that gets renamed to the old file atomically. This changeset allows increasing the size of the tmpfs in the `resources` block. Because this is a memory resource, we need to include it in the memory we allocate for scheduling purposes. The task is already prevented from using more memory in the tmpfs than the `resources.memory` field allows, but can bypass that limit by writing to the tmpfs via `template` or `artifact` blocks. Therefore, we need to account for the size of the tmpfs in the allocation resources. Simply adding it to the memory needed when we create the allocation allows it to be accounted for in all downstream consumers, and then we'll subtract that amount from the memory resources just before configuring the task driver. For backwards compatibility, the default value of 1MiB is "free" and ignored by the scheduler. Otherwise we'd be increasing the allocated resources for every existing alloc, which could cause problems across upgrades. If a user explicitly sets `resources.secrets = 1` it will no longer be free. Fixes: #2481 Ref: https://hashicorp.atlassian.net/browse/NET-10070
tgross
force-pushed
the
f-jobspec-secrets-size
branch
from
7c82818
to
781e3a1
Compare
tgross
commented
Jul 29, 2024
tgross
force-pushed
the
f-jobspec-secrets-size
branch
from
8d8cd9f
to
c9e0327
Compare
tgross
requested review from
shoenig,
gulducat,
Juanadelacuesta,
jrasell and
schmichael
and removed request for
schmichael
shoenig
reviewed
Aug 5, 2024
shoenig
approved these changes
Aug 5, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
On supported platforms, the secrets directory is a 1MiB tmpfs. But some tasks need larger space for downloading large secrets. This is especially the case for tasks using
templates, which need extra room to write a temporary file to the secrets directory that gets renamed to the old file atomically.This changeset allows increasing the size of the tmpfs in the
resourcesblock. Because this is a memory resource, we need to include it in the memory we allocate for scheduling purposes. The task is already prevented from using more memory in the tmpfs than theresources.memoryfield allows, but can bypass that limit by writing to the tmpfs viatemplateorartifactblocks.Therefore, we need to account for the size of the tmpfs in the allocation resources. Simply adding it to the memory needed when we create the allocation allows it to be accounted for in all downstream consumers, and then we'll subtract that amount from the memory resources just before configuring the task driver.
For backwards compatibility, the default value of 1MiB is "free" and ignored by the scheduler. Otherwise we'd be increasing the allocated resources for every existing alloc, which could cause problems across upgrades. If a user explicitly sets
resources.secrets = 1it will no longer be free.Fixes: #2481
Ref: https://hashicorp.atlassian.net/browse/NET-10070