You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Motivation: Why do you think this is important?
As we write many plugins in flytekit, secrets are very important for many plugins. For non Cloud cases, where IAM can help access the resource without needing explicit secrets, other services need secret to access these resources. We recommend using Vault, which writes kube-secret files, but users may use Confidant, kube-secrets, and other potential solutions. For non K8s cases, users may use AWS KMS etc.
The best way to ensure that for plugin-authors and users, the way the secrets are provisioned an attained should be an implementation detail. Flytekit should provide a canonical way of accessing secrets
Goal: What should the final outcome look like, ideally?
Users/Plugin writers should be able to provide secret keys during declaration of a task.
At runtime, the actual values for these secrets should be accessible through the context.
SECRET1_KEY="secret1"SECRET2_KEY="secret2"@task(secrets=[SECRET1_KEY, SECRET2_KEY])deffoo(x: int) ->int:
...
v1=flytekit.current_context().secrets.get(SECRET1_KEY)
v2=flytekit.current_context().secrets.get(SECRET2_KEY)
...
# For not function tasks, it would mostly not make sense to expose the api of arbitrary secrets to users# but the required secrets should be documented# But for plugin writers it should be possible to doclassMySQLTask(SQLTask):
_SEC1_KEY="secret1"def_init__(...):
super().__init__(..., secrets=[_SEC1_KEY])
# Every plugins should have a table of various config parameters available and various secrets required
Describe alternatives you've considered
Implicit usage of Vault of pod by users
The text was updated successfully, but these errors were encountered:
kumare3
added
enhancement
New feature or request
untriaged
This issues has not yet been looked at by the Maintainers
and removed
untriaged
This issues has not yet been looked at by the Maintainers
labels
Mar 3, 2021
Motivation: Why do you think this is important?
As we write many plugins in flytekit, secrets are very important for many plugins. For non Cloud cases, where IAM can help access the resource without needing explicit secrets, other services need secret to access these resources. We recommend using Vault, which writes kube-secret files, but users may use Confidant, kube-secrets, and other potential solutions. For non K8s cases, users may use AWS KMS etc.
The best way to ensure that for plugin-authors and users, the way the secrets are provisioned an attained should be an implementation detail. Flytekit should provide a canonical way of accessing secrets
Goal: What should the final outcome look like, ideally?
Users/Plugin writers should be able to provide secret keys during declaration of a task.
At runtime, the actual values for these secrets should be accessible through the context.
Describe alternatives you've considered
Implicit usage of Vault of pod by users
The text was updated successfully, but these errors were encountered: