-
Notifications
You must be signed in to change notification settings - Fork 909
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Refactor service agent management #2423
Conversation
blueprints/data-solutions/data-platform-foundations/03-orchestration.tf
Outdated
Show resolved
Hide resolved
blueprints/data-solutions/data-platform-foundations/04-transformation.tf
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is really great. Dropped a couple of comments.
One more poitn we should maybe tackle is inter-service dependency: one example I'm currently facing in real life is Dataproc requiring KMS grants on the GCE, GCS, and its own service agent.
73e6973
to
bada713
Compare
Waiting for e2e to finish before merging |
This PR significantly improves how Fabric handles service agents. This is a very large PR but IMO it will simplify things moving forward.
The main drivers for this PR are
This PR introduces the following changes:
service-agents.yaml
. The idea is to run the script manually from time to timeproject.service_accounts
output is split intoproject.service_agents
andproject.default_service_accounts
. The Cloud Service Service Agent is handled like any other service agent withinservice-agents.yaml
.service_agents_config
) to control the default behaviour in regards to service agent handling. By default we enable:project.service_encryption_key_ids
variable is renamed toproject.service_encryption_key_ids
for clarity. (Suggestions for a better name are welcome!)There's a lot of noise in this PR as a lot of modules, blueprints and tests changed. For reviewers, please focus on the changes within the
modules/project
directory. While many files have been modified, the core logic changes reside there.