Use separate AWS accounts for Data Products / Teams

[bagg3rs]

User Story

As a Data Platform team member
I want to teams to be able to use AWS services and take ownership of their data
So that we can secure access to data and can allow innovation and not be held back by our platform

Value / Purpose

• Ownership is clear the AWS Account = Data Domain / Data Product
• Governance, security and operational requirements can be applied separately if required
• Reduce the scope of a security breach
• AWS tooling e.g. 🧑‍✈️ Run AWS SageMaker pilot for our users #1262 can sit within the account which it needs to access the data, similar with Quicksight.
• Adheres to AWS recommendations

Questions

• Derived data products
• Managing requests to data owners
• How could we enable existing teams (use terraform modules or some such on CP / DP)
• Governance of these accounts and controlling that
• Making sure accounts are managed securely and notifying owners and then managers if actions are not taking place.

Hypothesis

If we give teams their own "storage" accounts
Then we give them the ability to use tooling within that account and we stop being a blocker (although we will still have controls on accounts to stop or alert on e.g. spend)

Additional Information

Would need AzureAD to be tied into AWS IAM Identity Center to ease user access

Definition of Done

• Proposal created and discussed
• Request Data Platform Data Domain Account (Data of the DP)
• Request Data Platform Data Worker Account (to run the tooling)
• Request Data Domain for SageMaker PoC to get feedback from users.

[github-actions]

This issue is being marked as stale because it has been open for 60 days with no activity. Remove stale label or comment to keep the issue open.

[github-actions]

This issue is being closed because it has been open for a further 7 days with no activity. If this is still a valid issue, please reopen it, Thank you!

[bagg3rs]

Added to #3107