S3 buckets should have server-side encryption enabled

[dsotirho-ucsc]

{
    "GeneratorIds": [
        "aws-foundational-security-best-practices/v/1.0.0/S3.4"
    ]
}

https://docs.aws.amazon.com/securityhub/latest/userguide/securityhub-standards-fsbp-controls.html#fsbp-s3-4

---

• Security design review completed; the Resolution of this issue does not …
  • … affect authentication; for example:
    • OAuth 2.0 with the application (API or Swagger UI)
    • Authentication of developers with Google Cloud APIs
    • Authentication of developers with AWS APIs
    • Authentication with a GitLab instance in the system
    • Password and 2FA authentication with GitHub
    • API access token authentication with GitHub
    • Authentication with
  • … affect the permissions of internal users like access to
    • Cloud resources on AWS and GCP
    • GitLab repositories, projects and groups, administration
    • an EC2 instance via SSH
    • GitHub issues, pull requests, commits, commit statuses, wikis, repositories, organizations
  • … affect the permissions of external users like access to
    • TDR snapshots
  • … affect permissions of service or bot accounts
    • Cloud resources on AWS and GCP
  • … affect audit logging in the system, like
    • adding, removing or changing a log message that represents an auditable event
    • changing the routing of log messages through the system
  • … affect monitoring of the system
  • … introduce a new software dependency like
    • Python packages on PYPI
    • Command-line utilities
    • Docker images
    • Terraform providers
  • … add an interface that exposes sensitive or confidential data at the security boundary
  • … affect the encryption of data at rest
  • … require persistence of sensitive or confidential data that might require encryption at rest
  • … require unencrypted transmission of data within the security boundary
  • … affect the network security layer; for example by
    • modifying, adding or removing firewall rules
    • modifying, adding or removing security groups
    • changing or adding a port a service, proxy or load balancer listens on
• Documentation on any unchecked boxes is provided in comments below

[hannes-ucsc]

Some of our buckets contain managed-access metadata. The shared config bucket contains sensitive TF state. Enabling encryption at rest with AWS-managed keys should be relatively straight forward. There might be head aches with migration of existing objects.

[achave11-ucsc]

@hannes-ucsc: "Spike to investigate if existing objects are automatically encrypted when encryption is enabled on a bucket. If not this would become much more complicated."

[achave11-ucsc]

S3-controls.html#s3-4
Note:

⚠️ Important
Amazon S3 now applies server-side encryption with Amazon S3 managed keys (SSE-S3) as the base level of encryption for every bucket in Amazon S3. Starting January 5, 2023, all new object uploads to Amazon S3 are automatically encrypted at no additional cost and with no impact on performance. The automatic encryption status for S3 bucket default encryption configuration and for new object uploads is available in AWS CloudTrail logs, S3 Inventory, S3 Storage Lens, the Amazon S3 console, and as an additional Amazon S3 API response header in the AWS Command Line Interface and AWS SDKs. For more information, see Default encryption FAQ.

Which indicates that server side encryption has been automatically enforced by S3. This was confirmed by the S3.4 control in Security Hub showing as passing in the dev account. This is also true in the anvildev and prod accounts.

However, an additional configuration to encrypt objects via a KMS managed key is available. For this configuration existing objects need to be imported with the new encryption key.

When you configure an S3 Bucket Key, objects that are already in the bucket do not use the S3 Bucket Key. To configure an S3 Bucket Key for existing objects, you can use a COPY operation. For more information, see Configuring an S3 Bucket Key at the object level.

[achave11-ucsc]

We enabled Bucket inventory to confirm whether existing objects are encrypted. We'll re-triage to examine said inventory.

[achave11-ucsc]

There wasn't any concrete information regarding the missing header row, however one might be able to make an inference. The selected CSV format in the shared bucket inventory config has a note that reads…

"Choose this format if you plan to use Batch Operations …".

Furthermore, in the same user guide, section Converting empty version ID strings to null strings step nine reads …

"To use this CSV file as input for an S3 Batch Operations job, you must remove the header row, because Batch Operations doesn't support header rows on CSV manifests."

This lead me to the assumption that the CSV inventory that AWS S3 generates is compatible to use with Batch Operations without having to manually remove the header row for the report to be used with Batch Operations.

Also note that the inventory report also produces a dated name folder which contains a manifest.json file with "fileSchema": "Bucket, Key, VersionId, IsLatest, IsDeleteMarker".

For the missing fields described in Amazon S3 Inventory list one can select more fields to be included in the inventory by checking the desired fields in Additional metadata fields - optional from the Inventory Configuration.

[achave11-ucsc]

@hannes-ucsc: "We enabled the additional columns, we'll discuss when Abraham is back."

[achave11-ucsc]

@hannes-ucsc: "The inventory on the shared bucket revealed that all versioned objects in the bucket are encrypted. The only rows that had a blank encryption status were the IsDeleteMarker, which is expected. The reason that all versions, including versions from previous years are already encrypted is that we migrated the shared bucket to a renamed bucket in 2023. I currently have a change on a branch enabling inventory on all buckets, once this lands can reexamine the inventories of all buckets in order to find any objects that we would have to delete or encrypt."

[hannes-ucsc]

The inventory is now enabled everywhere. Assignee to check it all buckets and all object for encryption status. Check the finding from the description. No objects should be unencrypted and the finding should be fixed.

[dsotirho-ucsc]

Assignee to check [in] all buckets and all object for encryption status. Check the finding from the description. No objects should be unencrypted and the finding should be fixed.

The Security Hub rule S3.4 is passing in all four accounts.
All buckets have server side encryption enabled.
Not all objects inside all buckets* have encryption enabled, specifically:

• platform-hca-dev
  • data-browser.dev.lungmap.net
  • dev.explore.lungmap.net
• platform-hca-prod
  • archive-preview.humancellatlas.org
  • data-browser.lungmap.net

*Note: The objects in the "logs" and "trail" buckets were not checked due to the large number of objects they contain.

Bucket	Encrypted Files	Unencrypted Files
dev
cc-dev.explore.singlecell.gi.ucsc.edu	2854	0
cc-dev.singlecell.gi.ucsc.edu	901	0
data-browser.dev.lungmap.net	0	753
dev.archive-preview.singlecell.gi.ucsc.edu	59	0
dev.explore.lungmap.net	67	1
dev.explore.singlecell.gi.ucsc.edu	67	0
dev.singlecell.gi.ucsc.edu	946	0
edu-ucsc-gi-platform-hca-dev-awsconfig.us-east-1	3195	0
edu-ucsc-gi-platform-hca-dev-logs.us-east-1	-	-
edu-ucsc-gi-platform-hca-dev-shared.us-east-1	981	0
edu-ucsc-gi-platform-hca-dev-storage-abrahamsc.us-east-1	2	0
edu-ucsc-gi-platform-hca-dev-storage-daniel.us-east-1	2	0
edu-ucsc-gi-platform-hca-dev-storage-daniel2.us-east-1	2	0
edu-ucsc-gi-platform-hca-dev-storage-dev.us-east-1	107	0
edu-ucsc-gi-platform-hca-dev-storage-hannes.us-east-1	2	0
edu-ucsc-gi-platform-hca-dev-storage-nadove.us-east-1	32	0
edu-ucsc-gi-platform-hca-dev-storage-nadove2.us-east-1	2	0
edu-ucsc-gi-platform-hca-dev-storage-nadove3.us-east-1	2	0
edu-ucsc-gi-platform-hca-dev-storage-sandbox.us-east-1	107	0
edu-ucsc-gi-platform-hca-dev-trail.us-east-1	-	-
prod
archive-preview.humancellatlas.org	161	501
data-browser.explore.lungmap.net	67	0
data-browser.lungmap.net	0	753
edu-ucsc-gi-platform-hca-prod-awsconfig.us-east-1	1194	0
edu-ucsc-gi-platform-hca-prod-logs.us-east-1	-	-
edu-ucsc-gi-platform-hca-prod-shared.us-east-1	66	0
edu-ucsc-gi-platform-hca-prod-storage-prod.us-east-1	346	0
edu-ucsc-gi-platform-hca-prod-trail.us-east-1	-	-
org-humancellatlas-data-browser-dcp2-prod	63	0
org-humancellatlas-data-portal-dcp2-prod	934	0
anvildev
edu-ucsc-gi-platform-anvil-dev-awsconfig.us-east-1	3	0
edu-ucsc-gi-platform-anvil-dev-browser.us-east-1	4	0
edu-ucsc-gi-platform-anvil-dev-consortia.us-east-1	5	0
edu-ucsc-gi-platform-anvil-dev-logs.us-east-1	-	-
edu-ucsc-gi-platform-anvil-dev-portal.us-east-1	6	0
edu-ucsc-gi-platform-anvil-dev-shared.us-east-1	5	0
edu-ucsc-gi-platform-anvil-dev-storage-achave11.us-east-1	2	0
edu-ucsc-gi-platform-anvil-dev-storage-anvilbox.us-east-1	0	0
edu-ucsc-gi-platform-anvil-dev-storage-anvildev.us-east-1	8	0
edu-ucsc-gi-platform-anvil-dev-storage-nadove4.us-east-1	4	0
edu-ucsc-gi-platform-anvil-dev-storage-nadove5.us-east-1	2	0
edu-ucsc-gi-platform-anvil-dev-trail.us-east-1	-	-
anvilprod
edu-ucsc-gi-platform-anvil-prod-awsconfig.us-east-1	5	0
edu-ucsc-gi-platform-anvil-prod-browser.us-east-1	8	0
edu-ucsc-gi-platform-anvil-prod-consortia.us-east-1	5	0
edu-ucsc-gi-platform-anvil-prod-logs.us-east-1	-	-
edu-ucsc-gi-platform-anvil-prod-portal.us-east-1	6	0
edu-ucsc-gi-platform-anvil-prod-shared.us-east-1	3	0
edu-ucsc-gi-platform-anvil-prod-storage-anvilprod.us-east-1	5	0
edu-ucsc-gi-platform-anvil-prod-storage-hammerbox.us-east-1	2	0
edu-ucsc-gi-platform-anvil-prod-storage-nadove6.us-east-1	0	0
edu-ucsc-gi-platform-anvil-prod-trail.us-east-1	-	-

dev

prod

anvildev

anvilprod

[dsotirho-ucsc]

Assignee to consult bucket inventory instead of custom script to determine encryption status

[dsotirho-ucsc]

Confirmed all S3 objects in all accounts are encrypted.

Note: There were some unencrypted entries listed in the bucket inventory, however all of these rows also had the IsDeleteMarker flag set to true. Spot checking about 10 of these entries flagged as a delete marker confirmed that the object referenced is not present in the related bucket. This Google sheet contains all these unencrypted delete marker entries.

Full list of inventories processed:

daniel@Crispin ~/Downloads/AWS_S3_Inventory/2023-09-05 $ ls -1
edu-ucsc-gi-platform-anvil-dev-awsconfig.us-east-1.csv
edu-ucsc-gi-platform-anvil-dev-browser.us-east-1.csv
edu-ucsc-gi-platform-anvil-dev-consortia.us-east-1.csv
edu-ucsc-gi-platform-anvil-dev-logs.us-east-1.csv
edu-ucsc-gi-platform-anvil-dev-portal.us-east-1.csv
edu-ucsc-gi-platform-anvil-dev-shared.us-east-1.csv
edu-ucsc-gi-platform-anvil-dev-storage-achave11.us-east-1.csv
edu-ucsc-gi-platform-anvil-dev-storage-anvilbox.us-east-1.csv
edu-ucsc-gi-platform-anvil-dev-storage-anvildev.us-east-1.csv
edu-ucsc-gi-platform-anvil-dev-storage-nadove4.us-east-1.csv
edu-ucsc-gi-platform-anvil-dev-storage-nadove5.us-east-1.csv
edu-ucsc-gi-platform-anvil-dev-trail.us-east-1.csv
edu-ucsc-gi-platform-anvil-prod-awsconfig.us-east-1.csv
edu-ucsc-gi-platform-anvil-prod-browser.us-east-1.csv
edu-ucsc-gi-platform-anvil-prod-consortia.us-east-1.csv
edu-ucsc-gi-platform-anvil-prod-logs.us-east-1.csv
edu-ucsc-gi-platform-anvil-prod-portal.us-east-1.csv
edu-ucsc-gi-platform-anvil-prod-shared.us-east-1.csv
edu-ucsc-gi-platform-anvil-prod-storage-anvilprod.us-east-1.csv
edu-ucsc-gi-platform-anvil-prod-storage-hammerbox.us-east-1.csv
edu-ucsc-gi-platform-anvil-prod-storage-nadove6.us-east-1.csv
edu-ucsc-gi-platform-anvil-prod-trail.us-east-1.csv
edu-ucsc-gi-platform-hca-dev-awsconfig.us-east-1.csv
edu-ucsc-gi-platform-hca-dev-logs.us-east-1.csv
edu-ucsc-gi-platform-hca-dev-shared.us-east-1.csv
edu-ucsc-gi-platform-hca-dev-storage-abrahamsc.us-east-1.csv
edu-ucsc-gi-platform-hca-dev-storage-daniel.us-east-1.csv
edu-ucsc-gi-platform-hca-dev-storage-dev.us-east-1.csv
edu-ucsc-gi-platform-hca-dev-storage-hannes.us-east-1.csv
edu-ucsc-gi-platform-hca-dev-storage-nadove.us-east-1.csv
edu-ucsc-gi-platform-hca-dev-storage-nadove2.us-east-1.csv
edu-ucsc-gi-platform-hca-dev-storage-sandbox.us-east-1.csv
edu-ucsc-gi-platform-hca-dev-trail.us-east-1.csv
edu-ucsc-gi-platform-hca-prod-awsconfig.us-east-1.csv
edu-ucsc-gi-platform-hca-prod-logs.us-east-1.csv
edu-ucsc-gi-platform-hca-prod-shared.us-east-1.csv
edu-ucsc-gi-platform-hca-prod-storage-prod.us-east-1.csv
edu-ucsc-gi-platform-hca-prod-trail.us-east-1.csv

[dsotirho-ucsc]

@hannes-ucsc: "Assignee to file ticket against data browser and/or portal repositories about the buckets that still have unencrypted objects. They all seem to be owned by Clever Canary. After ticket has been created, please start a conversation about this with Dave in our Slack channel."

[dsotirho-ucsc]

Assignee to file ticket against data browser and/or portal repositories about the buckets that still have unencrypted objects.

• Enable server-side encryption for all S3 objects data-browser#3647

After ticket has been created, please start a conversation about this with Dave in our Slack channel.

• Slack thread

[achave11-ucsc]

@hannes-ucsc: "There are no unencrypted buckets in anvilprod anymore, so we can remove #5170 from the list of blocked tickets."

[hannes-ucsc]

DataBiosphere/data-browser#3647 (comment)