Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add CDN for archived data in S3 #1264

Merged

Conversation

Mr0grog
Copy link
Collaborator

@Mr0grog Mr0grog commented Jan 31, 2023

This adds a CloudFront distribution to serve data from our "data snaphsots" S3 bucket at https://archives.getmyvax.org. The goal here is mainly to prevent people from potentially driving up S3 costs by making requests we can't control or cache against the bucket. (Per @TylerHendrickson’s guidance.)

This is a first step for #1180. After we confirm this is deployed and working, we’ll need to revoke public read access from the S3 bucket.

@Mr0grog
Copy link
Collaborator Author

Mr0grog commented Feb 1, 2023

OK, I’ve implemented this both directly with the AWS provider and using a module from Cloud Posse that makes the setup significantly simpler. I think the simplicity is probably worthwhile, but there are some caveats worth logging here:

  • It might be broken! It triggers one deprecation warning in Terraform which should be OK, but this issue suggests there may be bigger problems: version 4.0.0 of the aws provider contains breaking changes for the CDN module cloudposse/terraform-aws-cloudfront-s3-cdn#217. It’s possible I’m just not using the problematic features, or maybe it’s since been fixed and nobody closed the issue. I guess we’ll see. 🤷

  • It does so many things that properly understanding how to configure it correctly can be a bit confusing. See, for example, how many commits I made so I could see what Terraform’s plan output was and adjust (e.g. several settings determine whether alternate domain names are used, and the implications of that is not clearly documented anywhere). There is definitely some cost to this abstraction.

    (OTOH, I had forgotten to create an IPv6 DNS record in my manual version, and this module takes care of that. Good defaults, mostly.)

@Mr0grog Mr0grog changed the title Add CloudFront for accessing archived data in S3 Add CDN for archived data in S3 Feb 1, 2023
@Mr0grog Mr0grog merged commit a09fd4d into main Feb 1, 2023
@Mr0grog Mr0grog deleted the 1180-dont-just-put-bare-s3-buckets-in-the-public-spotlight branch February 1, 2023 00:27
Mr0grog added a commit that referenced this pull request Feb 1, 2023
In #1264, I used Cloud Posse's cloudfront-s3-cdn module to create a CloudFront distribution in front of our data snapshots bucket. It turned out to have some bugs that weren't obvious until trying to actually deploy and create resources. This *should* address those. (We'll see if more show up!)
Mr0grog added a commit that referenced this pull request Feb 1, 2023
In #1264, I used Cloud Posse's cloudfront-s3-cdn module to create a CloudFront distribution in front of our data snapshots bucket. It turned out to have some bugs that weren't obvious until trying to actually deploy and create resources. This *should* address those. (We'll see if more show up!)
Mr0grog added a commit that referenced this pull request Feb 1, 2023
Looks like I need to flip this back on in order for CloudFront to be able to read the bucket now that it's private. Oops! (This is a follow-on to #1264 and #1267.)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant