Feature Request: S3 on Outposts Support

[bflad]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Description

Today, AWS has announced Amazon S3 on Outposts support.

Amazon S3 on Outposts delivers object storage to your on-premises AWS Outposts environment to meet local data processing and data residency needs. Using the S3 APIs and features, S3 on Outposts makes it easy to store, secure, tag, retrieve, report on, and control access to the data on your Outpost. AWS Outposts is a fully managed service that extends AWS infrastructure, services, and tools to virtually any data center, co-location space, or on-premises facility for a truly consistent hybrid experience.

HashiCorp has been working with the S3 and DataSync service teams to investigate adding near-term support for this functionality. This issue is a meta-tracking issue for potential individual features, which will be broken out into separate issues. Once the initial scope is defined, this tracking issue will be closed in preference of the individual feature requests.

Features and Enhancements

• New Data Source: aws_s3control_bucket
• New Resource: aws_s3control_bucket
• New Resource: aws_s3control_bucket_lifecycle_configuration
• New Resource: aws_s3control_bucket_policy
• New Resource: aws_s3outposts_endpoint
• resource/aws_datasync_location_s3: Add agent_arns and s3_storage_class arguments
• resource/aws_s3_access_point: Support S3 on Outposts

Example End-to-End Terraform Configuration

# This example is for illustrative purposes to show parts of a real-world use case.

# Outposts Configuration

data "aws_outposts_outpost" "example" {
  # ... configuration ...
}

resource "aws_vpc" "outpost" {
  cidr_block = "10.0.0.0/16"
}

resource "aws_security_group" "outpost" {
  vpc_id = aws_vpc.outpost.id
}

resource "aws_subnet" "outpost" {
  availability_zone = data.aws_outposts_outpost.example.availability_zone
  cidr_block        = cidrsubnet(aws_vpc.outpost.cidr_block, 8, 0)
  outpost_arn       = data.aws_outposts_outpost.example.arn
  vpc_id            = aws_vpc.outpost.id
}

resource "aws_s3outposts_endpoint" "outpost" {
  outpost_id        = data.aws_outposts_outpost.test.id
  security_group_id = aws_security_group.outpost.id
  subnet_id         = aws_subnet.outpost.id
}

resource "aws_s3control_bucket" "outpost" {
  bucket     = "example"
  outpost_id = data.aws_outposts_outpost.example.id
}

resource "aws_s3control_bucket_lifecycle_configuration" "example" {
  bucket = aws_s3control_bucket.outpost.arn

  rule {
    id = "example"

    expiration {
      days = 123
    }

    filter {
      prefix = "example"
    }
  }
}

resource "aws_s3control_bucket_policy" "outpost" {
  bucket = aws_s3control_bucket.outpost.arn
  policy = "{...}"
}

resource "aws_s3_access_point" "outpost" {
  bucket = aws_s3control_bucket.outpost.arn
  name   = "outpost"

  vpc_configuration {
    vpc_id = aws_vpc.outpost.id
  }
}

resource "aws_datasync_agent" "outpost" {
  # ... configuration ...
}

resource "aws_datasync_location_s3" "outpost" {
  agent_arns    = [aws_datasync_agent.outpost.arn]
  s3_bucket_arn = aws_s3_access_point.outpost.arn
  s3_storage_class = "OUTPOSTS"
  subdirectory  = "/"

  s3_config {
    bucket_access_role_arn = aws_iam_role.example.arn
  }
}

# Region Configuration

resource "aws_vpc" "region" {
  cidr_block = "10.1.0.0/16"
}

resource "aws_security_group" "region" {
  vpc_id = aws_vpc.region.id
}

resource "aws_subnet" "region" {
  availability_zone = data.aws_outposts_outpost.example.availability_zone
  cidr_block = cidrsubnet(aws_vpc.region.cidr_block, 8, 0)
  vpc_id = aws_vpc.region.id
}

resource "aws_s3_bucket" "region" {
  bucket = "example"
}

resource "aws_datasync_location_s3" "region" {
  s3_bucket_arn = aws_s3_bucket.region.arn
  subdirectory  = "/example/prefix"

  s3_config {
    bucket_access_role_arn = aws_iam_role.example.arn
  }
}

# Bridging Configuration

resource "aws_datasync_task" "example" {
  destination_location_arn = aws_datasync_location_s3.region.arn
  name                     = "example"
  source_location_arn      = aws_datasync_location_s3.outposts.arn
}

References

• Announcement
• Blog Post
• Outposts FAQs
• S3 on Outposts Product Website
• S3 on Outposts User Guide
• S3 Control API Reference
• S3 on Outposts API Reference
• AWS Go SDK v1.35.0
• AWS Go SDK service/s3control
• AWS Go SDK service/s3outposts

[bflad]

Much of the support for this is landing in version 3.13.0 of the Terraform AWS Provider, later this week. See the Features and Enhancements section above for links to the functionality covered and remaining items. For further feature requests or bug reports with this functionality not covered here, please file a new issue following the issue template. Thanks. 👍

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!