S3 Region

[Nugine]

It's not clear that how to deal with s3 regions in s3s now.

[St4NNi]

Hi, we have a use case that would really benefit from being able to access the region from a single s3s endpoint. What is the current status here? Is it possible to extract the S3 region from the headers?

Thanks in advance!

[Nugine]

we have a use case that would really benefit from being able to access the region from a single s3s endpoint.

It would be helpful to know more details about the use case. We can discuss design ideas here.

What is the current status here?

s3s has two places that mention the region now. The status is TODO.

s3s/crates/s3s/src/host.rs

Line 11 in e015724

// pub(crate) region: Option<Cow<'a, str>>,

s3s/crates/s3s/src/sig_v4/methods.rs

Line 232 in e015724

ans.push_str(region); // TODO: use a `Region` type

Is it possible to extract the S3 region from the headers?

Yes it is possible. The information is available in S3Request. I guess we just need to add some utils about regions.

[St4NNi]

It would be helpful to know more details about the use case. We can discuss design ideas here.

Sure, our data management solution has the idea of realms, a collection of multiple tenants that have agreed on certain rules (like metadata formats, naming conventions, data usage agreement requirements, etc.). Data is made available via s3s. In an ideal world, we would like to separate the realms as much as possible (including, for example, the ability to have only realm-unique bucket names).

We thought about "misusing" S3 regions because they more or less provide the functionality we want, except in our use case regions are not geographically separated, but logically.

I have hoped that this could work without a dedicated subdomain aka. <region>.<hostname> but having a deeper look at the official S3 specs it seems like only some requests (like CreateBucket) allow for regions to be specified directly, others like GetObject expect the region to be part of the hostname.

Official AWS S3 host header configuration

The main problem with subdomains in an environment where the regions might change dynamically is that wildcard TLS certificates only work for a single layer, so while the dns entry *.foo.bar will work for baz.bat.foo.bar a wildcard TLS certificate will only match one level e.g. bat.foo.bar

[Nugine]

The main problem with subdomains in an environment where the regions might change dynamically is that wildcard TLS certificates only work for a single layer, so while the dns entry *.foo.bar will work for baz.bat.foo.bar a wildcard TLS certificate will only match one level e.g. baz.bat

• Wildcard SSL certificate for second-level subdomain is not possible due to RFC6125.
• AWS has a list of all regions.

So dynamic regions can not be in the hostname.
The realm information can be put in query strings or custom headers. But there may be some compatibility problems.

[St4NNi]

This could work if dynamic regions will only use the legacy path-style requests but this is also not ideal.