Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

S3InputFormat support buckets with "." in their name #2132

Closed
ehochmuth opened this issue Apr 9, 2017 · 5 comments
Closed

S3InputFormat support buckets with "." in their name #2132

ehochmuth opened this issue Apr 9, 2017 · 5 comments
Milestone

Comments

@ehochmuth
Copy link
Contributor

ehochmuth commented Apr 9, 2017

The current regular expression in S3InputFormat that parses the bucket and key/prefix only allows bucket names with "-". Extends the S3InputFormat regular expression to support both "-" and "." in bucket names.

https://github.com/locationtech/geotrellis/blob/master/s3/src/main/scala/geotrellis/spark/io/s3/S3InputFormat.scala#L200

#2133

@notthatbreezy
Copy link

Does it make sense to lean on a standard library class to limit other edge cases like this?

For instance, URI seems to satisfy the functionality here:

scala> import java.net.URI
import java.net.URI

scala> val s3URI = new URI("s3://username:[email protected]/path/to/objects")
s3URI: java.net.URI = s3://username:[email protected]/path/to/objects

scala> s3URI.getHost
res10: String = bucket.with.periods

scala> s3URI.getPath
res11: String = /path/to/objects

scala> s3URI.getScheme
res12: String = s3

scala> s3URI.getUserInfo
res13: String = username:password

@ehochmuth
Copy link
Contributor Author

Or make the username and password part of the backend-profile and then just let amazon do the work: com.amazonaws.services.s3.AmazonS3URI

@ehochmuth
Copy link
Contributor Author

Moving over to Pull Request proposal: #2133

@lossyrob
Copy link
Member

Thanks @ehochmuth. We usually keep an issue open, and the PR can say "Fixes #XXXX" in the description, which will automatically close the issue when the PR gets merged. Reopening and setting that in the description.

@lossyrob lossyrob reopened this Apr 11, 2017
@lossyrob lossyrob modified the milestones: 1.2, 1.1 Apr 26, 2017
@lossyrob
Copy link
Member

Fixed by #2133, not sure why it didn't close automatically.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants