Skip to content

Commit

Permalink
storageccl: improve support for non-S3 endpoints
Browse files Browse the repository at this point in the history
Remove the requirement for a region if an endpoint is specified by using
a bogus default region string.

If an endpoint is specified, use the old style S3 urls
(http://endpoint/bucket instead of http://bucket.endpoint/).

These two changes were tested with Minio and DO Spaces and both now work
as expected (without specifying a region and no weird endpoint URL).

Fixes #24224
Fixes #21412

Release note (sql change): Improve support for S3-compatible endpoints
in BACKUP, RESTORE, and IMPORT. The AWS_REGION parameter is no longer
required. Services like Digital Ocean Spaces and Minio now work correctly.
  • Loading branch information
maddyblue committed Aug 8, 2018
1 parent 0ffea0c commit 9f58dce
Showing 1 changed file with 4 additions and 1 deletion.
5 changes: 4 additions & 1 deletion pkg/ccl/storageccl/export_storage.go
Original file line number Diff line number Diff line change
Expand Up @@ -529,7 +529,7 @@ func makeS3Storage(
if conf.Endpoint != "" {
config.Endpoint = &conf.Endpoint
if conf.Region == "" {
return nil, errors.New("s3 region must be specified when using custom endpoints")
region = "default-region"
}
client, err := makeHTTPClient(settings)
if err != nil {
Expand All @@ -552,6 +552,9 @@ func makeS3Storage(
}
}
sess.Config.Region = aws.String(region)
if conf.Endpoint != "" {
sess.Config.S3ForcePathStyle = aws.Bool(true)
}
return &s3Storage{
bucket: aws.String(conf.Bucket),
conf: conf,
Expand Down

0 comments on commit 9f58dce

Please sign in to comment.