Allow for alternate S3 backends

[cleverdevil]

It'd be great if an alternate endpoint could be specified in the configuration for those of us who are using S3-compatible services (like internal S3/Ceph deployments or DreamHost's DreamObjects).

[benwerd]

Cool idea!

On Wed, Aug 5, 2015 at 2:14 PM, Jonathan LaCour [email protected]
wrote:

It'd be great if an alternate endpoint could be specified in the
configuration for those of us who are using S3-compatible services (like
internal S3/Ceph deployments or DreamHost's DreamObjects).

—
Reply to this email directly or view it on GitHub
#4.

Ben Werdmuller
http://goog_1933028737
benwerd.com | werd.io

+1 (312) 488-9373

[cleverdevil]

I'm not a PHP programmer, unfortunately (at least, its been years since I've touched PHP), so I would struggle to get this done myself. That said, I was able to play around a little bit, and at least get uploading working.

You can provide a base_url parameter to the AWS factory, along with the key and secret, in Main.php:

$aws = \Aws\Common\Aws::factory(array(
    'key'     => \Idno\Core\site()->config()->aws_key,
    'secret'  => \Idno\Core\site()->config()->aws_secret,
    'base_url' => 'http://objects.dreamhost.com',
));

Once you do this, uploading to DreamObjects works, but it doesn't appear to actually properly load the uploads. Probably just need a few more tweaks elsewhere. Ideally, I'd like to be able to provide aws_key, aws_secret, aws_region, and aws_base_url as options. In addition, it'd be nice if I could provide a CNAME for accessing the content, if it has been CDN'd. With DreamObjects, we allow for mapping a CNAME to a bucket, and the contents will be pushed globally, similar to CloudFront. At that point, all that's necessary is to use the CNAME rather than the standard URL lookup.

Continuing to enjoy Known. Thanks for your hard work!

[benwerd]

The latest commit allows you to include the following two optional settings in your config.ini:

• aws_base_url: set the base URL as you've described
• aws_suppress_region: don't list a region in the factory settings

Additionally, you can use the following setting to set a CloudFront-style base URL:

• cloudfront

You're making me aware that all of the terminology here is service-specific and doesn't have to be. We should consider changing the options and terminology to be service-agnostic.

[cleverdevil]

Cool. Testing now... I'll report back :)

Let me know if y'all need a free account with DreamObjects. Happy to provide one for testing purposes!

[cleverdevil]

Doesn't appear to work, yet. The content is uploaded to DreamObjects successfully, but it looks like the S3 filesystem layer is having trouble reading the content back down again, for some reason. Not sure if its properly using the base URL.

I haven't enabled the cloudfront stuff yet.

[benwerd]

I wonder if it's a file permissions issue?

I would very much appreciate a testing DreamObjects testing account if you're in the position to give one! That would allow us to test more directly.

[cleverdevil]

Go ahead and signup for the free trial - https://www.dreamhost.com/cloud/storage/ - and then ping me at jonathan at dreamhost via email, and I'll mark it as free.

[Lewiscowles1986]

I'm fairly certain this now works. I'll spin up a copy against minio in docker this weekend to confirm

https://github.com/idno/amazon-s3/pull/8/files#diff-576b8cc9365c70c29fd239c0595ab95eR24-R35