Cannot add x-amz-server-side-encryption header to S3 upload part

[jmou]

I have an S3 bucket with a policy restricting to only KMS encryption (similar to http://docs.aws.amazon.com/AmazonS3/latest/dev/UsingServerSideEncryption.html but with 'aws:kms'). I am attempting to do a multipart upload as below:

multipart = self.s3.create_multipart_upload(
    Bucket=bucket, Key=key, ServerSideEncryption='aws:kms')
part = self.s3.upload_part(Bucket=bucket, Key=key,
                           UploadId=multipart['UploadId'],
                           PartNumber=1, Body=fh)

This fails, apparently because the upload part request does not include an x-amz-server-side-encryption header.

  File "/home/jmou/src/blocks/_blocks_venv/local/lib/python2.7/site-packages/botocore/client.py", line 328, in _api_call
    raise ClientError(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (AccessDenied) when calling the UploadPart operation: Access Denied

Unlike PutObjectRequest, UploadPartRequest appears not to have a ServerSideEncryption member to set the x-amz-server-side-encryption header. This header is documented at http://docs.aws.amazon.com/AmazonS3/latest/API/mpUploadUploadPart.html (this documentation appears incomplete since aws:kms is not listed as a valid value).

Without the bucket policy, the code sample executes successfully (and the resulting object is encrypted); but this no longer enforces that all uploads use encryption.

The boto3 documentation at https://boto3.readthedocs.org/en/latest/reference/services/s3.html also states that SSECustomerAlgorithm can be set to aws:kms, but AFAICT that only sets the header for non-KMS SSE; the documentation appears to be faulty in listing aws:kms as an example value.

[jmou]

Actually, it appears that x-amz-server-side-encryption is not supported by the REST API when doing an upload part operation. Maybe this is actually a bug in the REST API?

[kyleknap]

I can confirm this issue. It looks like that we have to update our model so that you can set the x-amz-server-side-encryption headers for the UploadPart operation, which looks like a valid header when uploading parts: http://docs.aws.amazon.com/AmazonS3/latest/API/mpUploadUploadPart.html

[jmou]

Any update on this? It would be great to enforce KMS on large uploads.

[mtdowling]

aws:kms encryption is specified on the CreateMultipartUpload operation and not on each part. Amazon S3 currently supports aws:kms SSE encryption for the following operations: PutObject, CreateMultipartUpload, CopyObject, POST object (e.g., form style requests). Specifying aws:kms encryption on the UploadPart operation is invalid as it is not an accepted value.

There is an issue in that we do not support ServerSideEncryption for UploadPart. However, AES256 is the only supported option for ServerSideEncryption for the UploadPart operation: see http://docs.aws.amazon.com/AmazonS3/latest/dev/KMSUsingRESTAPI.html. I'll work on fixing this, but it won't fix you bucket policy issue.

I'm not sure it's possible based on this document (http://docs.aws.amazon.com/AmazonS3/latest/dev/using-with-s3-actions.html), but you'll need to modify your bucket policy to only enforce KMS encryption on Initiate Multipart Upload operations. As far as getting this work via a policy to enforce KMS on all multi-part uploads, I suggest starting a forum thread on the Amazon S3 developer forums so that you can interact directly with S3 engineers to see if there is a workaround or if they will consider it for a feature request: https://forums.aws.amazon.com/forum.jspa?forumID=24