trying to upload empty file fails with a NotImplemented error

[rustyrazorblade]

I was trying to sync a directory to s3 with an empty file, and got this error:

A header you provided implies functionality that is not implemented

the command I used:

aws --region us-west-2 --delete s3 sync puppet s3://shift.staging.puppet
upload failed: puppet/manifests/nodes/jenkins.pp to s3://shift.staging.puppet/manifests/nodes/jenkins.pp```

[parshap]

👍 I've run into this too

[rustyrazorblade]

The same issue was present in the node.js version: aws/aws-sdk-js#15

[jamesls]

I'm trying to reproduce this, but I'm not able to. Can someone share a set of steps?

$ aws --version
aws-cli/1.2.0 Python/2.7.5 Darwin/12.5.0
# Create a new bucket
$ aws s3 mb s3://test-empty-file-sync/ --region us-west-2

# Create two files, one is empty, one is not empty
$ touch /tmp/syncme/emptyfile
$ echo notempty > /tmp/syncme/notempty

# Try to sync the local files to S3.
$ aws s3 sync /tmp/syncme/ s3://test-empty-file-sync --region us-west-2 --delete
upload: ../../../tmp/syncme/notempty to s3://test-empty-file-sync/notempty
upload: ../../../tmp/syncme/emptyfile to s3://test-empty-file-sync/emptyfile

# Verify we see two files, one with length 0.
$ aws s3 ls s3://test-empty-file-sync

Bucket: test-empty-file-sync
Prefix:

      LastWriteTime     Length Name
      -------------     ------ ----
2013-10-21 13:44:03          0 emptyfile
2013-10-21 13:44:03          9 notempty

[parshap]

I was consistently getting this error the day I posted in this issue (3 days ago), but now the exact same command and exact same file are uploading just fine! I I have not updated my awscli package or anything. Maybe something was changed in the S3 API to fix this.

Anyway, it works for me now. Thanks.

[jamesls]

Ok, I've added a set of integration tests that verify both cp/sync with empty files works as expected (#426).

I'm going to go ahead and close this now since this appears to be working. If anyone is still running into this, feel free to provide a set of repro steps and we'll investigate.

[fletchowns]

I'm still able to hit this one. Maybe it's only in Python 2.7.3? Or Content-Type issue because of the extension? edit: nope, extension doesn't matter

$ aws --version
aws-cli/1.1.2 Python/2.7.3 Linux/3.2.0-40-virtual

$ aws s3 mb s3://empty-file-issue-399-test/ --region us-west-2
make_bucket: s3://empty-file-issue-399-test/

$ mkdir /tmp/syncme
$ touch /tmp/syncme/empty.jpg
$ echo notempty > /tmp/syncme/notempty.jpg

$ aws s3 sync /tmp/syncme/ s3://empty-file-issue-399-test/ --region us-west-2
upload failed: ../../tmp/syncme/empty.jpg to s3://empty-file-issue-399-test/empty.jpg
A server error (NotImplemented) occurred: A header you provided implies functionality that is not implemented
upload: ../../tmp/syncme/notempty.jpg to s3://empty-file-issue-399-test/notempty.jpg

$ aws s3 ls s3://empty-file-issue-399-test/ --region us-west-2

Bucket: empty-file-issue-399-test
Prefix: 

      LastWriteTime     Length Name
      -------------     ------ ----
2013-10-21 21:54:53          9 notempty.jpg

Here's a snippet from the --debug output:

2013-10-21 21:58:28,285 - botocore.handlers - DEBUG - Checking for DNS compatible bucket for: https://s3-us-west-2.amazonaws.com/empty-file-issue-399-test/empty.jpg
2013-10-21 21:58:28,285 - botocore.handlers - DEBUG - URI updated to: https://empty-file-issue-399-test.s3.amazonaws.com/empty.jpg
2013-10-21 21:58:28,285 - botocore.auth - DEBUG - Calculating signature using hmacv1 auth.
2013-10-21 21:58:28,285 - botocore.auth - DEBUG - HTTP request method: PUT
2013-10-21 21:58:28,286 - botocore.auth - DEBUG - StringToSign:
PUT

image/jpeg
Mon, 21 Oct 2013 21:58:28 GMT
/empty-file-issue-399-test/empty.jpg
2013-10-21 21:58:28,286 - botocore.endpoint - DEBUG - Sending http request: <PreparedRequest [PUT]>
2013-10-21 21:58:28,319 - botocore.awsrequest - DEBUG - Redirect received, rewinding stream: <open file u'/tmp/syncme/empty.jpg', mode 'rb' at 0x1cc7270>
2013-10-21 21:58:28,350 - botocore.response - DEBUG - Response Body:
<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>NotImplemented</Code><Message>A header you provided implies functionality that is not implemented</Message><Header>Transfer-Encoding</Header><RequestId>97C1FA6F5B4DF7A1</RequestId><HostId>uWab8wrDBDh/UeUny98vRVx5G2vRKJppkX0JO3gZRrmVFRbnyeW8SaSb91WQ2zH2</HostId></Error>
2013-10-21 21:58:28,351 - botocore.hooks - DEBUG - Event needs-retry.s3.PutObject: calling handler <botocore.retryhandler.RetryHandler object at 0x1a84d10>
2013-10-21 21:58:28,351 - botocore.retryhandler - DEBUG - No retry needed.
2013-10-21 21:58:28,352 - botocore.hooks - DEBUG - Event after-call.s3.PutObject: calling handler <awscli.errorhandler.ErrorHandler object at 0x1a75190>
2013-10-21 21:58:28,352 - awscli.errorhandler - DEBUG - HTTP Response Code: 501
2013-10-21 21:58:28,353 - awscli.customizations.s3.tasks - DEBUG - A server error (NotImplemented) occurred: A header you provided implies functionality that is not implemented
Traceback (most recent call last):
  File "/<snip>/virtualenv/local/lib/python2.7/site-packages/awscli/customizations/s3/tasks.py", line 75, in _execute_task
    getattr(filename, filename.operation_name)()
  File "/<snip>/virtualenv/local/lib/python2.7/site-packages/awscli/customizations/s3/fileinfo.py", line 292, in upload
    response_data, http = operate(self.service, 'PutObject', params)
  File "/<snip>/virtualenv/local/lib/python2.7/site-packages/awscli/customizations/s3/utils.py", line 122, in operate
    http_response, response_data = operation.call(**kwargs)
  File "/<snip>/virtualenv/local/lib/python2.7/site-packages/botocore/operation.py", line 82, in call
    parsed=response[1])
  File "/<snip>/virtualenv/local/lib/python2.7/site-packages/botocore/session.py", line 550, in emit
    return self._events.emit(event_name, **kwargs)
  File "/<snip>/virtualenv/local/lib/python2.7/site-packages/botocore/hooks.py", line 158, in emit
    response = handler(**kwargs)
  File "/<snip>/virtualenv/local/lib/python2.7/site-packages/awscli/errorhandler.py", line 45, in __call__
    raise ServerError(msg)
ServerError: A server error (NotImplemented) occurred: A header you provided implies functionality that is not implemented

[jamesls]

Could you try upgrading to 1.2.0? It looks like this issue is fixed in 1.2.0.

[parshap]

For what it's worth I've been on 1.1.2 this whole time.

edit:

$ aws --version
aws-cli/1.1.2 Python/2.7.3 Linux/3.2.0-23-generic-pae

[fletchowns]

My bad, I should have tried that before replying.

I'm still able to hit it with 1.2.0, using the same commands from my previous comment.

$ aws --version
aws-cli/1.2.0 Python/2.7.3 Linux/3.2.0-54-virtual

$ aws s3 ls s3://empty-file-issue-399-test/ --region us-west-2
upload: ../tmp/syncme/notempty.jpg to s3://empty-file-issue-399-test/notempty.jpg
upload failed: ../tmp/syncme/empty.jpg to s3://empty-file-issue-399-test/empty.jpg
A server error (NotImplemented) occurred: A header you provided implies functionality that is not implemented

[jamesls]

Hmm, it looks like the only difference is that you're using 2.7.3. I'll give that a try and see if I can repro this.

[jamesls]

Tried it on 2.7.3, still not able to repro. Tried on mac/linux and not able to reproduce. Could you provide the part of the debug log that shows the params being sent? It should look like this:

2013-10-22 06:10:49,086 - botocore.operation - DEBUG - Operation:PutObject called with kwargs: {'body': <open file u'/home/ec2-user/foo/empty.jpg', mode 'rb' at 0x271af60>, 'bucket': u'test-empty-copy2', 'key': u'empty.jpg', 'content_type': 'image/jpeg'}

I'm wondering if some extra header is being added to the parameters, which might cause this issue.

[fletchowns]

Thanks for looking into this further jamesls. Here's some more of the debug output, starting with that params line:

2013-10-22 17:53:34,988 - botocore.operation - DEBUG - Operation:PutObject called with kwargs: {'body': <open file u'/tmp/syncme/empty.jpg', mode 'rb' at 0x2ae7300>, 'bucket': u'empty-file-issue-399-test', 'key': u'empty.jpg', 'content_type': 'image/jpeg'}
2013-10-22 17:53:34,988 - botocore.operation - DEBUG - Creating parameter objects for: Operation:PutObject
2013-10-22 17:53:34,990 - botocore.endpoint - DEBUG - Making request for Operation:PutObject (verify_ssl=True) with params: {'headers': {u'Content-Type': 'image/jpeg'}, 'uri_params': {u'Bucket': u'empty-file-issue-399-test', u'Key': u'empty.jpg'}, 'payload': <botocore.payload.Payload object at 0x7faf48438b10>}
2013-10-22 17:53:34,991 - botocore.endpoint - DEBUG - Building URI for rest endpoint.
2013-10-22 17:53:34,991 - botocore.endpoint - DEBUG - Templated URI path: /{Bucket}/{Key}
2013-10-22 17:53:34,991 - botocore.endpoint - DEBUG - Templated URI query_params: 
2013-10-22 17:53:34,991 - botocore.endpoint - DEBUG - Rendered path: /empty-file-issue-399-test/empty.jpg
2013-10-22 17:53:34,992 - botocore.endpoint - DEBUG - Rendered query_params: 
2013-10-22 17:53:34,992 - botocore.hooks - DEBUG - Event before-auth.s3: calling handler <function fix_s3_host at 0x2562e60>
2013-10-22 17:53:34,992 - botocore.handlers - DEBUG - Checking for DNS compatible bucket for: https://s3-us-west-2.amazonaws.com/empty-file-issue-399-test/empty.jpg
2013-10-22 17:53:34,992 - botocore.handlers - DEBUG - URI updated to: https://empty-file-issue-399-test.s3.amazonaws.com/empty.jpg
2013-10-22 17:53:34,993 - botocore.auth - DEBUG - Calculating signature using hmacv1 auth.
2013-10-22 17:53:34,993 - botocore.auth - DEBUG - HTTP request method: PUT
2013-10-22 17:53:34,993 - botocore.auth - DEBUG - StringToSign:
PUT

image/jpeg
Tue, 22 Oct 2013 17:53:34 GMT
/empty-file-issue-399-test/empty.jpg
2013-10-22 17:53:34,994 - botocore.endpoint - DEBUG - Sending http request: <PreparedRequest [PUT]>
2013-10-22 17:53:35,026 - botocore.response - DEBUG - Response Body:
<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>NotImplemented</Code><Message>A header you provided implies functionality that is not implemented</Message><Header>Transfer-Encoding</Header><RequestId>F279CC26B9F7BD2D</RequestId><HostId>pcVOpzt/GneVmbFK/ErAnQfnzT9ItFQtQnBIu1/OXDnzm2FEsdx3qGEOcQK/jTQU</HostId></Error>

[fletchowns]

I was still on botocore==0.19.0

After a pip install --upgrade botocore which brought me up to 0.21.0, I'm no longer hitting the error when syncing empty files.