Getting "IllegalLocationConstraintException" on 's3.create_bucket'

[EpicWink]

Getting an exception when trying to create a bucket in a mock_s3'd test in pytest. These exact tests were passing fine 3 days before this ticket was created. Even installing the same versions of packages of when the tests were passing doesn't help: the same error occurs.

In addition, if I don't have a default region set, I get "BucketAlreadyExists", but with a default region set, I get "IllegalLocationConstraintException".

This has been verified in a fresh Ubuntu Docker image.

Environment

All packages are pip-installed (latest, with pip v18.0)

• moto: 1.3.3
• pytest: 3.7.1
• boto3: 1.7.69
• botocore: 1.10.69
• more-itertools: 4.3.0
• pendulum: 2.0.3

Other packages

Jinja2-2.10 MarkupSafe-1.0 Pillow-5.2.0 PyYAML-3.13 Pygments-2.2.0 alabaster-0.7.11 asn1crypto-0.24.0 atomicwrites-1.1.5 attrs-18.1.0 aws-xray-sdk-0.95 azure-common-1.1.14 azure-nspkg-2.0.0 azure-storage-0.36.0 babel-2.6.0 boto-2.49.0 certifi-2018.4.16 cffi-1.11.5 chardet-3.0.4 colorama-0.3.9 cookies-2.2.1 coverage-4.5.1 cryptography-2.3 docker-3.4.1 docker-pycreds-0.3.0 docutils-0.14 floto-0.0.3 ibm-cos-sdk-2.1.2 ibm-cos-sdk-core-2.1.2 ibm-cos-sdk-s3transfer-2.1.2 idna-2.7 imagesize-1.0.0 jmespath-0.9.3 jsondiff-1.1.1 jsonpickle-0.9.6 mock-2.0.0 numpy-1.15.0 packaging-17.1 pbr-4.2.0 pendulum-2.0.2 pluggy-0.7.1 py-1.5.4 pyaml-17.12.1 pycparser-2.18 pyparsing-2.2.0 pytest-cov-2.5.1 pytest-timeout-1.3.1 python-dateutil-2.7.3 python-json-logger-0.1.9 pytz-2018.5 pytzdata-2018.5 requests-2.19.1 responses-0.9.0 s3transfer-0.1.13 six-1.11.0 snowballstemmer-1.2.1 sphinx-1.7.6 sphinx-rtd-theme-0.4.1 sphinxcontrib-websupport-1.1.0 structlog-18.1.0 termcolor-1.1.0 urllib3-1.23 websocket-client-0.48.0 werkzeug-0.14.1 wrapt-1.10.11 xmltodict-0.11.0

Test snippet

Was working, now failed:

import pytest
import boto3
import moto

@moto.mock_s3
class TestList:
    @pytest.fixture
    def bucket(self):
        boto3.resource("s3").create_bucket(Bucket="spambucket")
        return boto3.resource("s3").Bucket("spambucket")

    def test_list(self, bucket):
        print(list(bucket.objects.iterator))

Also now fails (don't know if it worked before):

import pytest
import boto3
import moto

class Test_list:
    @pytest.fixture
    @moto.mock_s3
    def bucket(self):
        boto3.resource("s3").create_bucket(Bucket="spambucket")
        return boto3.resource("s3").Bucket("spambucket")

    def test_list(self, bucket):
        print(list(bucket.objects.iterator))

Error traceback

Pytest output:

>       boto3.resource("s3").create_bucket(Bucket="spambucket")

tests/test_s3.py:42: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/usr/local/lib/python3.6/dist-packages/boto3/resources/factory.py:520: in do_action
    response = action(self, *args, **kwargs)
/usr/local/lib/python3.6/dist-packages/boto3/resources/action.py:83: in __call__
    response = getattr(parent.meta.client, operation_name)(**params)
/usr/local/lib/python3.6/dist-packages/botocore/client.py:314: in _api_call
    return self._make_api_call(operation_name, kwargs)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <botocore.client.S3 object at 0x7f227d004940>, operation_name = 'CreateBucket', api_params = {'Bucket': 'spambucket'}

    def _make_api_call(self, operation_name, api_params):
        operation_model = self._service_model.operation_model(operation_name)
        service_name = self._service_model.service_name
        history_recorder.record('API_CALL', {
            'service': service_name,
            'operation': operation_name,
            'params': api_params,
        })
        if operation_model.deprecated:
            logger.debug('Warning: %s.%s() is deprecated',
                         service_name, operation_name)
        request_context = {
            'client_region': self.meta.region_name,
            'client_config': self.meta.config,
            'has_streaming_input': operation_model.has_streaming_input,
            'auth_type': operation_model.auth_type,
        }
        request_dict = self._convert_to_request_dict(
            api_params, operation_model, context=request_context)
    
        handler, event_response = self.meta.events.emit_until_response(
            'before-call.{endpoint_prefix}.{operation_name}'.format(
                endpoint_prefix=self._service_model.endpoint_prefix,
                operation_name=operation_name),
            model=operation_model, params=request_dict,
            request_signer=self._request_signer, context=request_context)
    
        if event_response is not None:
            http, parsed_response = event_response
        else:
            http, parsed_response = self._endpoint.make_request(
                operation_model, request_dict)
    
        self.meta.events.emit(
            'after-call.{endpoint_prefix}.{operation_name}'.format(
                endpoint_prefix=self._service_model.endpoint_prefix,
                operation_name=operation_name),
            http_response=http, parsed=parsed_response,
            model=operation_model, context=request_context
        )
    
        if http.status_code >= 300:
            error_code = parsed_response.get("Error", {}).get("Code")
            error_class = self.exceptions.from_code(error_code)
>           raise error_class(parsed_response, operation_name)
E           botocore.exceptions.ClientError: An error occurred (IllegalLocationConstraintException) when calling the CreateBucket operation: The unspecified location constraint is incompatible for the region specific endpoint this request was sent to.

/usr/local/lib/python3.6/dist-packages/botocore/client.py:612: ClientError

[csmurton]

This is an issue with a regression in pytest 3.7.1 decorator handling rather than moto. I dropped back to pytest 3.7.0 and the exception went away; see pytest-dev/pytest#3774.

[EpicWink]

It may also be a regression with botocore as well

[monkut]

Seeing similar here on django testing without pytest. I suspect this is a botocore related issue.

[davidp94]

It seems to be related to botocore yes.

[aalvrz]

I am also experiencing this issue. Furthermore, it seems that this causes the tests to actually create the test bucket in S3.

[piercefreeman]

Was seeing this same problem until I dropped down boto3 versions.

"boto3>=1.6.16,<1.8",
"botocore>=1.9.16,<1.11",

Also covered in this PR: #1801

[spulec]

Going to close as a dupe of #1801. Let me know if that isn't the case.