Cannot use smart_open.s3.iter_bucket because of missing boto dependancy.

[roamdam]

Problem description

It seems that 1.11 causes import problems with boto. boto does not seem to appear as a dependancy for the package anymore, while it is still imported in smart_open.s3. That does not throw an error when installing but does when I try to use smart_open.s3.iter_bucket.

Steps/code to reproduce the problem

I've made the following to be sure, on a brand new virtual environment

mkdir test_repo && cd test_repo
virtualenv venv
pip3 install smart-open
python3 test-import.py

With test-import.py being :

from smart_open.s3 import iter_bucket

iter_bucket("some bucket")

Here is the result from python3 test-import.py :

$ python3 test-import.py 
unable to import 'smart_open.gcs', disabling that module
unable to import 'smart_open.s3', disabling that module
Traceback (most recent call last):
  File "test-import.py", line 1, in <module>
    from smart_open.s3 import iter_bucket
  File "/Users/**/test-repo/venv/lib/python3.6/site-packages/smart_open/__init__.py", line 29, in <module>
    from .s3 import iter_bucket as s3_iter_bucket
  File "/Users/**/test-repo/venv/lib/python3.6/site-packages/smart_open/s3.py", line 15, in <module>
    import boto
ModuleNotFoundError: No module named 'boto'

Versions

Please provide the output of:

import platform, sys, smart_open
print(platform.platform())
print("Python", sys.version)
print("smart_open", smart_open.__version__)

The above import of smart_open does not work either, but I'm using version 1.11.0

Darwin-18.7.0-x86_64-i386-64bit
Python 3.6.7 (v3.6.7:6ec5cf24b7, Oct 20 2018, 03:02:14) 
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.57)]
smart_open 1.11.0

Checklist

Before you create the issue, please make sure you have:

• Described the problem clearly
• Provided a minimal reproducible example, including any required data
• Provided the version numbers of the relevant software

[mpenkov]

Thank you for letting us know. We've released version 1.11.1 to deal with this problem. Please upgrade.

[piskvorky]

@mpenkov how come the testing system did not catch this? Did the recent refactoring break the CI?

[mpenkov]

I'm not sure how the tests missed this.

The refactoring did not break CI. The builds pass.

[piskvorky]

Scary.

[mpenkov]

I looked into the logs some more. One of the test dependencies had a dependency on boto, so it was getting installed that way.

Collecting boto>=2.36.0 (from moto[server])

  Using cached https://files.pythonhosted.org/packages/23/10/c0b78c27298029e4454a472a1919bde20cb182dab1662cec7f2ca1dcc523/boto-2.49.0-py2.py3-none-any.whl