cannot find module installed by pipenv

[jasonkuhrt]

❯ cat Pipfile
[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

[dev-packages]
isort = "*"
yapf = "*"
pre-commit = "*"
pytest = "*"
pylama = "*"
codecov = "*"
coverage = "*"
pytest-cov = "*"
"boto3" = "*"
rope = "*"

[requires]
python_version = "3.6"

❯ pyre check
 ƛ Found 1 type error!
source/lib/step_functions_client.py:14:0 Undefined import [21]: Could not find a module corresponding to import `boto3`.

[dark]

Sorry, but it's not clear to me how to use that Pipfile information. Could you please post a minimal source code that displays the error, along with the output of pyre --debug check? Thanks.

[jasonkuhrt]

Ok I will post more details as soon as I can.

[jasonkuhrt]

@dark here is a minimal repro:

Do the following:

> mkdir repro && cd repro && pyre init
> echo 'import boto3;import pprint;pprint.pprint(boto3)' > main.py && \
  pipenv install boto3 && \
  python main.py && \
  pyre check

Then you will see various output, but of interest:

<module 'boto3' from '/Users/jasonkuhrt/.pyenv/versions/3.6.5/lib/python3.6/site-packages/boto3/__init__.py'>

 ƛ Found 1 type error!
main.py:1:0 Undefined import [21]: Could not find a module corresponding to import `boto3`.

Python ran and found the import just fine. pyre however did not. False negative. Let me know if you need more information.

[dark]

Thanks for the info.

The main reason for which this is happening is that stubs for boto3 are not available in typeshed, hence Pyre cannot resolve that import.

In theory you could make Pyre parse the boto3 sources by providing /Users/jasonkuhrt/.pyenv/versions/3.6.5/lib/python3.6/site-packages/ to the --search-path option of pyre check, or equivalently, if that directory is already part of your PYTHONPATH, by passing --preserve-pythonpath to the pyre command.

That said, it looks like the sources of boto3 are not annotated either, so Pyre won't be able to do much. The only "real" solution would be for the authors of boto3 to publish stub files to help type checkers.

[jasonkuhrt]

It sounds like the best thing that can come out of this issue practically then is better user feedback that attempts to automate what you've said in context of different packages?

[Zebradil]

Explicit --search_path works, but if I place it inside .pyre_configuration file it doesn't work.

[dark]

@Zebradil The two configuration methods should be equivalent.

Could you please post the full command and configuration that you have tried? Adding --debug to the commandline would also help. Thanks.

[Zebradil]

Okay, I created a script for reproduce the issue, but now it's working as expected. I'm confused :-) I'll investigate further.

The script, just in case:

#!/bin/bash

rm -rf pyre-test
mkdir pyre-test
cd pyre-test
echo 'import pymysql;import pprint;pprint.pprint(pymysql)' > app.py
pipenv --three
pipenv install pymysql
pipenv run python app.py

pipenv install -d pyre-check
pipenv run pyre --search-path "$(pipenv --venv)/lib/python3.6/site-packages/"

pipenv run pyre init
cat .pyre_configuration \
    | jq --arg path "$(pipenv --venv)/lib/python3.6/site-packages/" '. + {search_path: [$path]}' \
    > .pyre_configuration
pipenv run pyre

[dark]

Regarding the original issue, we have added better wording/documentation recently.
As for the last few comments, feel free to open a new issue if the problem presents itself again.