bootstrap.py build error

[nextgenusfs]

I keep getting the following error when trying to build locally with bootstrap. The circleci build worked locally but then build failed after I tried to merge. I don't know how to troubleshoot this. I thought could be a python 2/3 issue -- but haven't made any progress on identifying that has error. My base conda installation is py2, but I installed/activated the bootstrap build from a py3.6 environment.

Thanks,
Jon

$ bioconda-utils build recipes config.yml --docker --mulled-test --git-range master
....
21:59:41 BIOCONDA INFO Building and testing subdag 1 of 1 (35 recipes)
21:59:46 BIOCONDA INFO Determining expected packages
Setting build platform. This is only useful when pretending to be on another platform, such as for rendering necessary dependencies on a non-native platform. I trust that you know what you're doing.
No numpy version specified in conda_build_config.yaml.  Falling back to default numpy value of 1.11
Adding in variants from internal_defaults
Adding in variants from /tmp/miniconda/miniconda/conda_build_config.yaml
Adding in variants from /tmp/miniconda/miniconda/lib/python3.6/site-packages/bioconda_utils/bioconda_utils-conda_build_config.yaml
21:59:46 BIOCONDA INFO Nothing to be done for recipe recipes/alfred
21:59:46 BIOCONDA INFO Determining expected packages
Attempting to finalize metadata for amptk
Solving environment: ...working... done
Solving environment: ...working... done
21:59:52 BIOCONDA INFO BUILD START recipes/amptk
22:00:34 BIOCONDA ERROR COMMAND FAILED: docker run -t --net host --rm -v /private/var/folders/hl/nt3kbhd97bd861ccjmd17hmr0000gn/T/tmpr2k9oko0/build_script.bash:/opt/build_script.bash -v /tmp/miniconda/miniconda/conda-bld:/opt/host-conda-bld -v /Users/jon/software/bioconda-recipes/recipes/amptk:/opt/recipe -e CONDA_SHLVL=2 -e CONDA_PROMPT_MODIFIER=(base)  -e CONDA_EXE=/Users/jon/miniconda2/bin/conda -e CONDA_PREFIX_1=/Users/jon/miniconda2/envs/py36 -e LANG=en_US.UTF-8 -e CONDA_PYTHON_EXE=/Users/jon/miniconda2/bin/python -e CONDA_DEFAULT_ENV=base -e CONDA_ROOT=/tmp/miniconda/miniconda -e CONDA_ALLOW_SOFTLINKS=false -e HOST_USER_ID=501 tmp-bioconda-builder /bin/bash /opt/build_script.bash
22:00:34 BIOCONDA ERROR STDOUT+STDERR:

# >>>>>>>>>>>>>>>>>>>>>> ERROR REPORT <<<<<<<<<<<<<<<<<<<<<<

    Traceback (most recent call last):
      File "/opt/conda/lib/python3.6/site-packages/conda/cli/main.py", line 98, in main
        return activator_main()
      File "/opt/conda/lib/python3.6/site-packages/conda/activate.py", line 632, in main
        print(activator.execute(), end='')
      File "/opt/conda/lib/python3.6/site-packages/conda/activate.py", line 166, in execute
        return getattr(self, self.command)()
      File "/opt/conda/lib/python3.6/site-packages/conda/activate.py", line 152, in activate
        return self._finalize(self._yield_commands(self.build_activate(self.env_name_or_prefix)),
      File "/opt/conda/lib/python3.6/site-packages/conda/activate.py", line 272, in build_activate
        deactivate_scripts = self._get_deactivate_scripts(old_conda_prefix)
      File "/opt/conda/lib/python3.6/site-packages/conda/activate.py", line 518, in _get_deactivate_scripts
        prefix, 'etc', 'conda', 'deactivate.d', '*' + self.script_extension
      File "/opt/conda/lib/python3.6/posixpath.py", line 80, in join
        a = os.fspath(a)
    TypeError: expected str, bytes or os.PathLike object, not NoneType

`$ /opt/conda/bin/conda shell.posix activate`

  environment variables:
                 CIO_TEST=<not set>
    CONDA_ALLOW_SOFTLINKS=false
        CONDA_DEFAULT_ENV=base
                CONDA_EXE=/Users/jon/miniconda2/bin/conda
           CONDA_PREFIX_1=/Users/jon/miniconda2/envs/py36
    CONDA_PROMPT_MODIFIER=(base)
         CONDA_PYTHON_EXE=/Users/jon/miniconda2/bin/python
               CONDA_ROOT=/tmp/miniconda/miniconda
              CONDA_SHLVL=2
                 INFOPATH=/opt/rh/devtoolset-2/root/usr/share/info
          LD_LIBRARY_PATH=/opt/rh/devtoolset-2/root/usr/lib64:/opt/rh/devtoolset-2/root/usr/lib
                  MANPATH=/opt/rh/devtoolset-2/root/usr/share/man:
                     PATH=/opt/rh/devtoolset-2/root/usr/bin:/usr/local/sbin:/usr/local/bin:/usr/
                          sbin:/usr/bin:/sbin:/bin:/root/bin
               PYTHONPATH=/opt/rh/devtoolset-2/root/usr/lib64/python2.6/site-
                          packages:/opt/rh/devtoolset-2/root/usr/lib/python2.6/site-packages
       REQUESTS_CA_BUNDLE=<not set>
            SSL_CERT_FILE=<not set>

     active environment : None
            shell level : 2
       user config file : /root/.condarc
 populated config files : /root/.condarc
          conda version : 4.5.11
    conda-build version : 3.15.1
         python version : 3.6.6.final.0
       base environment : /opt/conda  (writable)
           channel URLs : https://conda.anaconda.org/conda-forge/linux-64
                          https://conda.anaconda.org/conda-forge/noarch
                          https://conda.anaconda.org/bioconda/linux-64
                          https://conda.anaconda.org/bioconda/noarch
                          https://repo.anaconda.com/pkgs/main/linux-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/free/linux-64
                          https://repo.anaconda.com/pkgs/free/noarch
                          https://repo.anaconda.com/pkgs/r/linux-64
                          https://repo.anaconda.com/pkgs/r/noarch
                          https://repo.anaconda.com/pkgs/pro/linux-64
                          https://repo.anaconda.com/pkgs/pro/noarch
          package cache : /opt/conda/pkgs
                          /root/.conda/pkgs
       envs directories : /opt/conda/envs
                          /root/.conda/envs
               platform : linux-64
             user-agent : conda/4.5.11 requests/2.18.4 CPython/3.6.6 Linux/4.9.93-linuxkit-aufs centos/6.10 glibc/2.12
                UID:GID : 0:0
             netrc file : None
           offline mode : False


An unexpected error has occurred. Conda has prepared the above report.
If submitted, this report will be used by core maintainers to improve
future releases of conda.
Would you like conda to send this report to the core maintainers?
[y/N]: 
No report sent. To permanently opt-out, use

    $ conda config --set report_errors false



22:00:34 BIOCONDA ERROR BUILD FAILED recipes/amptk

[dpryan79]

I don't get this particular error any more. The primary issue with this package is that it's not being installed as a normal python package, so you'll inevitably run into issues where import amptk can't work, due to the module itself not being properly installed (c.f. #13164 ). My suggestion is to restructure amptk as a proper python package, with a setup.py.

[nextgenusfs]

Thanks @dpryan79. I've found some time to restructure the code and have bumped to v1.3.0 as had to change quite a few things to get it properly packaged. It now installs locally with pip and import amptk functions, etc. But of course different problems when I tried to test the recipe, following the bootstrap.py directions I get an import error which seems related to pandas when I run bioconda-utils.

$ ./bootstrap.py /tmp/miniconda
.
.
.
$ source ~/.config/bioconda/activate
(base) $ bioconda-utils lint recipes config.yml --git-range master
Traceback (most recent call last):
  File "/tmp/miniconda/miniconda/bin/bioconda-utils", line 6, in <module>
    from bioconda_utils.cli import main
  File "/tmp/miniconda/miniconda/lib/python3.6/site-packages/bioconda_utils/cli.py", line 13, in <module>
    import pandas
  File "/tmp/miniconda/miniconda/lib/python3.6/site-packages/pandas/__init__.py", line 19, in <module>
    "Missing required dependencies {0}".format(missing_dependencies))
ImportError: Missing required dependencies ['numpy']

[dpryan79]

Please run the following from that environment:

conda install -y git pip --file https://raw.githubusercontent.com/bioconda/bioconda-utils/0.15.7/bioconda_utils/bioconda_utils-requirements.txt
pip install git+https://github.com/bioconda/[email protected]

I'd have to check what bootstrap.py is doing, it might need to be updated.

Edit: I took a look at bootstrap.py and it should be doing all of that. I'm not sure why numpy is missing then.

[nextgenusfs]

Not sure what is going on. I tried again with a fresh instance and got a little bit further, but then ended up again with the first error. Obviously something is conflicting on my system. I was able to run the circleci build and that passed locally so I pushed a new PR for the release on PyPi. #13553.