bioconda-utils build discrepancy Docker vs Bioconda

[t-neumann]

Hi @bioconda/core Team,

I mentioned this already in PR #10199:

There are some discrepancies between the bioconda-utils-build-env Docker container (https://hub.docker.com/r/bioconda/bioconda-utils-build-env/) build and what you guys are doing on your side to build the conda packages, causing my package build to be corrupt, even though the local build works without issues.

This is what I do to setup the local build:

First start the Docker container:

docker run -ti bioconda/bioconda-utils-build-env /bin/bash

Then checkout the bioconda-recipes and build slamdunk locally:

git clone https://github.com/bioconda/bioconda-recipes.git
cd bioconda-recipes/
bioconda-utils build recipes config.yml --force --packages slamdunk

Then I create one environment with the local build and one with the bioconda build:

conda create -n slamdunkLocal -c local slamdunk
conda create -n slamdunkBioconda -c bioconda slamdunk

Now the local environment perfectly fine creates the symlink to the custom Nextgenmap build and Nextgenmap runs fine:

source activate slamdunkLocal
(slamdunkLocal) [root@58fdb0c78e9e bioconda-recipes]# ls -lah /opt/conda/envs/slamdunkLocal/lib/python2.7/site-packages/slamdunk/contrib/ngm
lrwxrwxrwx 1 root root 28 Aug  4 06:43 /opt/conda/envs/slamdunkLocal/lib/python2.7/site-packages/slamdunk/contrib/ngm -> nextgenmap/bin/ngm-0.5.2/ngm
source deactivate

The bioconda build however replaces this symlink with a copy apparently, causing ngm to crash with an error.

source activate slamdunkBioconda
(slamdunkBioconda) [root@58fdb0c78e9e bioconda-recipes]# ls -lah /opt/conda/envs/slamdunkBioconda/lib/python2.7/site-packages/slamdunk/contrib/ngm
-rwxrwxr-x 2 root root 466 Aug  3 13:48 /opt/conda/envs/slamdunkBioconda/lib/python2.7/site-packages/slamdunk/contrib/ngm
(slamdunkBioconda) [root@58fdb0c78e9e bioconda-recipes]# /opt/conda/envs/slamdunkBioconda/lib/python2.7/site-packages/slamdunk/contrib/ngm
/opt/conda/envs/slamdunkBioconda/lib/python2.7/site-packages/slamdunk/contrib/ngm: line 16: /opt/conda/envs/slamdunkBioconda/lib/python2.7/site-packages/slamdunk/contrib/ngm-core: No such file or directory
/opt/conda/envs/slamdunkBioconda/lib/python2.7/site-packages/slamdunk/contrib/ngm: line 16: exec: /opt/conda/envs/slamdunkBioconda/lib/python2.7/site-packages/slamdunk/contrib/ngm-core: cannot execute: No such file or directory
source deactivate

What is going on here - I thought the bioconda-utils build from bioconda-utils-build-env container is exactly what is used to build the final package?
And is there a workaround for this?

[t-neumann]

The Docker container FS running the bioconda-utils build was apparently not supporting hardlinks. Therefore the softlink was not replaced on that build system which is why ngm kept working. This was replaced by a hardlink for the actual bioconda build, causing ngm to crash.

Fixed the softlink/hardlink issue on my side and did not touch the recipe. Thanks to @bgruening for helping me pin this down!