arrow-cpp 7.0.0 is incompatible with tensorflow-gpu (fails with requires libprotobuf >=3.19.4,<3.20.0a0)

[holzman]

Solution to issue cannot be found in the documentation.

• I checked the documentation.

Issue

arrow-cpp 7.0.0 is not compatible with tensorflow-gpu (arrow-cpp 6.0.1 works fine).

To reproduce:

FROM centos:centos7
ARG MAMBAFORGE=https://github.com/conda-forge/miniforge/releases/download/4.11.0-4/Mambaforge-Linux-x86_64.sh
RUN curl -sSL $MAMBAFORGE -o /tmp/mambaforge.sh && bash /tmp/mambaforge.sh -bfp /usr/local
RUN mamba install --yes arrow-cpp==7.0.0 tensorflow-gpu

The error:

Encountered problems while solving:
  - package arrow-cpp-7.0.0-py39h116028d_0_cpu requires libprotobuf >=3.19.4,<3.20.0a0, but none of the providers can be installed

I have another configuration (that I haven't simplified, so reproducer not attached) that complains similarly:

Encountered problems while solving:
  - package arrow-cpp-7.0.0-py37h1be195e_2_cuda requires libprotobuf >=3.19.4,<3.20.0a0, but none of the providers can be installed

Installed packages

_libgcc_mutex             0.1                 conda_forge    conda-forge
_openmp_mutex             4.5                       1_gnu    conda-forge
brotlipy                  0.7.0           py39h3811e60_1003    conda-forge
bzip2                     1.0.8                h7f98852_4    conda-forge
c-ares                    1.18.1               h7f98852_0    conda-forge
ca-certificates           2021.10.8            ha878542_0    conda-forge
certifi                   2021.10.8        py39hf3d152e_1    conda-forge
cffi                      1.15.0           py39h4bc2ebd_0    conda-forge
charset-normalizer        2.0.12             pyhd8ed1ab_0    conda-forge
colorama                  0.4.4              pyh9f0ad1d_0    conda-forge
conda                     4.11.0           py39hf3d152e_0    conda-forge
conda-package-handling    1.7.3            py39h3811e60_1    conda-forge
cryptography              36.0.1           py39h95dcef6_0    conda-forge
icu                       69.1                 h9c3ff4c_0    conda-forge
idna                      3.3                pyhd8ed1ab_0    conda-forge
krb5                      1.19.2               hcc1bbae_3    conda-forge
ld_impl_linux-64          2.36.1               hea4e1c9_2    conda-forge
libarchive                3.5.2                hccf745f_1    conda-forge                                                                            [54/101]libcurl                   7.81.0               h2574ce0_0    conda-forge
libedit                   3.1.20191231         he28a2e2_2    conda-forge
libev                     4.33                 h516909a_1    conda-forge
libffi                    3.4.2                h7f98852_5    conda-forge
libgcc-ng                 11.2.0              h1d223b6_12    conda-forge
libgomp                   11.2.0              h1d223b6_12    conda-forge
libiconv                  1.16                 h516909a_0    conda-forge
libmamba                  0.21.2               h3985d26_0    conda-forge
libmambapy                0.21.2           py39h8bfa403_0    conda-forge
libnghttp2                1.47.0               h727a467_0    conda-forge
libnsl                    2.0.0                h7f98852_0    conda-forge
libsolv                   0.7.19               h780b84a_5    conda-forge
libssh2                   1.10.0               ha56f1ee_2    conda-forge
libstdcxx-ng              11.2.0              he4da1e4_12    conda-forge
libuuid                   2.32.1            h7f98852_1000    conda-forge
libxml2                   2.9.12               h885dcf4_1    conda-forge
libzlib                   1.2.11            h36c2ea0_1013    conda-forge
lz4-c                     1.9.3                h9c3ff4c_1    conda-forge
lzo                       2.10              h516909a_1000    conda-forge
mamba                     0.21.2           py39hfa8f2c8_0    conda-forge
ncurses                   6.3                  h9c3ff4c_0    conda-forge
openssl                   1.1.1l               h7f98852_0    conda-forge
pip                       22.0.3             pyhd8ed1ab_0    conda-forge
pybind11-abi              4                    hd8ed1ab_3    conda-forge
pycosat                   0.6.3           py39h3811e60_1009    conda-forge
pycparser                 2.21               pyhd8ed1ab_0    conda-forge
pyopenssl                 22.0.0             pyhd8ed1ab_0    conda-forge
pysocks                   1.7.1            py39hf3d152e_4    conda-forge
python                    3.9.10          h85951f9_2_cpython    conda-forge
python_abi                3.9                      2_cp39    conda-forge
readline                  8.1                  h46c0cb4_0    conda-forge
reproc                    14.2.3               h7f98852_0    conda-forge
reproc-cpp                14.2.3               h9c3ff4c_0    conda-forge
requests                  2.27.1             pyhd8ed1ab_0    conda-forge
ruamel_yaml               0.15.80         py39h3811e60_1006    conda-forge
setuptools                60.9.3           py39hf3d152e_0    conda-forge
six                       1.16.0             pyh6c4a22f_0    conda-forge
sqlite                    3.37.0               h9cd32fc_0    conda-forge
tk                        8.6.12               h27826a3_0    conda-forge
tqdm                      4.62.3             pyhd8ed1ab_0    conda-forge
tzdata                    2021e                he74cb21_0    conda-forge
urllib3                   1.26.8             pyhd8ed1ab_1    conda-forge
wheel                     0.37.1             pyhd8ed1ab_0    conda-forge
xz                        5.2.5                h516909a_1    conda-forge
yaml                      0.2.5                h7f98852_2    conda-forge
yaml-cpp                  0.6.3                he1b5a44_4    conda-forge
zlib                      1.2.11            h36c2ea0_1013    conda-forge
zstd                      1.5.2                ha95c52a_0    conda-forge

Environment info

active environment : None
       user config file : /root/.condarc
 populated config files : /usr/local/.condarc
          conda version : 4.11.0
    conda-build version : not installed
         python version : 3.9.10.final.0
       virtual packages : __linux=3.10.0=0
                          __glibc=2.17=0
                          __unix=0=0
                          __archspec=1=x86_64
       base environment : /usr/local  (writable)
      conda av data dir : /usr/local/etc/conda
  conda av metadata url : None
           channel URLs : https://conda.anaconda.org/conda-forge/linux-64
                          https://conda.anaconda.org/conda-forge/noarch
          package cache : /usr/local/pkgs
                          /root/.conda/pkgs
       envs directories : /usr/local/envs
                          /root/.conda/envs
               platform : linux-64
             user-agent : conda/4.11.0 requests/2.27.1 CPython/3.9.10 Linux/3.10.0-1160.41.1.el7.x86_64 centos/7.9.2009 glibc/2.17
                UID:GID : 0:0
             netrc file : None
           offline mode : False

[jakirkham]

Can you please try creating a brand new environment?

[jakirkham]

Should add tensorflow was migrated in PR ( conda-forge/tensorflow-feedstock#189 ), but maybe some combination of builds were missed (or something else I'm not aware of)

[holzman]

I get the same result if I create a clean environment. Curiously, I noticed that the package order matters -- if I instead run mamba install --yes tensorflow-gpu arrow-cpp==7.0.0, it fails instead with:

Encountered problems while solving:
  - nothing provides __cuda needed by tensorflow-2.7.0-cuda102py310hcf4adbc_0

[jakirkham]

Could you please try mamba create -n <some test env> --yes tensorflow-gpu libprotobuf=3.19? If that doesn't work, could you please file a tensorflow issue with the results from that attempt?

[holzman]

Thanks for the help - this is indeed a problem with tensorflow-gpu 2.7.0 and not arrow-cpp. I'll file a bug there.

The reason I didn't see this issue with arrow-cpp 6.0.1 is that 6.0.1 requires an earlier libprotobuf, so mamba installs tensorflow-gpu 2.6.2 instead.

[jakirkham]

Of course. Thanks!

Yep that make sense.