-
Notifications
You must be signed in to change notification settings - Fork 32
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Compiling PyAT with MPI #363
Comments
@lfarv, I see that you have removed requirements.txt, why so? It is very convenient to have a file storing all dependencies required to run the code that you can use to build your venv with a single pip command |
In the new packaging system, the requirements are handled by |
What about pip install -e . , I think it calling develop, will it work in this case? |
@swhite2401: yes, it works. It's the "editable" mode, you have to build either from the source distribution (called "sdist", a tar.gz file), or from a local directory. |
btw I still think having the requirements file is useful as you can call it with pip -r, also many people use conda environments (default at ESRF) for which you need the requirements file -> conda install --file requirements.txt Why not keep it? It doesn't arm no? |
@swhite2401: the problem with this new system is that it privileges a "declarative" approach, in config files like pyproject.toml and setup.cfg, trying to remove all "executable" files (*.py files) for safety reasons: avoid potentally malicious installation procedures. |
Ok , but why remove the |
@swhite2401: Can't you use pip under Conda? I thought it was working, and it takes care of everything. Otherwise we should envisage a conda distribution for AT, but that's another story. It's not recommended to keep |
Anyway |
It was un-related to MPI, just a general question. |
@swhite2401: I see you point, but I just tried to follow as much as I could the new packaging method, which is now mature, which is more flexible, allowing different build back-ends (
It's also clearly indicated that Concerning conflicts between packages, the best solution is usually to separate different projects in different virtual environments, but if you need everything together, it may be a problem. |
Yes I understand, this was just for my culture, for now the solution is to add a .pth file in the conda environement. As you said we may explore what it would take for |
Hello @lfarv, any news on this front? We are starting to use MPI more extensively and this issue needs to be resolve. |
Hi @swhite2401 : Yes, I am again working on the updated packaging method (removing $ MPI=1 python setup.py install Things are moving slowly with |
Ah very good news! No rush and enjoy vacations! (I am also away for a couple weeks) |
Solved in #483 |
The last PyAT release uses the new python packaging method based on
pyproject.toml
. With this setup, the requirements for building the project (not installing) must be declared inpyproject.toml
. So at the moment, at attempt for building with MPI using:fails because
mpi4py
is not declared as a build dependency inpyproject.toml
. This even ifmpi4py
is already installed, because the build occurs in a virtual env. Andmpi4py
cannot be declared unconditionally, which would make it mandatory for all builds. The installation requirement can be declared insetup.cfg
in the[options.extras_require]
section, but not the build one…So at the moment, the way to build with MPI is to use the old method:
which now triggers a deprecation warning.
I do not know how the build requirements can be adjusted depending on an environment variable, or any other selection method…
The text was updated successfully, but these errors were encountered: