Support noarch:python packages

[ProgramFan]

noarch:python packages are now widely used in channels such as plotly, pyviz and conda-forge. Yet constructor does not support them correctly. This PR addresses this issue.

The idea of this PR is to follow the noarch:python specification in https://www.anaconda.com/condas-new-noarch-packages/ and imitate conda install. install.py is modified to do the following things:

1. move site-packages/* into lib/pythonX.Y/site-packages
2. move python-scripts/* into bin
3. update prefix in the above files
4. create wrappers for entry points found in info/link.json
5. fix paths in conda meta.

Python files are not compiled at the moment but can be added. I test this PR with https://github.com/ProgramFan/anaconda-plus/ and it works as expected.

[ProgramFan]

This PR addresses #86

[ankostis]

On Windows it seems it doesn't work at all, correct?

[ProgramFan]

Have not tested it on windows. But why does it not work?

[ankostis]

It's as if nothing has run from your new actions.

[ProgramFan]

This is strange. At least the files shall be copied or moved to python specific paths. These code shall be portable among OSes. The entry points can be problematic.

[ankostis]

It's strange or the trunk constructor has stopped working on Windows or i'm doing something completely wrong:

I cloned the conda/constructor repo and installed it with pip install -e <repo>, and overwritten exiting install.py with your PR (bc i couldn't see your fork).

When i build and execute the installer, the expanded folder is missing completely all traces of the noarch packages - it's as if the move worked only for removing the files, but not actually relocating them. Ofcourse it is not possible to import the noarch libraries from python repl.

[ankostis]

(assuming that i did manage to do the tests cleanly)
the un-patched master here(16b5021), on Windows it produces installations where the noarch packages are installed(!) in the proper Lib\site-packages folder.
It's just the entry-points missing.

With your patch, nothing works.

[ProgramFan]

I see. The problem here is that the site-package path on Windows is <prefix>/Lib/site-packages but I hard coded it as <prefix>/lib/pythonX.Y/site-packages. You can find these files on the latter path.

[ProgramFan]

Where shall the console scripts be placed on windows? Can these scripts be python scripts?

[jakirkham]

IIUC they are placed in <prefix>/Scripts/.

[ProgramFan]

Addressed in the latest commits. Would you please test it again? @ankostis

[ankostis]

I just tested again under Windows, unfortunately, no improvement: neither entrypoints nor the python-projects exist ANYWHERE in the constructed folder. I mean, there is no "rogue" site-packages folder, and neither existed before, with b79cfa2

[ProgramFan]

Maybe the windows package goes through a different code path. Help is welcome.

[ankostis]

Maybe the windows package goes through a different code path.

It seems so.

But i never undestood why constructor has to replicate the installation code in the first place,
and not use the conda install command?
Just to save a couple Mbytes the conda package comprises?

[mingwandroid]

It's chicken and egg. You need a single exe conda otherwise that duplicates the normal conda.

[ankostis]

But it's futile to chase conda internals - around the corner is conda-5 with new file-formats, and probably pyodide packages.

It wouldn't be a chicken-egg if constructor kept the adhoc OS-specific installation-routines here just to install theconda command and its dependencies in a temporary python-env, and then use that env to launch conda install pkgs/*, before removing the temporary Python.

[mingwandroid]

But it's futile to chase conda internals

Indeed, I wouldn't do that personally, I'd compile conda as a single executable and also as a normal conda-package. That's what we're exploring at present.

The other idea I had is to rewrite the core install-conda-package-from-tarball code in rust as a static exe and have both our Python conda and constructor use that instead.

But it's all just work to be done, no one disagrees that it should be supported and supported in a maintainable way.

[msarahan]

I think the intent of this PR is implemented in #263. Can this PR be closed, or does it have other value that is worth keeping?

[jakirkham]

^ @ProgramFan?

[ProgramFan]

Let's close it~

[github-actions]

Hi there, thank you for your contribution!

This pull request has been automatically locked because it has not had recent activity after being closed.

Please open a new issue or pull request if needed.

Thanks!