-
Notifications
You must be signed in to change notification settings - Fork 168
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support for noarch conda packages? #86
Comments
If it is we should fix it. |
Sorry, I missed this near the end of the README:
In that case my question is: can this be a feature request, or is it not part of the plan? Thanks |
@mingwandroid @csoja is noarch support going to be added to constructor? |
To add a concrete example: If I add a norarch package to my constructor definition today, it builds fine but upon running it, all the noarch packages wind up in Perhaps a path prefix missing somewhere in the noarch case? |
I've been cautioning people not to use It'd be nice to get some of this fixed, contributions are welcome. |
Interesting. We're just starting to dabble in the noarch space and I, at least, was not aware there was a litany of issue. Happy to try to contribute a fix for this initial problem here, but no one should wait on me if they get to it first. |
@mingwandroid are you recommending against the use of |
I personally wouldn't use it until all of the issues are fixed. It's a bit chicken and egg though, I mean if no one uses it then no one will run into the problems and attempt to fix them. I'm not even sure what can be done about .pyc files in locked down scenarios. Everyone's use-case is different though. With my Anaconda Distro hat on I have to take a very conservative approach. |
@mingwandroid I've a strong interest to push |
This is actually an issue now that conda-forge is starting to push some pkgs with Right now I think its impossible to create a working constructor based on new pkgs from conda-forge, for example six wont work now that it has |
@danielfrg, not to my knowledge. As a team we're all a bit too busy to fix this. The fix would seem to necessarily intrude heavily into the solver code which would be probably be unpleasant for all involved. @mbargull, sure, sounds good and thanks for helping out (as usual). Same thing though, we're trying to get the next release out so are all too busy to do anything for this at present. I'll have some time to think about it in a couple of weeks, but I'd love to get your take on how to approach it. I am leaning towards taking the core installer part of conda out of conda (and Python actually) and adding low-level support that would be useful for menuinst to that. This could include conversion from e.g. .png (or .svg?) to platform specific icons. |
I just tested this on I am not sure why was this introduced in #43 if it actually doesn't work, i guess it work for noarch pkgs that are not python libs but most of them are noarch-python pkgs so it break most of the time. And it actually doesn't break, like other people reported the installer finishes but libs are impossible to import. I guess the solution would be for a flag to remove the noarch path of the channel (https://github.com/conda/constructor/blob/master/constructor/fcp.py#L181) Even better would be to actually support noarch-python pkgs but I am afraid i don't understand enough to know what the problem is. Is it the postlink script of the noarch-python pkgs? |
We've moved back to using The work for un-noarch-ifying noarch python packages is done in |
The core problem with Also these 3 bits of installation logic are not the same code as what This is why I want to extract the core package installation logic into a (fully static) standalone executable (that is bundled with the installers), then each of the 3 installer front-ends do little more than run this executable. |
Got it. Thanks for explaining that, i know there is a lot of context i dont have. Hopefully there is an easy fix for having constructor working again with conda-forge now that they have some noarch pkgs. other than manually listing everysingle version on the constructor, that is really not that bad. |
@mingwandroid One short-term fix is just to blacklist all |
I'm fine with it, we may need @ilanschnell to review and merge it though (presumably you'd need a release/tag with this change?), unless can target our new branch where I'm happy to review and merge it. |
Selfishly, this patch means that I've already figured out what I need to fix and build internally so that I can continue using constructor at my day-job. Putting my open-source contributor hat on, it'd be a shame to not release/tag with this update since, without it, anyone that is currently using constructor 1.5.5 and targeting the conda-forge channel will start producing "bad" constructor installer scripts if they unknowingly pull in a noarch python package. I'd like to suggest branching at 1.5.5, applying this patch and releasing a 1.5.6.
What's the new branch? Are you going to be cutting releases from that new branch? What's the deal with the 1.6.0 tag? I don't see that version available anywhere? |
Here is the branch: https://github.com/conda/constructor/tree/cross_conda_interface I don't know the answers to the rest of your questions, I'll try to bring them up with people who might. |
@mingwandroid any updates here? Would be lovely to get #115 in and a new version made for the folks starting to use noarch packages |
I'm on vacation, pinging @csoja. |
Oh have fun 😎 ⛰ 🏖 🍸 ! Sorry for interrupting your vacation 😞 |
No problem! Hopefully we can get constructor releases sorted out for you soon. |
Are there any bugs with conda proper that you're aware of, or is that statement more specific to constructor? |
@mbargull Same question for you. Are you aware of any implementation problems or bugs for conda specifically w.r.t. The only issue/bug that I'm currently aware of relates to these packages already installed into and environment, and the switching the python version of an environment. |
Other than constructor (which is hugely important both for AD and anyone wanting to make installers) there are issues reported/suspected/rumoured with
Hopefully we can get these all investigated and addressed if appropriate. Sorry this is a little vague. |
Just want to avoid spreading FUD about conda. Issues no. 2 and no. 3 could potentially be something going on with conda. Issues no. 1 and no. 4 are out of the conda repository's control. Are there any tickets I missed regarding issues no. 2 and no. 3 with conda? Again, just trying to avoid FUD, which the vagueness can engender. |
@nehaljwani - thanks for working on this! I can confirm the Python 3 version for MacOS X works fine and tqdm imports without error. Do you have an estimated time frame for when the changes to constructor will be pushed to this repository? I am interested in using constructor to build an installer for an environment for a SciPy 2019 tutorial (in a little over a week). Is it possible the updated constructor code will be pushed here before then? |
I am also building python distributions for my institution and we use conda-forge noarch packages. We also distribute our software using constructor and the ability to remove python from the installation tree is highly wanted. Wait to test the new constructor. BTW, if you want a temporary solution for noarch packages on Linux, you can use my patch in PR #256 by either install it using pip or through https://anaconda.org/programfan/constructor. @astrofrog |
@nehaljwani I tested The anaconda prompt is missing from this Windows installer, so I had to open a cmd window and activate the base conda environment myself with the activate script, but this is a separate problem. |
The prompt was missing because I forgot to add console_shortcut package to the installer. My bad. Thank you for testing! I'm pretty close to releasing the changes, probably by the end of this week. |
Hello folks, I have another set of preview installers available. Please test these out too, it will be a huge help in making the first release of constructor v3 as quick as possible. https://repo.anaconda.com/pkgs/misc/previews/anaconda/2019.06/ |
Works well too on Linux/x86_64 |
Works well on macOS Mojave. Only the .sh installer is tested. |
@carterbox, are you able to retest with the new installers? |
I was able to install miniconda, import tqdm, and uninstall miniconda with all of Windows installers. |
hello @nehaljwani any updates on this ? |
I have pushed a lot of changes to the v3 branch on this repo. Please test it out. Also,
|
@nehaljwani your branch works fine (at least on linux64), I could build/install/test otconda (https://github.com/openturns/otconda) without the post_install workaround. The only change was adding the required conda exe to the call:
For the record, here is the recipe patch against constructor-feedstock I used to build constructor:
|
@nehaljwani why arent we allowed to omit the DIRECTORY argument with
|
@jschueller Whoops. I might get backlash for saying this, but ... pull requests accepted 😅 |
v3 is now its using argparse instead of the conda options parser, and DIRECTORY is a positional argument so its required, and I dont how if its possible to disable that argument if '--test' is given, |
Thanks a lot for your efforts! Although I followed your instructions mentioned above, I receive the following error message during installation,: Upon clicking on Ignore, the installer resumes, however ending up with an incomplete installation (i.e. there is no python.exe whatsoever): From the installation log, it seems that the initial error stems from a file (_conda-script.py) not being found.
The installer executed and finished successfully (see parts of creation log below -> Let me know if you need more details. I just did not want to paste that many lines of log output):
What might be of interest is that there is no _conda-script.py mentioned in the creation log. Other files that are listed in the installation log are also listed in the creation log. I tested with and without elevated user rights. I appreciate any hints that might lead to a solution! |
@FelixS90 I have pushed a few more commits to v3 branch. Could you please try again? If you still face an issue, could you please share your construct.yaml? |
@nehaljwani Thanks a lot for your quick reply!
I just pulled your latest changes, but the error unfortunately remains the same.
The output from my environment's
I created it using
where
Does that help? |
@FelixS90 i think i had a similar problem (incomlete installations). Everything was fixed when i reinstalled a clean miniconda-base. |
added in #263. Please open new, more specific issues for any problems you encounter with the new code. |
https://docs.continuum.io/anaconda-repository/admin/noarch-packages includes this:
I couldn't find an issue about this here, so I wanted to check: is this still the case?
The text was updated successfully, but these errors were encountered: