Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Skip compiler activation (on Linux) #213

Merged

Conversation

jakirkham
Copy link
Member

@jakirkham jakirkham commented Jul 11, 2022

Checklist

  • Used a personal fork of the feedstock to propose changes
  • Bumped the build number (if the version is unchanged)
  • Reset the build number to 0 (if the version changed)
  • Re-rendered with the latest conda-smithy (Use the phrase @conda-forge-admin, please rerender in a comment in this PR for automated rerendering)
  • Ensured the license file is being packaged.

Fixes conda-forge/ctng-compiler-activation-feedstock#74

For packages that depend on r-base and also compile packages of their own, the inclusion and activation of compilers pulled in by r-base can conflict with those needed by the recipe as part of the package build. Particularly this comes up when the packaging being built is cross-compiled. To this end, this PR switches the compilers installed by r-base to ones that are not activated.

cc @isuruf @hcho3

@jakirkham
Copy link
Member Author

@conda-forge-admin, please lint

@conda-forge-linter
Copy link

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

@jakirkham
Copy link
Member Author

@conda-forge-admin, please restart CI

@jakirkham
Copy link
Member Author

@conda-forge-admin, please re-render

@github-actions
Copy link
Contributor

Hi! This is the friendly automated conda-forge-webservice.

I tried to rerender for you, but it looks like there was nothing to do.

This message was generated by GitHub actions workflow run https://github.com/conda-forge/r-base-feedstock/actions/runs/2652484346.

@jakirkham jakirkham force-pushed the skip_comp_activation branch 2 times, most recently from af16a0b to 54b67ad Compare July 11, 2022 21:30
This should avoid some issues encountered when cross-compiling other
downstream packages dependent on `r-base` that are doing their own
cross-compilation.
@jakirkham jakirkham force-pushed the skip_comp_activation branch from 54b67ad to 7df8719 Compare July 11, 2022 21:34
@jakirkham
Copy link
Member Author

Windows failure is perplexing

+ curl -C - -o D:/bld/r-base-split_1657575445209/work/etc/curl-ca-bundle.crt -SLO http://www.stats.ox.ac.uk/pub/Rtools/goodies/multilib/curl-ca-bundle.crt
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100   341  100   341    0     0   1333      0 --:--:-- --:--:-- --:--:--  2066

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
curl: (60) SSL certificate problem: certificate has expired
More details here: https://curl.haxx.se/docs/sslcerts.html

curl performs SSL certificate verification by default, using a "bundle"
 of Certificate Authority (CA) public keys (CA certs). If the default
 bundle file isn't adequate, you can specify an alternate file
 using the --cacert option.
If this HTTPS server uses a certificate signed by a CA represented in
 the bundle, the certificate verification probably failed due to a
 problem with the certificate (it might be expired, or the name might
 not match the domain name in the URL).
If you'd like to turn off curl's verification of the certificate, use
 the -k (or --insecure) option.

https://dev.azure.com/conda-forge/feedstock-builds/_build/results?buildId=534500&view=logs&jobId=a70f640f-cc53-5cd3-6cdc-236a1aa90802&j=a70f640f-cc53-5cd3-6cdc-236a1aa90802&t=f5d15007-a01c-5ad8-c9ce-4d519d3b275f

@hmaarrfk
Copy link
Contributor

Yes, i think that the windows opensl certificates had a bug. the m2 builds pull in a very old version of openssl where this isn't fixed.

I had to work around using git for this reason in conda-forge/x264-feedstock#28

@hcho3
Copy link

hcho3 commented Jul 19, 2022

Any updates on this one?

@@ -3,9 +3,9 @@ channel_sources:
channel_targets:
- conda-forge main
libblas:
- 3.8 *netlib
- 3.9 *netlib
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jakirkham what do you think about reverting this file, and merging? This is really to help unix users move forward. We can deal with m2- stuff later?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If that will fix the Windows CI issue, happy to try. Didn't quite follow how this is connected though

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it wont. it will just keep this file consistent with the last windows build.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i guess it isnt really important. i would merge :/ wile waiting for a fix

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah ok. Yeah that makes sense.

Netlib 3.9 may be connected to the CI issues seen here on Windows. So
revert to get CI working on Windows.
@hmaarrfk
Copy link
Contributor

I think the reverted will not help the build pass.

The actual fix, is removing the curl lines from https://github.com/jakirkham-feedstocks/r-base-feedstock/blob/skip_comp_activation/recipe/build-r-base.sh#L308=

and listing things as dependencies

@hmaarrfk
Copy link
Contributor

m2- ssl is just broken at the moment.

@hmaarrfk
Copy link
Contributor

I had to do something similar conda-forge/x264-feedstock#28

(Which I guess I'll merge when I have time to rerun crtical packages like ffmpeg in case of breakage)

@jakirkham
Copy link
Member Author

Gotcha. I don't feel confident enough to mess with that. So would rather leave that up to someone who knows what is going on there

@jakirkham
Copy link
Member Author

@conda-forge/core, any thoughts on next steps here? 🙂

@hmaarrfk
Copy link
Contributor

@conda-forge/help-r

@BEFH
Copy link

BEFH commented Jul 27, 2022

It's a major issue that R has not been updated in months since the new version. Just allow this to go through minus windows if you can't fix windows

@hmaarrfk
Copy link
Contributor

hmaarrfk commented Aug 5, 2022

@BEFH jakirkham and I are mostly trying to facilitate understanding of overarching issues that are specifics to conda-forge peculiarities. We don't always have the time to take an active role in maintaining every single repository.

As I understand it, you would benefit from this repository getting up and running again. Help in doing this would be greatly appreciated.

You should feel free forking from jakirkham's branch, and working toward addressing the curl incompatibilities with m2- that I outlined and move them to explicit source dependencies. The work on this PR is quite clean in terms of commit history so that should make it easy to get going.

This help can help other maintainers more readily accept the PR (or the new one you open).

@dpryan79
Copy link
Contributor

dpryan79 commented Aug 5, 2022

@conda-forge-admin please rerender

@github-actions
Copy link
Contributor

github-actions bot commented Aug 5, 2022

Hi! This is the friendly automated conda-forge-webservice.

I tried to rerender for you, but it looks like I wasn't able to push to the skip_comp_activation branch of jakirkham-feedstocks/r-base-feedstock. Did you check the "Allow edits from maintainers" box?

NOTE: PRs from organization accounts cannot be rerendered because of GitHub permissions.

This message was generated by GitHub actions workflow run https://github.com/conda-forge/r-base-feedstock/actions/runs/2802647228.

@BEFH
Copy link

BEFH commented Aug 5, 2022

@hmaarrfk, this is an absolutely astonishing reply. Anaconda and conda-forge are positioning anaconda as a development environment that can be used in the real world for a wide variety of coding and analytical tasks. Anaconda even has paid plans. To say "We don't always have the time to take an active role in maintaining every single repository." for a key repo that 10s of thousands of other packages rely on is also bizarre, and the thumbs-down is just petty.

To let a major, key part sit out of date for over 2 months and 2 minor versions, when the only blocker is windows is an absolutely bizarre decision. Telling users "why don't you just fix it?" is even worse. I don't develop on Windows. I have no experience with Windows build toolchains. I don't have time to learn right now.

But there are alternatives for both Python and R environments. Poor maintenance and snide comments are the kind of things that drive away users who actually want to get work done. This isn't some small hobby project by aone or two devs; it's a key part of a large ecosystem with a commercial product. If you need more assistance, ask for it from Anaconda or from other conda-forge devs. Get your act together or Anaconda is going to become an also-ran.

@izahn
Copy link

izahn commented Aug 5, 2022

@hmaarrfk, this is an absolutely astonishing reply. Anaconda and conda-forge are positioning anaconda as a development environment that can be used in the real world for a wide variety of coding and analytical tasks. Anaconda even has paid plans. To say "We don't always have the time to take an active role in maintaining every single repository."
for a key repo that 10s of thousands of other packages rely on is also bizarre, and the thumbs-down is just petty.

@BEFH hmaarrfk isn't paid by Anaconda (as far as I know) and conda-forge isn't his responsibility any more than it is yours. His comment that he can't take an active role in maintaining every single repository is perfectly reasonable. Welcome to the world of community-driven open source software!

I personally agree that the thumbs-down with no comment or elaboration wasn't very nice -- but it wasn't hmaarrfk who did that!

To let a major, key part sit out of date for over 2 months and 2 minor versions, when the only blocker is windows is an absolutely bizarre decision. Telling users "why don't you just fix it?" is even worse.

On the contrary, this is the only way. Conda-forge will only continue to exist as long as there is a pipeline of users becoming contributors.

I don't develop on Windows. I have no experience with Windows build toolchains. I don't have time to learn right now.

I feel you. Building for so many platforms and architectures is a major pain, and often puts contributors in the position of being asked to work on stuff that they don't care about and will never use. Personally I would love to abandon Windows in conda-forge entirely, but that will probably never happen.

But there are alternatives for both Python and R environments. Poor maintenance and snide comments are the kind of things that drive away users who actually want to get work done. This isn't some small hobby project by aone or two devs; it's a key part of a large ecosystem with a commercial product.

Many open-source projects have some commercial component, but the commercial and community components are often separate as they are in the case of Anaconda and conda-forge. There is some cooperation, but Anaconda hasn't committed itself to maintaining conda-forge as far as I know.

If you need more assistance, ask for it from Anaconda or from other conda-forge devs. Get your act together or Anaconda is going to become an also-ran.

This is pretty unreasonable. Why do you think any of this is hmaarrfk's responsibility?

@izahn
Copy link

izahn commented Aug 5, 2022

@BEFH jakirkham and I are mostly trying to facilitate understanding of overarching issues that are specifics to conda-forge peculiarities. We don't always have the time to take an active role in maintaining every single repository.

As I understand it, you would benefit from this repository getting up and running again. Help in doing this would be greatly appreciated.

I think @BEFH just wanted an updated r-base package for linux, as was already done over in #205.

@BEFH
Copy link

BEFH commented Aug 5, 2022

@izahn, I never originally called out @hmaarrfk. All I did was suggest that a release was done without windows support until the windows release could be fixed.

It's a major issue that R has not been updated in months since the new version. Just allow this to go through minus windows if you can't fix windows

@hmaarrfk responded with a snide missive that basically read like "if you don't like the pace of development, fix it yourself." That's why I responded how I did. I never expected either of you to quickly fix windows because there were clearly upstream issues.

I don't think it's unreasonable to prioritize fast releases on a key package that 10s of thousands of others rely on, and I don't think @hmaarrfk's response was reasonable. He could have ignored my post if he was too busy, or said that it was a priority but was taking longer than expected, or said it wasn't possible to temporarily disable one platform, or any number of responses.

If I've misread the post, I apologize. If it sounds like I'm specifically going after you or @hmaarrfk for the package being delayed, I apologize. But I don't think my OP was unreasonable, and I don't think it's unreasonable in general to expect some sort of urgency for packages like r-base to be released.

@BEFH
Copy link

BEFH commented Aug 5, 2022

As you mentioned, @izahn, there was just an updated release that worked by disabling certificate checking for Windows, and I do appreciate that.

@hmaarrfk
Copy link
Contributor

hmaarrfk commented Aug 5, 2022

Honestly, I dont really know what to say.

You asked what the holdup was. I gave a clear reason for my (in)action, and clear reasons why I interacted on specific topics (m2- limitations) that I was aware of.

It seems that the feedstock maintainers are OK skipping windows. I think that their choice to make, not mine (even as a core member).

I'm glad that got you going again.

@BEFH
Copy link

BEFH commented Aug 5, 2022

@hmaarrfk, as I said, my impatience was not with you specifically. I appreciate the work you do and was not specifically asking YOU to hurry up.

For a project this big, though, it shouldn't be down to individual maintainers to just decide they have time to fix major, key parts of the project. There should be some level of prioritization, and Anaconda should be dedicating paid devs to fix major stuff like this. It's a key part of their business model for the software to continue to be useful.

@izahn
Copy link

izahn commented Aug 5, 2022

@hmaarrfk, as I said, my impatience was not with you specifically. I appreciate the work you do and was not specifically asking YOU to hurry up.

For a project this big, though, it shouldn't be down to individual maintainers to just decide they have time to fix major, key parts of the project. There should be some level of prioritization, and Anaconda should be dedicating paid devs to fix major stuff like this. It's a key part of their business model for the software to continue to be useful.

Anaconda isn't that involved in conda-forge though, they have their own r channel with their own packages, maintained by Anaconda staff and separate from conda-forge. The official Anaconda r-base=4.2 package was released 2 months ago: https://anaconda.org/r/r-base/files . In general, the connection between Anaconda and conda-forge is much weaker than you seem to assume.

@beckermr
Copy link
Member

beckermr commented Aug 5, 2022

Hi. I am here to remind us all to follow the NumFocus code of conduct (https://numfocus.org/code-of-conduct):

Be kind to others. Do not insult or put down others. Behave professionally. Remember that harassment and sexist, racist, or exclusionary jokes are not appropriate for NumFOCUS.

All communication should be appropriate for a professional audience including people of many different backgrounds. Sexual language and imagery is not appropriate.

NumFOCUS is dedicated to providing a harassment-free community for everyone, regardless of gender, sexual orientation, gender identity and expression, disability, physical appearance, body size, race, or religion. We do not tolerate harassment of community members in any form.

Thank you for helping make this a welcoming, friendly community for all.

Please keep the discussion above civil and be respectful of the time and effort people volunteer for free to maintain conda-forge.

If this thread escalates further, we will have to take more intrusive moderation actions.

@dpryan79
Copy link
Contributor

dpryan79 commented Aug 6, 2022

On a different note, @jakirkham can you either rebase your PR on the current master or merge it into your branch? I'd like to merge this in but can't update your fork to clean up the merge conflict :)

@jakirkham
Copy link
Member Author

@conda-forge-admin, please re-render

@github-actions
Copy link
Contributor

github-actions bot commented Aug 8, 2022

Hi! This is the friendly automated conda-forge-webservice.

I tried to rerender for you, but it looks like there was nothing to do.

This message was generated by GitHub actions workflow run https://github.com/conda-forge/r-base-feedstock/actions/runs/2820705290.

@jakirkham
Copy link
Member Author

Thanks @dpryan79! 🙏 Have updated the PR to address merge conflicts. Please let me know if anything else is needed 🙂

@dpryan79
Copy link
Contributor

dpryan79 commented Aug 8, 2022

@jakirkham Much appreciated!

@dpryan79 dpryan79 merged commit 6a79d65 into conda-forge:main Aug 8, 2022
@jakirkham jakirkham deleted the skip_comp_activation branch August 8, 2022 22:03
@jakirkham
Copy link
Member Author

Great thanks Devon! 🙏

@jakirkham
Copy link
Member Author

Submitted PRs to apply the same change to 4.0.x and 4.1.x respectively. See below:

Note these are the current r-base versions used in conda-forge. Hence the need to backport this change to these versions.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Inclusion of CXXFLAGS from x86_64 & aarch64 Linux variants causes build error
8 participants