-
Notifications
You must be signed in to change notification settings - Fork 701
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Adding updated easyconfigs for R 3.3 foss and intel 2016a, along with… #3079
Conversation
… required patches for icc/icpc
@bmcgough Thanks a lot, looking forward to your many contributions in the future! The PR looks good at first sight, especially since you went through the trouble of updating the versions of the R packages too, fantastic! The first thing you should try to do is make Travis (which runs the unit tests on the whole easyconfigs collection) happy.
So, basically, you're missing easyconfigs for GDAL 2.1.0 in this PR. A next thing you could do is submit a test report, if you're willing to look into that.
I'll take a closer look now at things, and may include some inline remarks/questions too. |
|
||
preconfigopts = 'BLAS_LIBS="$LIBBLAS" LAPACK_LIBS="$LIBLAPACK"' | ||
configopts = "--with-lapack --with-blas --with-pic --enable-threads --with-x=no --enable-R-shlib --with-tcltk " | ||
configopts += '--with-tcl-config="$EBROOTTCL"/lib/tclConfig.sh --with-tk-config="$EBROOTTK"/lib/tkConfig.sh ' |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this is new, we don't seem to have this in other R easyconfigs
why did you (have to) add it?
is it related to #1863 at all?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
(Assuming you mean the TCL/TK configopts?)
The CRAN package tkrplot couldn't find the config files until I added this.
I am a bit confused about TCL/TK in R - is it even worth configuring with -noX? What does Tk do without X?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@boegel I also do the same for Tcl/Tk. See https://github.com/hpcugent/easybuild-easyconfigs/pull/3069/files#diff-b782c64de427c0b1534a81b8f2bd6789R21
For me enabling X is mandatory. Bioinformaticians need it to plot images
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actually, the -no-X11
is kind of a misnomer, see also #2261...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For me enabling X is mandatory. Bioinformaticians need it to plot images
+1
More broadly, the more that get enabled here the better:
> capabilities()
jpeg png tiff tcltk X11 aqua
TRUE FALSE TRUE TRUE FALSE FALSE
http/ftp sockets libxml fifo cledit iconv
TRUE TRUE TRUE TRUE TRUE TRUE
NLS profmem cairo ICU long.double libcurl
TRUE FALSE TRUE TRUE TRUE TRUE
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The default (non-versionsuffixed) R easyconfigs should be as fat as possible, probably, but that may mean pulling in a whole X11 stack if we want to be independent of the OS, cfr. #2624
I am just now working with travis-ci on another repo, so once I have that figured out, I will work toward getting the testing working. We have a script that I will add to the framework repo shortly called ezupdate.py (should we change the name?). It works for R and Python packages and uses the CRAN/PYPI APIs to take an existing package list, update each package to the newest version, and include any new dependent packages. We've tinkered with it for a while, and it seems to work pretty well at this point. The fiddly stuff comes from parsing the EB, which I assume we would not have to do if we integrated into EB fully. So I used that script to update the packages, but forgot that it would not update dependencies, which is why java was not updated. Also, probably why openssl was pulled in w/o me knowing it. It is our goal to keep R and Python (and maybe Perl in the future) up to date on every release - within a few days of release. We have that goal in-house here, and by using EB, we hope to benefit others with that work. I have the GDAL packages and will include them - I guess with a new PR that I will reference here? Also, I have R-bundle-Bioconductor-3.3 easyconfigs to commit, but I thought I would wait until this PR is resolved. |
@bmcgough maybe you are interested in this. I create all my R easyconfigs using it https://github.com/fgeorgatos/easybuild.experimental/tree/master/users/pneerincx The procedure I follow to update my R installation to a newer version is: Load current R module. In current R version run this to fetch list of installed libs:
I install a easyconfig for the new R version without any R library (the bare version) to my home folder. Load new R module. In new R version in my $HOME I run this to clone the libraries and upgrade to latest versions of the libs:
If I need any extra R library I use biocLite() to install it and then I use the "EasyConfig generator for R" to generate the easyconfig |
@bmcgough Since the number of easyconfigs is likely going to remain limited (and easy to review via That being said, either way is fine by me. A simple PR with newer versions of GDAL should b easy to get merged, and we can make Travis retest this PR on top of that quite easily. Contributing back the update script would be very useful. I'll leave the renaming up to you (maybe refer to 'extensions' in some way), but we can also look into integrating it into the EasyBuild framework itself (I can help out there, since this sort of functionality is very relevant for us too). Thanks a lot for engaging yourself in keeping the easyconfigs with extensions up to date, that would be very valuable to a lot of people indeed! We (at HPC-UGent) tend to do it mostly by user request. The Bioconductor easyconfig should definitely be a separate PR; feel free to already open a PR for that (that way it can already go through review and maybe even testing), just mention in the description that it depends on this PR by mentioning the PR ID. |
('libjpeg-turbo', '1.4.2'), # for plottting in R | ||
('Java', '1.8.0_72', '', True), # Java bindings are built if Java is found, might as well provide it | ||
('Tcl', '8.6.4'), # for tcltk | ||
('Tk', '8.6.4', '-no-X11'), # for tcltk |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
there's a Tcl/Tk 8.6.5, do we want to jump to that too?
Ah, such fun. cURL 7.47.0 is used by netCDF-4.4.0. There is no new version of netCDF, so if I move to cURL-7.49.0 I'll have to use a versionsuffix. Is it worth upgrading cURL? |
@bmcgough in that case, stick to cURL 7.47.0, it's probably not worth the trouble |
@bmcgough - the update script would indeed be very useful. |
…sPer to GDAL as it was pulling from OS and those easyconfigs already existed
@bmcgough ping on both this PR and the script for updating extension versions? |
One of my colleagues will submit the extensions update script. It is being re-worked a bit to interface with pipy via XML. As for the R easyconfigs, I am lost in dependency hell trying to get support enabled for cairo and the rest of the graphics options. I think I am near done, but am finding lots of libraries pulled in from our build host's OS (not a clean build host I'm sorry to say). I was going to try running travis on our fork of the easyconfigs repo in a minute to see if it is able to build these R packages. Any advice on doing that? I see there is a travis.yml file in the repo, but I imagine you have some config in the travis UI (ENV variables)? |
@bmcgough Travis will not actually build the easyconfigs, it will just run the unit tests, basically meaning parsing easyconfig files, checking whether all dependencies & patches are there, checking whether there are no version conflicts between dependencies, etc. I'm happy to give it a full build test on my end and submit a test report, but then you'll need to update the PR to have all required easyconfigs... |
@bmcgough oh boy, this is getting rather big... :) |
Sorry, didn't intend to push all the way up. These are testing in travis now in our fork. I have been working on R-3.3.0-foss-2016a-bare.eb so I was unprepared for the full packages and intel version dependencies, thus the additional pushes. Sorry. |
Test report by @boegel |
just one little remark - if we update X component frequently (and not only by toolchain version), then we will end up infinite amount of X packages. This is why we have to solve the issue what I started in #2579 (to make one install for X headers, and X libs (not yet in PR) |
@bmcgough seems like you've introduced conflicts with the current |
I'm not sure what I can do to help with the X headers/libs issue - if there is something specific I should do in this PR, let me know. As for getting out of sync with develop, I can find libXft-2.3.2-intel-2016a-fontconfig-2.11.95.eb and netCDF-4.4.0-foss-2016a.eb, but I'm not sure I'm looking for the diffs correctly. I should have the dependencies worked out for R-3.3 in my next push, which will be within the next 24 hours. |
@bmcgough did you perhaps make any progress with this? I am also looking into the installation of R 3.3 (3.3.1, released last week) and it would be nice if I could somehow (re)use your easyconfigs. |
@bmcgough ping on this? I'm very keen on including easyconfigs for the latest R 3.3.x with the updated |
(close/open to trigger Travis) |
ah, no, this needs a sync with @bmcgough see FredHutch#2 |
I updated all extensions in my easyconfig for R 3.3.1 to the latest version, see: Update: |
#3314 was merged with easyconfigs for R 3.3.1 with updates dependencies and extensions, for both |
@bmcgough do you think it's still useful to try and get this merged? |
sync with develop & resolve conflict
@bmcgough, this PR is being closed for the following reason(s): no activity for > 6 months, obsoleted by more recent PRs. |
… required patches for icc/icpc
This is my first (of hopefully many) easyconfig PRs. Please let me know if I've done this in the best way, and how I can improve future PRs.