Properly installing some dependency packages

[JILPulvino]

We'd like to install a few packages for all our users:

• nodejs with the ability to manage node versions
• gatsby
• A C and a C++ compiler as mentioned in the requirements for this R library (https://igraph.org/r/)
• igraph
• devtools for R
• Rcpp
• roxygen2
• rJava
• davidsjoberg/ggsankey

There are other R and python packages we'd like to system wide install, but happy to do that myself if you can say how. The above are ones that have just caused me headaches in the past installing on linux systems.

[JILPulvino]

additional request - is it possible to install alternative versions of R? It seems part of the issue with igraph is that a package has not been updated to version 4.x and is still reliant on version 3.6

[yuvipanda]

Hey @JILPulvino. Do you have experiences with building docker images yourselves? I think giving you control over that is the easiest thing to do long term - https://pilot.2i2c.org/en/latest/admin/howto/environment.html#bring-your-own-docker-image has some docs, but I can work closer in setting it up. This gives you full control over the environment, and you can update it without blocking on 2i2c staff in any way. It shouldn't require understanding Dockerfiles or similar if you don't want to.

If you don't want to do that yet, I can add these packages to the current base image. But medium / long term, we'll make a new image for you.

[JILPulvino]

Thank you! I don't have experience building my own. I don't mind the idea of maintaining one, but am a little intimidated about setting up initially. I'll read the docs and maybe experiment some on a toy repo and get back to you with challenges.

[JILPulvino]

@yuvipanda I've gone through much of the 2i2c documentation and better understand how I can bypass learning everything about docker. With that in mind, I created a binder folder with an environment.yml and an install.R file for now. Before testing those out and using the repo2docker tool, I'm wondering if you can point me in the direction of the type of configuration file I would need to install gatsby and node/npm?

I also did run jupyter-repo2docker <my-repo-url> and the build seemed to fail though the last output was the following: The command '/bin/sh -c TIMEFORMAT='time: %3R' bash -c 'time /tmp/install-miniforge.bash' && rm /tmp/install-miniforge.bash /tmp/environment.yml' returned a non-zero code: 1%

[yuvipanda]

So images built with repo2docker already contain node + npm. Based on my understanding from jupyterhub/repo2docker#573, you should be able to do npm install -g gatsby-cli in your postBuild file should do the trick.

[JILPulvino]

I changed from an environment.yml file to a requirements.txt file and still get the same error. This is the entire issue:
Step 31/50 : RUN TIMEFORMAT='time: %3R' bash -c 'time /tmp/install-miniforge.bash' && rm /tmp/install-miniforge.bash /tmp/environment.yml
---> Running in 5b664f39674a ++ dirname /tmp/install-miniforge.bash
+ cd /tmp
+ MINIFORGE_VERSION=4.9.2-2
+ MAMBA_VERSION=0.7.4
+ SHA256SUM=7a7bfaff87680298304a97ba69bcf92f66c810995a7155a2918b99fafb8ca1dc
+ URL=https://github.com/conda-forge/miniforge/releases/download/4.9.2-2/Mambaforge-4.9.2-2-Linux-x86_64.sh
+ INSTALLER_PATH=/tmp/miniforge-installer.sh
+ unset HOME
+ wget --quiet https://github.com/conda-forge/miniforge/releases/download/4.9.2-2/Mambaforge-4.9.2-2-Linux-x86_64.sh -O /tmp/miniforge-installer.sh
time: 24.513
+ chmod +x /tmp/miniforge-installer.sh
+ echo '7a7bfaff87680298304a97ba69bcf92f66c810995a7155a2918b99fafb8ca1dc /tmp/miniforge-installer.sh'
+ sha256sum --quiet -c -
+ bash /tmp/miniforge-installer.sh -b -p /srv/conda
PREFIX=/srv/conda
Unpacking payload ...
/lib64/ld-linux-x86-64.so.2: No such file or directory /lib64/ld-linux-x86-64.so.2: No such file or directory time: 0.258 time: 25.060 Removing intermediate container 5b664f39674a The command '/bin/sh -c TIMEFORMAT='time: %3R' bash -c 'time /tmp/install-miniforge.bash' && rm /tmp/install-miniforge.bash /tmp/environment.yml' returned a non-zero code: 1%

[yuvipanda]

Can you give me a link to your repo?

[JILPulvino]

Added you to the repo

[JILPulvino]

@yuvipanda I tried the jupyter-repo2docker on the original shared repo and ran into the same error as before. So I created a new repo (added you to it) and just copied over the environment.yml and install.r example files from the pilot-hubs repo to try and run jupyter-repo2docker. I ran into the same error.

[yuvipanda]

ok, let's unblock you by just installing these packages into our base image for now - and then we can try figure out a separate environment story. How does that sound, @JILPulvino?

@GeorgianaElena can you add the packages requested here to the base pilot hubs image? I think some of these are packages we already went through and installed for utoronto...

[JILPulvino]

Yep, that works for me. The only additional one is
devtools::install_github("davidsjoberg/ggsankey")

You mentioned that ultimately it might be best for me to just maintain a docker image. Is that still your recommendation? I've been muddling through a docker tutorial, but just wondering if it makes sense to keep going that route or to work on trying to get the docker repo I just created up and running via jupyter-repo2docker instead?

Thank you!

[GeorgianaElena]

Hey @JILPulvino 👋! I updated your top comment and checked the boxes for the pkgs that were installed into the base image. I hope that's ok. Please let me know if you spot issues with any of it.

[JILPulvino]

Thank you!

[choldgraf]

Hey all - we've got two unchecked boxes at the top. Do we still need to take action here or are we good to go?

[JILPulvino]

@choldgraf just an FYI on those two points. Based on my conversations with Yuvi, I'm looking to learn some docker which should mean that I can be responsible for ensuring that our environments have those installs. I was making some progress there, but just got delayed the last couple of weeks in putting together a proper container.

[choldgraf]

sounds good - that seems like a useful skillset to get comfortable with anyway, especially since repo2docker makes it relatively straightforward to do so!

[choldgraf]

@JILPulvino I'm going to close this issue since I don't think there's anything actionable on 2i2c's end. If you run into major issues with using your own Docker image, we can discuss in another issue or thread. That work? (feel free to re-open if you disagree and would like to keep tracking something here)

[JILPulvino]

@choldgraf the only reason I can see for leaving this open moving forward is that I previously wasn't able to get repo2docker to run - thus the pursuit of creating a docker image from scratch. Perhaps I should open an issue on the repo2docker repo instead though?

[choldgraf]

Ah that makes sense - if you run into issues maybe you can open an issue about that either in repo2docker or ask around here? It'll be easier to spot if it's explicitly scoped to repo2docker and such