-
Notifications
You must be signed in to change notification settings - Fork 8
VirtualBox list #15
Comments
@satra - where are the instruction for the fsl&heudiconv container? I was thinking that building an image might be nice (and not only creating a Dockerfile), but I can "cheat" and use existing layers. |
@djarecka - here you go:
|
I would add one Docker image to compare and use in the container lesson, e.g. the second image, so the neurodocker command is:
And one T1w image would be great, e.g. |
i will be cutting a new neurodocker release this week after i add more examples. by the way, in general i recommend running neurodocker with docker without |
another minor point, pre-compiled reprozip wheels can be installed with |
@kaczmarj - it did not work day before yesterday when i tried. pip complained about compiling which is why a bunch of those additional dependencies were added. @mjtravers, @yarikoptic - any chance you can take a look at this issue today? it would be good to cut a VM today or tomorrow if possible to have people play with it before we ask students to download. |
I am running a VM build now that incorporates the section 1, 3, and 4 instructions above and from Al. |
I have posted an updated VM: There are 2 conda environments set up named: "section1" and "section4":
For section3, the kaczmarj/neurodocker:master image has been pulled into Docker and the following files are in the home directory:
|
Thanks @mjtravers. I could I misunderstood @satra, but I thought that we include singularity/docker images inside, so people don't spend time to build them |
@mjtravers - I'll wait for the next version of VM and will test the new conda environments |
The next version will be out later tonight. I discovered a couple of issues after I ran the build. The build is also taking a bit longer. I will message when ready |
The updated VM is now available. I believe this version has everything for sections 1, 3, and 4. Download: https://training.repronim.org/repronim-training.ova |
@mjtravers - that seems large for what it contains. i'll see if i can download and check it out. for conda are you clearing out the environments post install? also for any apt-get are you using eatmydata or some such? also is the vagrant file somewhere? it may be slightly easier for me to build it than download it on my flight :) |
@satra No, I'm not doing any slimming of the file so I am sure there are a few GBs we could shave off. |
don't know if that helps, but just checked the size of the environments:
so together around 1.7 |
as soon as @jbpoline shares the notebooks, we can bring down the size of section 4. i'm sure all he needs is let me know when the packer file is available. i'll try to build it on our cluster remotely. it's going to take the rest of my flight to download that ova! |
you can also save about 200 MB by installing jupyter-notebook as the |
I've tested the section1 by running:
and I got:
So I didn't get any error, but I'm not sure if these is a proper output, should be "subject IDs, age of each subject, and the assessment ID" (the age looks pretty low to me for reading skills, but didn't read anything about the experiment) For the section 4, I only tested a few things: importing pandas, numpy, opening jupyter notebook and lab. It seems to be working fine now. |
I have a pull request in containing the VM build scripts. I will take a look at those tutorials later today and see how much we can slim down the file. There is a cleanup.sh script in place to add size-reducing code. Now that the code is out there, feel free to edit. |
@djarecka: My last pull request for PyNIDM had some changes to BIDSMRI2NIDM but I didn't explicitly copy the tool from it's development location ([https://github.com/incf-nidash/PyNIDM/tree/master/nidm/experiment/tools]) to the bin folder so those copies may be out of date. I'll take a look... |
@dbkeator - so the output should be different? |
@djarecka @mjtravers
In the OVA, first, I received an error that PyNIDM wasn't installed. So I issued the following command: Then I received an error that pybids wasn't installed. So, I issued the following command: Then, I received an error that urllib.parse doesn't have a module named quote. This was a curious error because urllib comes with python....which led me to the biggest problem: We have installed python2.7 via miniconda. PyNIDM was written for python 3.x and thus the current problem with the urllib package but likely many other downstream problems. So, I couldn't test the things Dorota did. @djarecka How did you test section1 with the current OVA file given the python 2.7 install? Thanks! |
@dbkeator - I did everything using the conda environment created by Matt, so first thing I did was That is the environment that is purely for this part, so should have everything, and if not @mjtravers should know. |
@jgrethe - thank you for the confirmation! |
@djarecka Got it, that worked. Appears the query is also working. I didn't realize the ages were funky.... |
BTW one personally anything issue for me is that shortcuts on Win+number configured as shortcuts to various heavy applications such as libreoffice |
i'm having some trouble importing the ova on our older virtualbox on our cluster. can someone verify the md5sum below?
(i can't test on osx as it will take me 4 days to download under my current connection) |
@djarecka verified that the above md5sum is correct. |
and in case others have the same import appliance issue on some flavor of linux/virtualbox combo, this link helps solve it: http://installfights.blogspot.com/2018/05/how-to-fix-virtualbox-error-when-you.html |
@satra @mjtravers @kaczmarj |
@jbpoline - i'm not really R-user, so I might be doing something wrong, but I tried to open the script |
@djarecka you are right : I think we need the R libraries @cmtgreenwood do you confirm ? I suppose we can always extract the R code from the Rmd files which should run on the VM - but we do need mvtnorm and ggplot2, right ? |
@jbpoline - my understanding was that r-studio (which is installed) can open Rmd and can run the specific script cells. This is what I tried and that's how I got the package errors. |
hum - I am no R person but it looks like we need these R "libraries" (equivalent of python packages) |
@jbpoline yes, i only tried to say that we don't need to "extract the R code from Rmd". |
@djarecka @jbpoline I am able to load those R packages onto the system and have them added to the VM build process. Kinda sure I am loading them right. I have a build going right now that will include the above R packages in r-base... plus the section 3 stuff for Yarik. The build will be done and posted for download later this evening. |
@satra .... plus I added the clean up for apt and conda referred to above. Will see if it reduces the VM size any with the in-progress build |
@mjtravers - thank you - let's see what this does. it would be nice if packer did some kind of pre-post step assessment of size. may give us an indication of where things are piling up. my internal calculations indicate this VM should not exceed much more than 5-6G as an ova. my wifi here is quite insufficient, so i'm trying to figure out how to run things remotely on our cluster. |
@satra Success on compacting the OVA. Your calculations are correct, the size of the file is 5.25GB. I have posted it to the training website. Note the new name (resulting from needing to clone the original OVA file as part of the compaction process): https://training.repronim.org/reprotraining.ova This file has the R libraries and section 2 python packages included. This file does not have the section 3 edits posted this morning. I have pulled those changes and they'll go in the next build. |
Thank @mjtravers ! This image is still not expected to have |
@djarecka Datalad is installed in conda env section2 version 0.10.0-rc5 |
@jbpoline @cmtgreenwood |
@mjtravers @djarecka @satra - i released neurodocker version 0.4.0.
|
yes knitr and markdown packages are needed to assemble a nice report. So I will fix the path information probably tomorrow, and upload another version. I will create plain text versions (i.e. *.R files) at the same time that do not need knitr and markdown |
@cmtgreenwood Celia: I moved your scripts into section4/section41 and my notebook in section4/section42 |
@jbpoline @cmtgreenwood
... and R-Studio |
@mjtravers Awesome - thanks ! |
managed to launch Rstudio successfully inside the updated VM. Looking for where the scripts are located in the VM. Will be back on this only tomorrow morning |
@cmtgreenwood for the scripts you have to clone this github repository. They are not part of the VM (but during training it will be cloned during git part) after you clone the repository, your scripts are in |
Good news and bad news:
However, knitr fails. It requested updates of two packages (catools, bitops) and apparently installed them OK, but then I received this message:Error in yaml::yaml.load(string, ...) :
|
Ok - so - room for improvement but in the worse case we should survive :) looks like a knitr "front" file is missing or something ... try re-installing knitr in the VM may be ? |
@cmtgreenwood If you send me the script that is having the issue, I can see if I can sort out a fix. |
@mjtravers - replacing issue #12 - just check off all the things you already have.
Core VM
general (see each section for additional local installs (listed under install)
FAIR Data - BIDS datasets
Computational basis
Neuroimaging Workflows
docker pull kaczmarj/neurodocker:master
) - will ask @kaczmarj to cut a new releaseStatistics for reproducibility
Others
The text was updated successfully, but these errors were encountered: