[WIP] Container friends

[vsoch]

Adding Charliecloud usage examples

[vsoch]

hey @fbartusch to keep you in the loop I'm implementing this in other containers! Here is Charliecloud. Also I hadn't looked at the dag plot before - it's beautiful! Is it not too hard to make for something general?

[fbartusch]

Hi @vsoch, I just had a second to look into the changes. I didn't know about Charliescloud, but I will try it later.

Also I hadn't looked at the dag plot before - it's beautiful! Is it not too hard to make for something general?

What do you mean with 'general'? If you have a snakemake workflow it's very easy to make a DAG up to any intermediate/final workflow step.
Do you mean generating a graphical representation of an DAG from a arbitrary workflow descriptions (CWL, snakemake, ...)?

[vsoch]

Yes! To be more specific - I was referencing the dag.svg file.

[fbartusch]

I don't know if there is a general way (e.g. tool) to produce such a svg file for arbitrary workflow management systems. I know that cwltool (the reference implementation of CWL) can produce a dot file (a textual representation of a dag) that can be turned into an image.

[vsoch]

cool! hey so I finished up this PR, and I'm going to do one more container technology (rtk) but wanted to see if you would be interested in taking this workflow a bit farther - basically running it across containers and doing something like compare output or resource usage. I want to present (or write something up too?) and since it's your workflow, was hoping to get your input and then have you participate too :) What do you think?

[fbartusch]

That sounds like a good idea and it's a good possibility to learn something about other container technologies. Just hit me when you finished rkt, and I can compare outputs and resource usage for larger input files.

[vsoch]

hey @fbartusch I'm looking more into this and the list is VERY long, e.g., udocker, img, smith, habitat.sh, etc. I think we should run with what we have (note I'm presenting in just about 4 weeks) and this base is representative of the folks that would be at the meeting. I'd rather focus on this small group of HPC specific containers than try to hit every single one, especially since it might take away from time needed to do more detailed stuff on our current set. We have 5, and I think that's a decent start, and if we want to add more down the line can do this. What do you think?

[fbartusch]

Ok, I'll concentrate on this set of 5. I'm just curious, what is this for a meeting and presentation?

[vsoch]

It's for this --> https://sea.ucar.edu/conference/2018/containers because I thought it would be fun! But beyond that, I think some kind of analysis (with a writeup) would be interesting and go farther than a single talk (where likely I'll spend a majority of time just talking about SCIF).