Skip to content

Commit

Permalink
packaging environments with makeself on Unix
Browse files Browse the repository at this point in the history
  • Loading branch information
saraedum committed Jan 25, 2021
1 parent 17bf757 commit 077f5dc
Show file tree
Hide file tree
Showing 2 changed files with 114 additions and 0 deletions.
5 changes: 5 additions & 0 deletions docs/source/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,11 @@ Use Cases
<https://conda.io/docs/user-guide/tasks/manage-environments.html#sharing-an-environment>`_,
and recreate the environment when needed.

- Packaging an environment as a single executable for those users who struggle
with installers and just want to double-click somewhere. (See our
instructions for :doc:`Linux and macOS <unix-binary>` or :doc:`Windows
<windows-binary>` for details.)

- *BETA*: Packaging a conda environment as a standard Cloudera parcel. This is
a newly added capability. It has been tested on a live cluster, but different
cluster configurations may produce different results. We welcome users to
Expand Down
109 changes: 109 additions & 0 deletions docs/source/unix-binary.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,109 @@
Bundle an Environment as a Single Executable
============================================

``conda-pack`` can be used to distribute conda environments as executable shell
scripts for Linux and macOS.


Packaging a Simple Binary
-------------------------

We will package the [normaliz](https://github.com/Normaliz/Normaliz) binary in
this example. It provides a command line tool which is compiled from C++ code.

Create an environment and `conda pack` it:

.. code-block:: bash
$ conda create -y -n normaliz normaliz=3.8.5
$ conda pack -n normaliz
Add an entrypoint that activates the environment and starts normaliz:

.. code-block:: bash
$ mkdir pack
$ tar -zxf normaliz.tar.gz -C pack
$ cat > pack/entrypoint.sh <<- EOF
#!/bin/sh
source bin/activate
conda-unpack
exec bin/normaliz $@
EOF
$ chmod +x pack/entrypoint.sh
Optional: reduce the size by removing files that are not needed here:
.. code-block:: bash
$ rm -rf pack/lib/*.a pack/usr/share pack/usr/include
$ find pack/lib -name '*.dylib' -type f -exec strip -S \{\} \; # macOS
$ find pack/lib -name '*.so' -type f -exec strip --strip-unneeded \{\} \; # Linux
Pack everything into a single shell script with [makeself](https://makeself.io/):
.. code-block:: bash
$ conda install -y makeself
$ makeself pack/ normaliz.run Normaliz ./entrypoint.sh
The shell script `normaliz.run` should now work for others on the same platform. Note that arguments to `bin/normaliz` need to be given after an initial `--` since earlier arguments are consumed by makeself:
.. code-block:: bash
$ ./normaliz.run -- --version
Normaliz 3.8.5
Packaging a Complex Environment
-------------------------------
Note that complex environments can be packaged in the same way. Here we package
the computer algebra system [SageMath](https://sagemath.org) which comes with a
Jupyter notebook interface:
.. code-block:: bash
$ conda create -y -n sagemath sage=9.2
$ conda pack -n sagemath
$ mkdir pack
$ tar -zxf sagemath.tar.gz -C pack
$ cat > pack/entrypoint.sh <<- EOF
#!/bin/sh
source bin/activate
conda-unpack
exec bin/sage --notebook=jupyter $@
EOF
$ chmod +x pack/entrypoint.sh
$ makeself pack/ sagemath.run SageMath ./entrypoint.sh
$ ./sagmath.run # opens a browser with Jupyter running SageMath
The above creates a huge bundle that takes a long time to pack and unpack (and
might exceed the available space in your `/tmp`.) You can speed up the process
by reducing the level of compression and by having your users uncompress things
permanently:
.. code-block:: bash
$ cat > pack/unpack.sh <<- EOF
#!/bin/sh
source bin/activate
conda-unpack
EOF
$ chmod +x pack/unpack.sh
$ cat > pack/sagemath.run <<- EOF
dir=$(CDPATH= cd -- "$(dirname -- "$0")" && pwd)
cd "$dir"
./bin/sage --notebook=jupyter $@
EOF
$ chmod +x pack/sagemath.run
$ mkdir tmp
$ TMPDIR=tmp/ makeself --complevel 6 --target ./sagemath-9.2 pack/ sagemath.install SageMath ./unpack.sh
The resulting shell script unpacks the environment into `./sagemath-9.2`.
Users could overwrite this with the `--target` parameter:
.. code-block:: bash
$ ./sagemath.install
$ ./sagemath-9.2/sagemath.run # opens a browser with Jupyter running SageMath

0 comments on commit 077f5dc

Please sign in to comment.