Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update 2017-11-11-environments.md #48

Merged
merged 1 commit into from
Apr 1, 2017
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
64 changes: 36 additions & 28 deletions _posts/2017-11-11-environments.md
Original file line number Diff line number Diff line change
@@ -1,62 +1,75 @@
---
layout: post
title: Working with different package versions
title: Managing software versioning using Conda environments
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wouldn't stick to just conda here. For some people that would mean switching their complete python stack (and most scientists I know are averse to switching working systems). The idea behind the post is to show people how they can use newer version of MDAnalysis while still keeping their old scripts running.

---

In science projects often take years until they are complete. During that time
your the libraries you are using will be updated and most have new features you
like to use. But sometimes the upgrade of a library means that old programs and
scripts will stop working. It can also be that you need a new library version
for another project you are working on. The solution to this problem is to
install the same programs multiple times.
Research projects can often take months to years to complete, however the
precise version of software they use will often have a shorter lifetime than
this. These new versions of software will often include new features which
might be of great use, but they might also introduce changes which break
your existing work and introduce compatibility issues with other pieces
of software. So whilst for existing projects we might wish to freeze
all pieces of software installed, for newer projects we instead want to
use the most up-to-date versions, leaving us needing to install multiple
versions of multiple different pieces of software.

In this post we will try to explain how conda and Python virtual environments
can be used to precisely manage the software used across many independent
research projects.

In this post I will explain how conda and python virtual envs can be used to
manage different package versions.

# Conda Environments

[conda]() is a general package manager for scientific applications. It is mostly
used for python packages but the system can be used with any programs. The
[conda-forge] community also provides a large collection of scientific software
for python, R and perl. Conda should be your first choice to manage different
[Conda](https://conda.io/docs/index.html) is a general package manager for scientific
applications. It is mostly used for Python packages but the system can be used with
any programs. The [conda-forge] community also provides a large collection of scientific software
for Python, R and perl. Conda should be your first choice to manage different
software versions.

In this guide we will only concentrate on creating and managing environments
In this guide we will concentrate only on creating and managing environments
with conda. For more information on general installation of package please refer
to the [official documentation]().

As a preparation add our conda-channel to your configuration
Software is made available through different conda channels, which each act as a
source for different software. When attempting to install packages into a conda
environment, these channels are searched. In this post we will be using the
MDAnalysis channel which you can add to your configuration like so:

{% highlight bash %}
conda config --add channels MDAnalysis
{% endhighlight %}

For each research project, it is advised that you create a new environment so that
the software used in each project does interfere across different projects.
To create a new environment for your next project that uses MDAnalysis in version
0.15.0 run:

{% highlight bash %}
conda create -n mda-15 MDAnalysis=0.15.0 -y
conda create -n myproject MDAnalysis=0.15.0 -y
{% endhighlight %}

This only create a new environment. You still have to activate it to be able to
use it.
This has created a new software environment called `myproject` but has not affected
anything currently! To have access to all the software installed within it we
must first activate it

{% highlight bash %}
source activate mda-15
source activate myproject
{% endhighlight %}

To list your environments
To list your available environments

{% highlight bash %}
conda env list
{% endhighlight %}

A nice feature of using conda-environments is that they are easy to share with
colleagues or transferred to other computers. To store the state of the
environment we created in a file called `mda-15-environment`
colleagues or transferred to other computers. This allows all team members on
a project to use an identical set of software and makes your research projects
to be reproducible. To store the state of the environment we created in a file
called `myproject-environment`

{% highlight bash %}
conda list --explicit --md5 -n mda-15 > mda-15-environment
conda list --explicit --md5 -n myproject > myproject-environment
{% endhighlight %}

You can now copy this file to a colleague or onto another computer. The first 3
Expand All @@ -68,11 +81,6 @@ lines also contain instructions how this file can be used with conda.
# platform: linux-64
{% endhighlight %}

Instead of just creating environments for specific versions of software packages
you can of course also create a new environment for each project you are working
on.


More information about conda environments can be found in
the [official documentation]().

Expand Down