Travis cache settings seem wrong

[mauritsvanrees]

Currently .travis.yml.bob has:

cache:
  pip: true
  directories:
  - eggs
  - downloads
...
before_install:
...
  - echo "download-cache = $HOME/buildout-cache/downloads" >> $HOME/.buildout/default.cfg
  - echo "eggs-directory = $HOME/buildout-cache/eggs" >> $HOME/.buildout/default.cfg

So shouldn't the cache: directories setting contain buildout-cache instead of eggs and downloads?
It is a bit hard to say for sure, because you don't see what ends up in the Travis cache. I thought I had seen a link where you could download or inspect a cache file, but I do not see it now.

[mauritsvanrees]

TL;DR: we should simply cache the eggs directory, and not set a buildout download-cache or eggs-directory. See bottom.

I have experimented a bit with the cache setup in collective.denyroles. Have a look at the buildout time in Travis.

The first build, with only one job in the matrix, took 6 minutes, and the next one less than 2 minutes, so the cache was working at that point. .travis.yml was:

cache:
  pip: true
  directories:
  - eggs
  - downloads
...
bin/buildout -N buildout:download-cache=downloads

Slightly different than current bobtemplates.plone, but working.

Later I added more jobs to the matrix, so the builds obviously started taking longer, but I also switched to what bobtemplates.plone currently has, or variants thereof. See this commit. That build took 12 minutes, probably with the previous cache still somewhat working, and later builds 17 to 18 minutes. .travis.yml:

cache:
  pip: true
  directories:
  - buildout-cache
...
before_install:
...
  - echo "[buildout]" > $HOME/.buildout/default.cfg
  - echo "download-cache = $HOME/buildout-cache/downloads" >> $HOME/.buildout/default.cfg
  - echo "eggs-directory = $HOME/buildout-cache/eggs" >> $HOME/.buildout/default.cfg
  - bin/buildout -N buildout:download-cache=downloads

That is a bit of a mess, now that I look at it, because this first sets the download cache to ~/buildout-cache/downloads and then overrides it to simply downloads. This is not how bobtemplates.plone does it. But it has the same problem: the eggs do not end up in the Travis cache. Here the eggs are stored in /home/travis/buildout-cache/eggs, but we tell Travis to cache buildout-cache, which is actually /home/travis/build/collective/collective.denyroles/buildout-cache.

Anyway, a later .travis.yml resulted in good caching, see build 12 taking 18 minutes and the following build 13 (triggered without changes) taking only 6 minutes:

cache:
  pip: true
  directories:
  - $HOME/buildout-cache
before_install:
...
  - echo "[buildout]" > $HOME/.buildout/default.cfg
  - echo "download-cache = $HOME/buildout-cache/downloads" >> $HOME/.buildout/default.cfg
  - echo "eggs-directory = $HOME/buildout-cache/eggs" >> $HOME/.buildout/default.cfg
  - bin/buildout -N

That works, so we could switch to that.
But then I wanted to simplify it. And that worked too. Travis ran job 14, and I restarted the job so it used the cache, and that took less than five minutes. .travis.yml:

cache:
  pip: true
  directories:
  - downloads
  - eggs
before_install:
...
  - bin/buildout -N

I now realise that the downloads directory will be empty, or actually not even created. It is only used when you set buildout:download-cache in default.cfg or on the command line. But we are only interested in the resulting eggs, so the downloaded packages in downloads/dist are needless extra baggage. If buildout downloads a non-egg, for example a varnish tarball, then it can be useful, but that is not usually the case.
So it can be further simplified. Current .travis.yml at master:

cache:
  pip: true
  directories:
  - eggs
before_install:
...
  - bin/buildout -N

Latest build number 15 takes less than 5 minutes for 5 jobs, with the longest job taking 1 minute and 9 seconds. That seems pretty good.

Does that last setup look good for general usage? I can make a PR then.

[idgserpro]

Good catch @mauritsvanrees. If we are only interested in eggs, it makes no sense to cache the downloads folder.

If you do a PR, might be you able to fix collective/buildout.plonetest#51 too.

[idgserpro]

This has been fixed in #409