Long and large cron jobs with travis

[calispac]

Latelly we have had integration of the simtel_array data analysis in digicampipe. And in the near future we will have files from the camera calibration.

So far we are unit-testing but with cron jobs from TravisCI we can try to schedule more complete tests.

The issue right now are:

1. Time limitation by TravisCI
2. Limited space inside the repo.

Regarding those points

1. The timeout is:

• A job produces no log output for 10 minutes
• A job on travis-ci.org takes longer than 50 minutes
• A job on travis-ci.com takes longer than 120 minutes
  (https://docs.travis-ci.com/user/customizing-the-build/#Build-Timeouts)

• Total size of GH repo ~ 1GB
• File size limitation = 100MB
• No idea what is the limit for Travis (assume same as GH)

So it seems not a solution to run these kind of tests with TravisCI in the long term. But I believe one can ask Travis to trigger some jobs on a machine/cluster of machines that has access to the MC and data files.

Any suggestions?

[dneise]

I would like to propose a more "low tech" way than travis...

• Find or buy a machine (~500,-)
• Install a recent Ubuntu LTS
• Download our crab-data-sample, and other input data, slowdata and stuff to that machine.

Write a cron-job which runs, dunno .... once per hour or every 2 hours or so... and does:

• fresh install of anaconda and of our software.
• run tests and analyses as much as you want.
• capture the entire stuff in a textfile and send it around as an email and/or present it on a website on that machine.

[dneise]

I have a PC here under my desk which could do that, but I'd assume one would rather have this machine in geneva or krakow?

[calispac]

Yes I think the plan is to use a machine that has already access to the "database". That would be even better if it is a machine we are already familiar with (one of the cluster we use, or a cluster that we will use).