Starting sync automatically?

[jonatanmindroad]

Is there a preferred way to sync datasets automatically? The use case would be for the dashboard to handle data that is continuously being uploaded.

I am considering extending the django container with a cronjob that calls python manage.py syncdataset periodically. But perhaps there is another obvious way to do this.

Any ideas?

[joefutrelle]

The sync_bin endpoint is used in this example script for periodically transferring data from an IFCB and syncing the bins to the dashboard.

https://github.com/WHOIGit/ifcbdb/blob/master/utilities/auto_transfer.py

[jonatanmindroad]

I looked into the included script, but in the end I went with my original idea.

If anyone else want to do the same I can give some notes. It turned out that cronjobs inside containers has its own challenges.

[TomAkaTma]

Hi @jonatanmindroad , any notes would be greatly appreciated - I'm about to try and do exactly the same thing. Thanks!

[jonatanmindroad]

@TomAkaTma Sorry for not answering earlier, I've been away. I can share the solution in full but I just want to sort out an issue with it first. The sync did not work as I intended when I was away and I want to figure out why.

EDIT: Found and fixed the error. Now it works as expected.

[jonatanmindroad]

Here are instructions for my setup:
https://gist.github.com/jonatanmindroad/0ed781a0cc6e12aa9de0655575b553b8

ping @TomAkaTma