Moving/inconsistent data locations

[pieterlukasse]

Moving/inconsistent data locations - a tough one since different foundations and NCI offer free storage, forcing us to change buckets, breaking links etc. Also, currently having to use very sensitive AWS keys to get the data.

Priority:

Desired solution:

Preferred or recommended technical approach:

• Miguel: a solution might be to use cavatica to create projects, but keeping up with updates is something bix ops struggles with seemingly.

DoD:

Questions

• Is this related to itemId=26122397?

Ideas

• Would it be possible to keep a list of "known locations" and have the data Extraction scripts consider multiple potential locations? In this way, maintenance could be reduced to simply updating a configuration file with a list of locations.

[forus]

I've started google document where I elaborate on the issue and possible solutions https://docs.google.com/document/d/1O2pkLMwIGrOtcBZ1BpFDg6POhkPEoX8UeSATDf1sZbk/edit?usp=sharing