Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BIOMAGE-1494] - Migration to add prefiltered attribute #191

Merged
merged 1 commit into from
Oct 11, 2021
Merged

[BIOMAGE-1494] - Migration to add prefiltered attribute #191

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
51 changes: 51 additions & 0 deletions migrations/dynamodb-migrations/2021-10-07T15:51:00Z-add-classifier-prefiltered.js
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
const Dyno = require('dyno');

let updated = 0;

module.exports = (record, dyno, callback) => {

if (!process.env.K8S_ENV) {
throw new Error('Environment (staging/production) must be specified.');
}

if(!record.processingConfig?.classifier) {
console.log(`${record.experimentId} skipped as it does not contain classifier in processingConfig`)
return callback();
}


if(!Object.prototype.hasOwnProperty.call(record.processingConfig?.classifier, 'enabled')) {
console.log(`${record.experimentId} skipped as it does not contain the right processingConfig schema`)
return callback();
}
Comment on lines +17 to +20
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does that actually happen? Are there experiments that don't have the right processingConfig schema?

Copy link
Contributor Author

@aerlaut aerlaut Oct 11, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, there are experiments that do not have processingConfig schema. Most are experiments that never run but does not get deleted (processingConfig is empty). Some does not have enabled in them, and that means they're old experiments.


const enabled = record.processingConfig.classifier.enabled

console.log(`${record.experimentId} flagged for updates with prefiltered value ${!enabled}`);

const attrValues = {
':prefiltered': !enabled,
};

// If you are running a dry-run, `dyno` will be null
if (!dyno) return callback();

dyno.updateItem({
Key: {experimentId: record.experimentId},
UpdateExpression: 'SET processingConfig.classifier.prefiltered = :prefiltered',
ExpressionAttributeValues: attrValues
}, (err) => {
if(err) {
console.log(`${record.experimentId} failed to update`);
throw new Error(err);
}
});

updated++;
callback();
}

module.exports.finish = (dyno, callback) => {
console.log(`Updated ${updated} records.`);
callback();
}