don't check message type if BackOff

[alexvpickering]

Background

sns_message_payload["MessageAttributes"]["type"]["Value"] doesn't exist when the message is "BackOff":

This bug causes worker and pipeline pods are not shut down after exiting. As a result, when a user returns to an experiment after a time of inactivity, a new worker pod will not be assigned and they won't be able to get new results. Persistent pods will also run up costs.

Link to issue

Link to staging deployment URL

Links to any Pull Requests related to this

Anything else the reviewers should know about the changes here

Changes

Code changes

Definition of DONE

Your changes will be ready for merging after each of the steps below have been completed:

Testing

• Unit tests written
• Tested locally with Inframock (with latest production data downloaded with cellenics experiment pull)
• Deployed to staging

To set up easy local testing with inframock, follow the instructions here: https://github.com/hms-dbmi-cellenics/inframock
To deploy to the staging environment, follow the instructions here: https://github.com/hms-dbmi-cellenics/cellenics-utils

Documentation updates

Is all relevant documentation updated to reflect the proposed changes in this PR?

• Relevant Github READMEs updated
• Relevant wiki pages created/updated

Approvers

• Approved by a member of the core engineering team
• (UX changes) Approved by vickymorrison (this is her username, tag her if you need approval)

Just before merging:

• After the PR is approved, the unstage script in here: https://github.com/hms-dbmi-cellenics/cellenics-utils is executed. This script cleans up your deployment to staging

Optional

• Photo of a cute animal attached to this PR