Non-required Android testing build

[tobrun]

I addition to our regular Android CI build, I'm thinking about introducing a separate non-required testing build for Android. In the past, our instrumentation tests have shown to be unreliable and flaky (see #6546 for more information). To be able to make progress on these and catching regression in the meantime, a non-required build seems to be the most logical next step.

@mapbox/gl @mapbox/android @mapbox/ios

[1ec5]

Note that a non-required build may allow you to merge, but GitHub will still show the commit as having failed a build in commit history.

[tobrun]

Another idea for this would be to only run this build when a PR was merged to master?
This doesn't show up as a part of the commit history builds but will probably result in showing/catching regressions when it's too late already. More information about the trigger map in http://devcenter.bitrise.io/webhooks/trigger-map/.

[1ec5]

I’m referring to this commit history, which indicates whether all the builds passed for a particular merged commit. I suppose we could make the build always pass but still send some sort of failure message to Slack or elsewhere, but the build would have diminished usefulness.

This sounds like a job for Danger, which can post an inline comment in the event of a failure without actually failing the build.

[tobrun]

Danger definitely sounds interesting (also for things as requiring changelog 😉). At the moment, we are focusing on hardening the tests which makes this issue obsolete. closing for now.