Message send state

[nplasterer]

Part of #516

• Add enum field of failed to delivery status for when unpublished never publishes
• Expose delivery status
• Expose delivery status filtering on find_messages
• Expose message id on send

In a follow up PR we need to figure out how to set the delivery status to failed after 3 unsuccessful retries to publish. @tuddman maybe you have some ideas on how to do that?

[tuddman]

maybe you have some ideas on how to do that?

Keeping a counter of attempts is pretty straightforward; in-memory or even stored-on-disk, Exponential back-off, too, if we want to get fancy.
I'm keen to learn in what condition(s) publishing could or would fail, and see if we can ensure that never happens.
(basically making it more correct over hardening it from when it's wrong/broken/failing. We can do both, but I'd want to start with the former)

[nplasterer]

In the follow up PR we should do the following

1. Mark the optimistic message as FAILED whenever its intent is marked as failed (set_group_intent_error)
2. Make sure that every possible error case in this method results in the corresponding intent either being marked as TO_PUBLISH or FAILED
3. At some point in the future revisit whether there is a possibility of an infinite loop of intent retries. Maybe we should call increment_intent_publish_attempt_count() wherever we reset an intent to TO_PUBLISH, and just set a really high retry limit, like 10?

For (2), maybe the most fool-proof way is to mark all MessageProcessingErrors as either retryable or non-retryable, and have the caller of the method catch all errors and mark the intent as TO_PUBLISH for retryable errors and FAILED for non-retryable errors

[richardhuaaa]

Looks great