Kafka Improvements

[iancooper]

The Kafka client is not fit for prime time right now, it has some glaring problems such as the inability to set a partition key, and problems around how the consumer handles errors. This PR will fix that

[iancooper]

The asynchronous nature of the callback, coupled with an Outbox, means I have made some changes to how producers work here:

• We now set a threshold for failed sends in the Outbox
• Because we always write to the Outbox, although we retry a send, we don't crash on a failed send
• Instead we just don't mark it as sent
• So the new producer properties tell us how many outstanding requests there can be in a time window
• Exceed that number, and then we throw an OutboxLimitExceeded error.

This also revealed a couple of defects in the existing InMemoryBox code

• A monitor lock has to be taken and released on the same thread, not different ones
• This meant that housekeeping processes were only running once
• This has been fixed, so that the process now always runs when triggered, but the thread will terminate without doing work if it can't get the lock

[holytshirt]

I need a beer after reading through that. I think it looks okay so going to bring it in after the tests work.

[holytshirt]

Merge had to a bit of manual merging locally due to the json stuff, but all fixed now

[iancooper]

Thanks mate

[iancooper]

The real issue is the failures around the CI for Kafka. It's not clear what is going wrong, but I think is post my adding the tests that run on Confluent