Enrich documentation related to Kafka #53
Conversation
|
I'd say it looks good, but specifying a particular topic partition might not be a good idea. High level consumers expect to be assigned partitions automatically. Therefore while the option is there, I'm not sure anyone should be encouraged to use it, unless you're doing something really specific. Otherwise 👍 |
|
@Steveb-p I agree, I wanted to highlight some options there, but I can also add a comment that this configuration option is advanced and you should be careful with it. |
| 'timestamp' => (new \DateTimeImmutable())->getTimestamp(), | ||
| 'messageId' => uniqid('kafka_', true), | ||
| ] | ||
| ]))) |
There was a problem hiding this comment.
I think this is a valid thing to show. However, I want to think through how someone will think about these options. I'm just guessing your mind would go like this (but I could totally be wrong, so tell me):
- You're using Kafka, and you decide you want to add a timestamp
- You search through Enqueue's Kafka adapter to figure out how. There's nothing in the docs, https://github.com/php-enqueue/enqueue-dev/blob/master/docs/transport/kafka.md, but you eventually find out that there is a setter method on the message object: https://github.com/php-enqueue/enqueue-dev/blob/c58e1af68b49612e597dc80a821444488f1af210/pkg/rdkafka/RdKafkaMessage.php#L142
- You then try to figure out how you can call that method on the Enqueue message via the transport
If I'm correct, then the header should be something like: How to set Custom Options onto Enqueue Message Objects and it should take about how the metadata are mapped to the setter methods of the underlying Enqueue message object.
There was a problem hiding this comment.
#55 might be an example of what I was thinking - user wanted to call a setter on the message object.
There was a problem hiding this comment.
And #58. So yea, apparently this is quite confusing!
There was a problem hiding this comment.
Sorry for the noise. I just added #61. Let me know if you still want to add some extra details. If you do, it might be best as an other example up in the new section I added.
Cheers!
There was a problem hiding this comment.
Yes, you got it. I can change it but I want to highlight Kafka because for AMQP there is some info already.
There was a problem hiding this comment.
So do you really need to have these timestamp and messageId attributes for Kafka? 🤔
There was a problem hiding this comment.
It just to highlight some options available to the user.
Hi @Steveb-p I'm very interested in been able to choose a partition because we have many topics configured in the wrong way due to our little knowledge about kafka when we started and i thing i'll be a very good workaround to consume messages in the order we want. I'm going to think how to do it and if i get something i will share it. Thank you. |
I searched some time to see how I can set a custom key and partition using symfony/messenger and maybe you consider it useful.