feature support watch event by key

[Liberxue]

when this's key value change , support notify key event

[roseduan]

Thank you for your attention.

What is your scenario? Please give me more information to help me to implement this.
btw, are you planning to use it or just learn?

[Liberxue]

Thank you for your attention.

What is your scenario? Please give me more information to help me to implement this. btw, are you planning to use it or just learn?
@roseduan
I'm currently working on a persistent cache of a segment tree
Have Found a repository that provides a very lightweight implementation, but it lacks event support. In my business scenario, there are many keys, and when the value associated with a key is changed, I need an event to watch it, similar to an Event-Driven Architecture (EDA).

[roseduan]

Would you like to accept this new feature? @Jeremy-Run

[Jeremy-Run]

@roseduan
OK, assign to me, but it will take some time.

[roseduan]

@roseduan OK, assign to me, but it will take some time.

OK, you can refer to the implementation in ETCD, I remember it has this feature.
And don`t rush to write the code, you can come up with a plan first, and we can discuss it.

[Jeremy-Run]

@roseduan OK, assign to me, but it will take some time.

OK, you can refer to the implementation in ETCD, I remember it has this feature. And don`t rush to write the code, you can come up with a plan first, and we can discuss it.

This is my plan, what do you think?

1. When the DB is opened, add whether to enable the key change notification parameter and a cached channel (the channel is used for consumer consumption)
2. If the parameter is true, perform the following steps
3. Design a circular queue to cache data
4. When batch commit, write the change information into the ring queue
5. Start a goroutine to read and process the information in the ring queue and put it into the channel
6. When the writing speed is much higher than the reading speed and the ring queue is not enough, write the overflow part into a list, wait for the ring queue to be read, and then put the data in the list into the ring queue to continue processing.

In addition, there is no strong guarantee for notification information, that is, only the notification content is stored in memory, and there may be cases where notification information is lost.

[roseduan]

@roseduan OK, assign to me, but it will take some time.

OK, you can refer to the implementation in ETCD, I remember it has this feature. And don`t rush to write the code, you can come up with a plan first, and we can discuss it.

This is my plan, what do you think?

1. When the DB is opened, add whether to enable the key change notification parameter and a cached channel (the channel is used for consumer consumption)
2. If the parameter is true, perform the following steps
3. Design a circular queue to cache data
4. When batch commit, write the change information into the ring queue
5. Start a goroutine to read and process the information in the ring queue and put it into the channel
6. When the writing speed is much higher than the reading speed and the ring queue is not enough, write the overflow part into a list, wait for the ring queue to be read, and then put the data in the list into the ring queue to continue processing.

In addition, there is no strong guarantee for notification information, that is, only the notification content is stored in memory, and there may be cases where notification information is lost.

we can implement it more straightforward, no need to add a list to store the overflow part.
we can create a fixed-size queue(the size can be an option), if the event is overflow, just remove the oldest data.

[Jeremy-Run]

@roseduan OK, assign to me, but it will take some time.

OK, you can refer to the implementation in ETCD, I remember it has this feature. And don`t rush to write the code, you can come up with a plan first, and we can discuss it.

This is my plan, what do you think?

1. When the DB is opened, add whether to enable the key change notification parameter and a cached channel (the channel is used for consumer consumption)
2. If the parameter is true, perform the following steps
3. Design a circular queue to cache data
4. When batch commit, write the change information into the ring queue
5. Start a goroutine to read and process the information in the ring queue and put it into the channel
6. When the writing speed is much higher than the reading speed and the ring queue is not enough, write the overflow part into a list, wait for the ring queue to be read, and then put the data in the list into the ring queue to continue processing.

In addition, there is no strong guarantee for notification information, that is, only the notification content is stored in memory, and there may be cases where notification information is lost.

we can implement it more straightforward, no need to add a list to store the overflow part. we can create a fixed-size queue(the size can be an option), if the event is overflow, just remove the oldest data.

That only needs to circulate the queue, the overflow directly overwrites the write, and the read pointer points to the next one of some write pointers. However this has the potential to lose events while the DB is running.

[roseduan]

Right, we can not store all events in memory, even the queue is huge enough, but the data may be more.
more events mean more memory cost, we can give the option to callers to create the fixed-size queue.

[Jeremy-Run]

Right, we can not store all events in memory, even the queue is huge enough, but the data may be more. more events mean more memory cost, we can give the option to callers to create the fixed-size queue.

Okay, any other points to discuss? If not, I will develop it according to this idea.

[roseduan]

Right, we can not store all events in memory, even the queue is huge enough, but the data may be more. more events mean more memory cost, we can give the option to callers to create the fixed-size queue.

Okay, any other points to discuss? If not, I will develop it according to this idea.

These are all my thoughts, thanks.

[roseduan]

@Liberxue This new feature has merged into main branch, you can try it as you wish.