chann: support to consume all data after Close the channel. #5
Conversation
chann.go
Outdated
| // first case will ways be selected. See #3. | ||
| default: | ||
| ch.q[0] = nilT // de-reference earlier to help GC | ||
| ch.q = ch.q[1:] |
There was a problem hiding this comment.
This reads like a previous implementation that was reported to be problematic, as discussed in #3. Could you help me to understand how your changes can improve the overall situation?
There was a problem hiding this comment.
Yep, in this PR, it behaves as follows
When an unbounded channel is closed
- If remaining data in input channel or buffer queue is larger than size of output channel, and no receiver will consume these data, the
unboundedProcessinggoroutine will block at https://github.com/golang-design/chann/pull/5/files#diff-95e80cac22cfd39c19faf1211d443965c49871890d6f92ee5d93fa719fc5f5daR213. This situation leaks goroutine as well as memory of the chann, but user can consume all data to release these resources. At the same time there is no cpu spin issue. - If remaining data can be sent to output channel, there is no goroutine leak issue.
| // Note if receiver doesn't consume all data that has been sent to input | ||
| // channel, the `unboundedProcessing` goroutine will leak forever. | ||
| // Ref: https://github.com/golang-design/chann/issues/3 | ||
| ch.out <- ch.q[0] |
There was a problem hiding this comment.
As you documented, if there is no receiver, we will leak a goroutine, and it seems not ideal. Isn't it? Or at least do we need to inform the users in the documentation?
There was a problem hiding this comment.
Hi @amyangfei can you take a look at this comment? I want to use this library soon
And as for
Considering the channel behavior in standard library, if user doesn't receive data from a buffered channel, data in the buffered channel is still kept in memory and has memory leak too, even if the channel is closed.
GC should be able to recycle the unreachable objects when we use built-in channel
Or maybe we can add another function NewUnbounded[T]() (*Chann[T], func()) that returns a cleanup function? like context's cancel function, it will remind the user that don't forget to invoke the function.
ref: #3
In order to fix this new bug: #3 (comment), this PR removes the default branch when sending backlog data into output channel.
The drawback is when there is no receiver or the receiver doesn't consume all data that has been sent to input channel, the
unboundedProcessinggoroutine will leak forever. IMO the user of this library should take care of this behavior.Considering the channel behavior in standard library, if user doesn't receive data from a buffered channel, data in the buffered channel is still kept in memory and has memory leak too, even if the channel is closed.