-
Notifications
You must be signed in to change notification settings - Fork 67
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
pkg/agent: trigger batch before grpc message size overflow #836
Conversation
Fixes: parca-dev#203 Signed-off-by: heylongdacoder <[email protected]>
Hello, I don't have much experience on writing concurrent code, might make some mistake. Thanks for guidance in advance. :D |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looking great!
I think we can simplify this code and make it more maintainable. We can get rid of one of the mutex locking/unlocking and remove one of the channels if we can make batch
method to receive a request (probably we should rename it to just send
after this change). With this flow will be easier and less error prone.
What do you think?
mtx *sync.RWMutex | ||
series []*profilestorepb.RawProfileSeries | ||
triggerBatch chan struct{} | ||
batchDone chan struct{} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we can get rid of batchDone
in here.
select { | ||
case b.batchDone <- struct{}{}: | ||
default: | ||
} | ||
} | ||
} | ||
|
||
func (b *BatchWriteClient) batch(ctx context.Context) error { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
func (b *BatchWriteClient) batch(ctx context.Context) error { | |
func batch(ctx context.Context, r *profilestorepb.WriteRawRequest) error { |
This method can receive the request and by this way we can get rid of the lock in this method if I'm not mistaken. This method could also omit the write client as a receiver if it's the case.
@heylongdacoder Let me know if you need any help with this. |
@kakkoyun thanks for the review and guidance! yes I need some help. I am not sure how would it works if we removed the Currently I got another idea to avoid the creation of these |
Outdated since #2958 We now use a totally new protocol that is apache-arrow based. |
Fixes: #203
Signed-off-by: heylongdacoder [email protected]