NETOBSERV-617: split big payloads in GRPC exporter #81
Conversation
| entries := make([]*pbflow.Record, 0, len(inputRecords)) | ||
| for _, record := range inputRecords { | ||
| entries = append(entries, flowToPB(record)) | ||
| } | ||
| return &pbflow.Records{ | ||
| Entries: entries, | ||
| var records []*pbflow.Records |
There was a problem hiding this comment.
Could it be better done in one pass rather than 2 passes?
There was a problem hiding this comment.
I'll do it despite I'm afraid it will look a bit less readable. Also consider that in the second step we aren't copying arrays but just creating references to subslices pointing to the data in the original "entries" slice, so the impact in performance shouldn't be really high (saving the original "entries" reference, which is about 24 bytes, plus the control of the second loop, which will be usually few iterations).
But anyway let me do the change and then decide which one do you prefer.
There was a problem hiding this comment.
no no, that's fine, you can leave it as it is
I though it involved copying slices
Only for GRPC exporter, payloads with more than 10000 flows (default configurable value) are split in multiple messages that are submitted sequentially.