-
Notifications
You must be signed in to change notification settings - Fork 174
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
New gofast protobuf code #236
Conversation
Gerald Rule: Copy Observability on Veneur and Unilog pull requestscc @stripe/observability |
We do marshal, within the |
Also, you'll have to update the build process in both Travis and and Jenkins. Updating the latter will fix the build error we're getting (though we have to update the former too, or else we're not properly testing the actual code we run). Scanning the |
Ok, ready for re-r @aditya-stripe now that I'm done fucking up the build. |
Dockerfile
Outdated
@@ -6,7 +6,8 @@ ENV GOPATH=/go | |||
RUN apt-get update | |||
RUN apt-get install -y zip | |||
RUN go get -u -v github.com/ChimeraCoder/gojson/gojson | |||
RUN go get -u -v github.com/golang/protobuf/protoc-gen-go | |||
RUN go get -u github.com/golang/protobuf/protoc-gen-go |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is there a reason for dropping the -v
flag here?
Maybe let's rebase this so lgtm other than that! |
37fcaa0
to
4b85f11
Compare
4b85f11
to
b2dc46c
Compare
re-approval for rebase |
Summary
Uses gogo-protobuf's
gofast
code generator for faster marshal and unmarshaling.Motivation
I stumbled into a comment somewhere that gogo-protobuf was way faster than stock protobuf on account of it not using reflection and generating more code for marshaling and unmarshaling. I gave it a whirl and it turns out to be way faster.
Using the same benchmark we made for #234 we get these results with the gogo-protobuf "gogo-fast" compiler:
Unmarshaling (which we do a lot of):
So it's 66% faster, and uses 54% less ram with one fewer alloc. Wow! I did not test marshaling since I don't know that we even do that in this code anywhere.
Marshaling (important for client libs):
Well, that's exciting. 90% reduction and zero allocations. I suspect it's because we're not writing it it to anything in the test. We emit it to a
Writer
in the code.Needs
I'm not certain how to add the gogo-protobuf binary to the
Dockerfile
so that this works with go-generate. I assume ago get
added won't DWIW, or will it?Test plan
gogo-protobuf says it is goprotobuf compatible, so I am relying on existing unit tests + a normal test cycle in QA.
r? @aditya-stripe