Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add proto3 encoding #1942

Closed
codefromthecrypt opened this issue Mar 3, 2018 · 5 comments
Closed

Add proto3 encoding #1942

codefromthecrypt opened this issue Mar 3, 2018 · 5 comments
Labels
enhancement model Modeling of traces

Comments

@codefromthecrypt
Copy link
Member

codefromthecrypt commented Mar 3, 2018

We should eventually map out a proto3 encoding even if this is just a direct translation from the json. We don't yet have an issue for this, but it is in response to #1644 and its parent.

In the mean time, we should at least make it possible in the java library to do this. Minimally, this requires adding zipkin2.codec.Encoding.PROTO3 to permit this format when using the async reporter.

@codefromthecrypt
Copy link
Member Author

codefromthecrypt commented Mar 11, 2018

Looking at docs and generated code in proto3, I think the following are true for our key functions. Note that I'm assuming zipkin will use a field number less than 128 for the span data (StackDriver is field number 1, so yeah works for them too).

  • int listSizeInBytes(int encodedSizeInBytes)
    The overhead is the field number then the size
    uint(field_number) + uint(encodedSizeInBytes)

  • int listSizeInBytes(List<byte[]> values)
    The overhead is repeating above for each element (as messages can't be packed)

@basvanbeek this seem to gel with your understanding of proto3?

It looks like what the generated code does:

    size = 0;
    for (int i = 0; i < traces_.size(); i++) {
      size += com.google.protobuf.CodedOutputStream
        .computeMessageSize(1, traces_.get(i));
    }

https://developers.google.com/protocol-buffers/docs/encoding

@codefromthecrypt
Copy link
Member Author

FYI @saturnism I will try to get this in by zipkin 2.6 which should land sometime in the next week or so depending on the UI-related changes that are still in flight. After that, it should be possible to make the zipkin-gcp sender using stackdriver-specific proto3 encoding even if we haven't defined a proto3 zipkin fomat yet

@jorgheymans
Copy link
Contributor

will this eventually lead to zipkin starting as a grpc server ? or it's really just the protobuf encoding of the traces at this point ?

@codefromthecrypt
Copy link
Member Author

codefromthecrypt commented Mar 19, 2018 via email

@codefromthecrypt
Copy link
Member Author

codefromthecrypt commented Mar 19, 2018 via email

codefromthecrypt pushed a commit that referenced this issue Mar 20, 2018
While we haven't yet defined a proto3 format, this allows one to be
defined. This also allows us to refine how stackdriver (or any future
translated format) works in zipkin reporter.

Implementation is relatively simple: this assumes a buffer of spans is
analogous to a proto3 repeated field, and that the field name itself is
ordinal <=127.

Fixes #1942
codefromthecrypt pushed a commit that referenced this issue Mar 20, 2018
While we haven't yet defined a proto3 format, this allows one to be
defined. This also allows us to refine how stackdriver (or any future
translated format) works in zipkin reporter.

Implementation is relatively simple: this assumes a buffer of spans is
analogous to a proto3 repeated field, and that the field name itself is
ordinal <=127.

Fixes #1942
@codefromthecrypt codefromthecrypt added enhancement model Modeling of traces labels Oct 26, 2018
abesto pushed a commit to abesto/zipkin that referenced this issue Sep 10, 2019
While we haven't yet defined a proto3 format, this allows one to be
defined. This also allows us to refine how stackdriver (or any future
translated format) works in zipkin reporter.

Implementation is relatively simple: this assumes a buffer of spans is
analogous to a proto3 repeated field, and that the field name itself is
ordinal <=127.

Fixes openzipkin#1942
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement model Modeling of traces
Projects
None yet
Development

No branches or pull requests

2 participants