Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactors elasticsearch service and span name query to be like c* #1560

Merged
merged 1 commit into from
Apr 14, 2017

Conversation

codefromthecrypt
Copy link
Member

@codefromthecrypt codefromthecrypt commented Apr 12, 2017

Service and span names are written to the same daily indexes as spans
and dependency links as the document type "servicespan".

The document is a simple structure, like:

{
  "serviceName": "MyServiceName",
  "spanName": "MySpanName",
}

When the "servicespan" type is missing from an index, or there's no
results returned, a fallback nested query is invoked.

Fixes #1526

@codefromthecrypt codefromthecrypt force-pushed the es-servicespan branch 2 times, most recently from e2173ff to 79f8a9b Compare April 13, 2017 06:46
@codefromthecrypt
Copy link
Member Author

ok code now sends service/span indexes in the same batch command. This should help with AWS transfer charges, especially since we gzip requests. cc @devinsba @semyonslepov

still need some code cleanup and other notes in the TODO

@codefromthecrypt
Copy link
Member Author

backport logic in place. going to give a stab at a backport script as particularly serviceNames being current helps.

@codefromthecrypt
Copy link
Member Author

I think the best way to backport is to sneak hidden feature into zipkin-dependencies (as it is already a spark job and can do everything)

@codefromthecrypt codefromthecrypt force-pushed the es-servicespan branch 2 times, most recently from a669071 to 948de67 Compare April 13, 2017 15:25
@codefromthecrypt
Copy link
Member Author

If someone wants to migrate old data, they'd need to do the following.

For each span in an index (ex zipkin-2017-02-14/span)
If span.name isn't empty, create a document like this for each service name in span.annotations[].endpoint.serviceName, span.binaryAnnotations[].endpoint.serviceName,

id = myServiceName|mySpanName
{ "serviceName":"myServiceName", "spanName":"mySpanName",

save the unique set of documents to the "servicespan" type in the same index (ex zipkin-2017-02-14/servicespan)

@codefromthecrypt
Copy link
Member Author

depends on #1562

@codefromthecrypt codefromthecrypt force-pushed the es-servicespan branch 2 times, most recently from a90ecd0 to e188444 Compare April 14, 2017 05:19
@codefromthecrypt
Copy link
Member Author

fixed a very silly test setup bug which consumed a lot of my last night.. will be ready to go on green

PS ignoring circleci because it thinks the date is different than it is (a routine problem unrelated to this code)

Service and span names are written to the same daily indexes as spans
and dependency links as the document type "servicespan".

The document is a simple structure, like:
```json
{
  "serviceName": "MyServiceName",
  "spanName": "MySpanName",
}
```

When the "servicespan" type is missing from an index, or there's no
results returned, a fallback nested query is invoked.
@codefromthecrypt codefromthecrypt merged commit 3c1fe5a into master Apr 14, 2017
@codefromthecrypt codefromthecrypt deleted the es-servicespan branch April 14, 2017 06:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant