Adds support for Microsoft Azure Application Insights(AppInsights) as a datastore for Zipkin #27

praveenbarli · 2017-05-10T08:57:59Z

This PR contains changes for supporting Microsoft Azure Application Insights as a data store for Zipkin.

updating recent commit in openzipkin:zipkin-azure

SergeyKanzhelev · 2017-06-09T18:16:12Z

...nfigure/storage-applicationinsights/src/main/resources/zipkin-server-applicationinsights.yml

+zipkin:
+  storage:
+    applicationinsights:
+      instrumentationKey: ${AI_INSTRUMENTATION_KEY:}


if it's a section applicationinsights - why we need to prefix it with AI_?

It is readable here in config file.. but during actual configuration user jus sets AI_INSTRU... and it would be much clearer to have prefix.

SergeyKanzhelev · 2017-06-09T18:17:19Z

...nfigure/storage-applicationinsights/src/main/resources/zipkin-server-applicationinsights.yml

+zipkin:
+  storage:
+    applicationinsights:
+      instrumentationKey: ${AI_INSTRUMENTATION_KEY:}


environment variable for instrumentation key that we typically use is APPINSIGHTS_INSTRUMENTATIONKEY https://github.com/Microsoft/ApplicationInsights-dotnet/blob/151509eac5e23fcbd09f3a8989a35d1099703957/src/Core/Managed/Shared/Extensibility/Implementation/TelemetryConfigurationFactory.cs#L22

It is not ideal as it uses a wrong name appinishgts, but it is what we use everywhere

I will change it to APPINSIGHTS.. If thats norm.... i just thought AI_IN... will make the key shorter..and in power shell I often type so less work :)

SergeyKanzhelev · 2017-06-09T18:19:07Z

...nfigure/storage-applicationinsights/src/main/resources/zipkin-server-applicationinsights.yml

+    applicationinsights:
+      instrumentationKey: ${AI_INSTRUMENTATION_KEY:}
+      applicationId: ${AI_APPLICATION_ID:}
+      apiKey: ${AI_API_KEY:}


how fast are you running to the query limit? Using Azure Active Directory authentication may eliminate this problem, but may be hard to implement. Just a thought in case you have an idea how to fix it

I ran few thousands and didn't face much issues. For higher loads .. I am thinking of rotating API_Keys as a solution ..but would like to know about AAD.

SergeyKanzhelev · 2017-06-09T18:28:28Z

...sights/src/main/java/zipkin/storage/applicationinsights/ApplicationInsightsSpanConsumer.java

+        }
+        String res = gson.toJson(jsonElement);
+        String msg = "{ \"Span\":" + res + "}";
+        telemetry.trackTrace(msg, SeverityLevel.Critical, spanProps);


I see you are using Application Insights like a storage. I'd advice to use Application Insights data model instead Use requests and dependencies for incoming and outgoing spans. Like explained here: https://docs.microsoft.com/azure/application-insights/application-insights-correlation#open-tracing-and-application-insights

SergeyKanzhelev · 2017-06-09T18:30:01Z

storage/applicationinsights/src/main/resources/ApplicationInsights.xml

+  <!-- HTTP request component (not required for bare API) -->
+
+  <TelemetryModules>
+    <Add


since you only track telemetry manually - you do not need any auto-collection modules and initializers.

SergeyKanzhelev · 2017-06-09T18:30:59Z

storage/applicationinsights/pom.xml

+    </dependency>
+    <dependency>
+      <groupId>com.microsoft.azure</groupId>
+      <artifactId>applicationinsights-web</artifactId>


do you monitor zipkin itself or just use it to send telemetry? I'd advice to use applicationinsights if you do not use any auto-collection modules

ok.. I am not monitoring zipkin. So, I will change this. Thanks!

By "applicaitoisisghts", Do you mean "applicationinsights-core"?

Merge from master to pick up latest changes

SergeyKanzhelev · 2017-06-23T22:08:07Z

autoconfigure/pom.xml

@@ -33,6 +35,7 @@

  <modules>
    <module>collector-eventhub</module>
+    <module>storage-applicationinsights</module>


what's the difference between collector and storage? Should we have Application Insights collector AND storage. So collection part can be used separately from the storage?

Collector is the transport protocol(default Http) you listen on ... for receiving spans. Right now we are using Http - so, I guess we might not have a separate AI collector unless you have a better business case where AI collector helps.
You can directly hit your storage with Spans if that is your question. It is fine as long as Zipkin REST API gets the right data while reading.
Collectors like Eventhub could have many clients other than storage.

Can collector be used in-process? So we do not need a sidecar container hosting it? Also it looks to me this is a regular setup. Like here:

Production configuration

Numerous collectors zipkin collectors per data centre – no UI enabled

One zipkin UI server per data centre – firewalled and hidden behind an apache web server to provide authentication and auditing. Not accessible for span collector service

marcote · 2017-06-25T01:45:59Z

Hi there. Any update around this PR approval? I'm trying to create a docker images (without luck so far) and this PR not approved yet makes things a little more difficult.
If interested, the docker image code is here: https://github.com/marcote/zipkin-appinisights-docker
Thanks

SergeyKanzhelev · 2017-06-25T05:29:48Z

@marcote, thanks for interest. @praveenbarli and I working on some refinements on how spans should be stored. This way we will be able to see spans natively in Application Insights and see Application Insights's collected data in Zipkin. We've estimated another week to complete this work

praveenkbarli · 2017-06-25T06:20:35Z

@marcote Nice to see you trying out this PR. We are working with admins @adriancole @aliostad to get this checked in. However I would like you to continue with your setup and let I and @SergeyKanzhelev know if you face any challenges. That way we can work in parallel and also a succesfull installation and testing in your system would be a positive push for admins to review/check in changes.

…com/praveenbarli/zipkin-azure into createautoconfigureappinsightmodule

…zipkin-azure into writetoaidatamodel

data model changes to add name, id and dependency for sr annotaiton

readme.md

…able

add zipkin span in json as custom property to requests/dependencies t…

codefromthecrypt · 2018-10-01T23:20:14Z

as discussed with @dhaval24 and others at dinner last week. The way to move things forward is to change this into a "write only" component by making it a combination of translator (zipkin2 -> azure span) a sender (for zipkin-reporter meaning apps can embed it), and possibly a write-only storage component which can allow a zipkin server to proxy spans to your service. Possibly you'd also want a brave component if you are using trace context format with some azure specific trace state.

key things that are still the case are that we'd want some end user involved to help justify the community efforts which result in new code. For example, even the existing code I have done dozens of releases and that's something we shouldn't do unless end users are benefiting.

For reference, here's the zipkin-gcp repo which has functionality you'd want https://github.com/openzipkin/zipkin-gcp

dhaval24 · 2018-10-02T16:54:36Z

@adriancole thanks a lot for bringing this back! Though, I still do not fully get this work, here is what my understanding is:

This would enable the ZipKin view for people using AzureMonitor on Zipkin UX. Am I correct ?

A question is:

Would the reverse be true? Will the spans emitted from sleuth able to be viewed on AzureMonitor UX for tracing ?

Apologies if this is a very rudimentary question, but I am very new to Sleuth / Zipkins / Brave world :) @nikmd23 can comment on the justification, we did had many customers who were using Application Insights for Java with SpringBoot applications and were interested in getting slueth spans reported to AzureMonitor and be able to view the E2E traces on Azure UX.

codefromthecrypt · 2018-10-02T23:02:08Z

If this is write-only then it wouldn't allow use of zipkin UI, rather only your UI. However, it is better to tease functionality apart like this as otherwise it is endless. For example, not only did this stall out, but also another attempt like influx. Rather than going for gold, probably best to start with writing, then see if there is value moving forward in readback etc.

…

On Wed, Oct 3, 2018 at 12:54 AM Dhaval Doshi ***@***.***> wrote: @adriancole <https://github.com/adriancole> thanks a lot for bringing this back! Though, I still do not fully get this work, here is what my understanding is: 1. This would enable the ZipKin view for people using AzureMonitor on Zipkin UX. Am I correct ? A question is: 1. Would the reverse be true? Will the spans emitted from sleuth able to be viewed on AzureMonitor UX for tracing ? Apologies if this is a very rudimentary question, but I am very new to Sleuth / Zipkins / Brave world :) @nikmd23 <https://github.com/nikmd23> can comment on the justification, we did had many customers who were using Application Insights for Java with SpringBoot applications and were interested in getting slueth spans reported to AzureMonitor and be able to view the E2E traces on Azure UX. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#27 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAD61_vp-oFzidth0UWrHoj0Q_4Zu5fiks5ug5pMgaJpZM4NWXbJ> .

codefromthecrypt · 2018-11-08T08:32:15Z

closing because.. well it has been well over a year

Merge pull request #1 from openzipkin/master

de546ef

updating recent commit in openzipkin:zipkin-azure

praveenbarli changed the title ~~Createautoconfigureappinsightmodule~~ Adds support for Microsoft Azure Application Insights(AppInsights) as a datastore for Zipkin May 10, 2017

praveenbarli force-pushed the createautoconfigureappinsightmodule branch 3 times, most recently from 7890521 to 74e297f Compare May 10, 2017 22:00

praveenbarli mentioned this pull request May 11, 2017

Add storage support (ex table service or docdb) #8

Open

praveenbarli force-pushed the createautoconfigureappinsightmodule branch 2 times, most recently from 7c36d14 to 90f2f24 Compare May 15, 2017 23:40

praveenbarli force-pushed the createautoconfigureappinsightmodule branch from 90f2f24 to e304bcd Compare June 5, 2017 05:28

Adds storage component for Application Insights (AppInsights)

e4401f2

praveenbarli force-pushed the createautoconfigureappinsightmodule branch from e304bcd to e4401f2 Compare June 5, 2017 20:38

SergeyKanzhelev suggested changes Jun 9, 2017

View reviewed changes

SergeyKanzhelev mentioned this pull request Jun 21, 2017

Zipkin to Application Insights module #33

Open

Sergey Kanzhelev and others added 2 commits June 23, 2017 10:53

Merge remote-tracking branch 'origin/master' into addai

3b4bc0a

Merge pull request #2 from SergeyKanzhelev/addai

5184e7c

Merge from master to pick up latest changes

SergeyKanzhelev reviewed Jun 23, 2017

View reviewed changes

praveenbarli and others added 11 commits June 26, 2017 14:33

parent verions updated from0.1.6 to 0.1.7

b894c0b

parent verions updated from0.1.6 to 0.1.7

f9912ea

Merge branch 'createautoconfigureappinsightmodule' of https://github.…

c563faf

…com/praveenbarli/zipkin-azure into createautoconfigureappinsightmodule

data model changes to add name, id and dependency for sr annotaiton

78a8ebd

data model changes to add name, id and dependency for sr annotaiton

219c4f1

Merge branch 'writetoaidatamodel' of https://github.com/praveenbarli/…

71d4cec

…zipkin-azure into writetoaidatamodel

removed unused refs

735e056

removed appinsights config, appinsights-web pkg

249284b

set operation id to hex val of traceid's, duration in millis

607f59c

Merge pull request #5 from praveenbarli/writetoaidatamodel

ab7fc99

data model changes to add name, id and dependency for sr annotaiton

readme.md

795b03a

Sergey Kanzhelev and others added 6 commits June 28, 2017 22:11

update spacing in configuraiton file

98cfa3d

fix script

ae56314

few things

534f785

Merge pull request #3 from SergeyKanzhelev/addai

2d16b45

readme.md

add zipkin span in json as custom property to requests/dependencies t…

a63bf61

…able

Merge pull request #6 from praveenbarli/addzipkinspanproperty

40950a2

add zipkin span in json as custom property to requests/dependencies t…

SergeyKanzhelev mentioned this pull request Nov 10, 2017

Cross-Component Telemetry Correlation for the Java SDK microsoft/ApplicationInsights-Java#457

Closed

codefromthecrypt closed this Nov 8, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Adds support for Microsoft Azure Application Insights(AppInsights) as a datastore for Zipkin #27

Adds support for Microsoft Azure Application Insights(AppInsights) as a datastore for Zipkin #27

praveenbarli commented May 10, 2017

SergeyKanzhelev Jun 9, 2017

praveenbarli Jun 9, 2017

SergeyKanzhelev Jun 9, 2017

praveenbarli Jun 9, 2017 •

edited

Loading

SergeyKanzhelev Jun 9, 2017

praveenbarli Jun 9, 2017 •

edited

Loading

SergeyKanzhelev Jun 9, 2017

SergeyKanzhelev Jun 9, 2017

SergeyKanzhelev Jun 9, 2017

praveenbarli Jun 9, 2017

praveenbarli Jun 27, 2017

SergeyKanzhelev Jun 23, 2017

praveenbarli Jun 23, 2017 •

edited

Loading

SergeyKanzhelev Jun 25, 2017

marcote commented Jun 25, 2017

SergeyKanzhelev commented Jun 25, 2017

praveenkbarli commented Jun 25, 2017

codefromthecrypt commented Oct 1, 2018

dhaval24 commented Oct 2, 2018

codefromthecrypt commented Oct 2, 2018 via email

codefromthecrypt commented Nov 8, 2018

Adds support for Microsoft Azure Application Insights(AppInsights) as a datastore for Zipkin #27

Adds support for Microsoft Azure Application Insights(AppInsights) as a datastore for Zipkin #27

Conversation

praveenbarli commented May 10, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

praveenbarli Jun 9, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

praveenbarli Jun 9, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

praveenbarli Jun 23, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

marcote commented Jun 25, 2017

SergeyKanzhelev commented Jun 25, 2017

praveenkbarli commented Jun 25, 2017

codefromthecrypt commented Oct 1, 2018

dhaval24 commented Oct 2, 2018

codefromthecrypt commented Oct 2, 2018 via email

codefromthecrypt commented Nov 8, 2018

praveenbarli Jun 9, 2017 •

edited

Loading

praveenbarli Jun 9, 2017 •

edited

Loading

praveenbarli Jun 23, 2017 •

edited

Loading