Bytecode Instrumentation Not Work on .NET 6 #2934
Comments
|
I can reproduce this issue using 2 sample code: v0.6.0I had 2 new commits on top of v0.6.0. See below branch: v1.0.0I had 2 new commits on top of v1.0.0. See below branch: |
|
@muhaook, I was able to execute your Ba integration on .NET 6 with some minor changes on self-hosted application. Steps to execute
You should be able to find logs both in OTel Auto log files and in Console. I do not have easy accessible IIS-env, so please let me know if it is working for you. You can find code here: https://github.com/Kielek/opentelemetry-dotnet-instrumentation/tree/mike-ba-integration |
|
I have made another tests. Server preparation
3. Install https://dotnet.microsoft.com/en-us/download/dotnet/thank-you/runtime-aspnetcore-6.0.21-windows-hosting-bundle-installer
4. Restart IIS
Application preparation:
OTel Auto preparation
Set up IIS
Enable OpenTelemetry
Import-Module .\OpenTelemetry.DotNet.Auto.psm1 -Verbose
Install-OpenTelemetryCore -LocalPath .\tracer.zip
Register-OpenTelemetryForIIS
In logs you should see something like this for native part: Logs from my tests: log.zip I need some beter steps to reproduce to verify your issue. |
|
Thanks @Kielek for the details. I tried to verify your fix on Windows 10 but still failed. Is it possible the CLR profiling API we are using in otel-net-instrument works on Windows 11 but does not work on Windows 10 ? Here are some details: My test env is Windows 10 Version 22H2 (OS Build 19045.3324)Deployed TestApplication.AspNetCore on IIS.I can see Made a build off your branch.I made a log change to make sure I am running the expected build on my env: Put contents in bin/tracer-home into tracer.zip Enable OpenTelemetry
Accessed webapplication and saw
|
|
Bytecode Instrumentation works on Piotr's Win10. So looks like this is my test env issue. |
|
It is strange - I tried to bring up brand-new VMs with Windows 10, 11, or 2022, all failed to instrument bytecode. All these VMs are our company's VM. Is it possible that some software running on the VMs blocks the bytecode instrumentation? I tried my personal laptop (windows 11), the instrumentation works. |
|
@muhaook, one more thing based on what I see in your logs, the standard instrumentation for |
|
Agree @Kielek , so this issue happened on both custom and standard bytecode instrumentation. I compared failed v.s. succeeded logs and found more suspicious. It is likely a regression in native code as we were all good with 0.6.0. comparison between succeeded log (on the lefthand side) and failed log (on the righthand side):in failed log, we can see 2 CLR profilers got brought up: one is for .NET 6; the other is for .NET Framework. in failed log, during shutdown: Attached please find the full failed and succeeded logs |
|
Hi @muhaook , I haven't been able to explore your logs files yet, but it appears that the AspNetCore application is being hosted on an application pool that has .NET Framework v4 running. Can you check if updating the application pool to |
|
Thanks @zacharycmontoya, I found we have this issue right after this commit: df78d34 I made a build after the above commit. even after updating the application pool to I also make a build before the above commit and it succeeded with .NET framework enabled on the application pool. So looks like .NET framework is irrelevant. Attached please find the logs: |
muhaook
changed the title
|
Hi @muhaook, thanks for identifying the commit you saw this break, that helps tremendously. Are you able to share the source code for the integration you are working on? If you need to scrub the code inside the methods that would be fine too and would help me debug this locally. Thanks! |
|
Hi @zacharycmontoya, I used to use Piotr's branch mike-ba-integration and saw this issue only happened on
I can not reproduce this issue on
The source code I am using to reproduce this issue now is on branch issue2934-postv060_1. This branch is on top of commit df78d34. The issue happened on my company laptop but did not happen on my VM on virtualBox. Attached please find logs collected on both envs: after comparing native logs, I found rejit did not happen on my laptop while it happend on virtualBox as expected: I have another branch issue2934-postv060_0 which is on top of the commit right before the commit df78d34. branch issue2934-postv060_0 works well on all test envs (laptops and VMs) |
|
As an update, at least one issue is being caused by both .NET Core and .NET Framework being loaded in the same process. We have a field in the CLR Profiler called In your latest |
|
Good news - bytecode instrumentation succeeded when Looks like this issue is caused by both .NET Core and .NET Framework being loaded in the same process. An observation is bytecode instrumentation was successful with 0.6.0 even though both .NET Core and .NET Framework got loaded in the same process there as well. |
|
Ah I see, we added some new code in df78d34 that immediately sets the @open-telemetry/dotnet-instrumentation-maintainers after 0.6.0 we can only instrument .NET Core apps hosted in IIS when their app pool is configured with @muhaook I think that once we determine the approach here, we can close the issue and then proceed with the follow-up work. Thank you for raising this issue |
I think that this is a good goal to have. This type of problem can be difficult to diagnose even if we have documentation for it, so it would be nice to have it just work. |
|
Summary Next Steps
|
Bug Report
Symptom
Describe the bug
I have some custom bytecode instrumentations which worked on otel-net-instrument 0.6.0.
But they partially failed on otel-net-instrument 1.0.0: failed on .NET 6 while succeeded on .NET framework 4.6.2.
Here is an example of my custom bytecode instrumentation:
1/ implemented integration class/methods like
2/ added an entry in src\OpenTelemetry.AutoInstrumentation\Configurations\TracerInstrumentation.cs
otel-net-instrument can be built successfully. I can see generated files such as
I have 2 sample applications: one is on .NET framework 4.6.2; the other is on .NET 6.
When I run the application on .NET framework 4.6.2, I looked into log and can see all bytecode instrumentation definitions got loaded:
furtherdown in the log, I can see bytecode got instrumented for my custom requirement:
All good for .NET framework.
When I run the application on .NET 6, I can see all bytecode instrumentation definitions got loaded as well:
but I cannot see any bytecode got instrumented.
Runtime environment (please complete the following information):
Attached please find logs:
otel-dotnet-auto-native-w3wp-3560---NET6.log
otel-dotnet-auto-native-w3wp-7268---NETFX462.log
The text was updated successfully, but these errors were encountered: