Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix linux crash from disassembler #2413

Merged
merged 7 commits into from
Sep 22, 2023
Merged

Fix linux crash from disassembler #2413

merged 7 commits into from
Sep 22, 2023

Conversation

timcassell
Copy link
Collaborator

@timcassell timcassell commented Aug 21, 2023

@timcassell timcassell requested a review from adamsitnik August 21, 2023 05:38
Copy link
Member

@adamsitnik adamsitnik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@timcassell thank you for providing a quick fix!

After reading dotnet/runtime#90794 (comment) I wonder whether we should simply unconditionally not try to map 0 and -1. Thoughts?

@timcassell

This comment was marked as outdated.

@timcassell
Copy link
Collaborator Author

timcassell commented Aug 21, 2023

@adamsitnik It looks like the IntelDisassembler already checks for it if (address > ushort.MaxValue) but the Arm64Disassembler does not.

if (address > ushort.MaxValue)
{
if (!IsVulnerableToAvInDac || IsCallOrJump(instruction))
{
TryTranslateAddressToName(address, isPrestubMD, state, isIndirect, depth, currentMethod);
}
}

But I guess that doesn't really catch it, because the address we're checking is ulong, so -1 is translated to ulong.MaxValue and passes the check.

@timcassell
Copy link
Collaborator Author

timcassell commented Aug 21, 2023

I changed it to always check for 0 or -1, though I'm not sure if it needs to only check that in non-windows.
Also, I'm not sure if addresses less than ushort.MaxValue are only invalid in x86, or also in ARM and if we should add that check in the Arm64Disassembler.

@timcassell
Copy link
Collaborator Author

@janvorli Is -1 address invalid in every environment? Also are addresses less than ushort.MaxValue invalid in every environment?

@leculver It might make sense to explicitly check for -1 address in ClrMd also.

@janvorli
Copy link
Member

Is -1 address invalid in every environment?

Yes, -1 is treated in a special way in DAC. See e.g.:
https://github.com/dotnet/runtime/blob/4a5695b6ea0e461e6527de057420872369345282/src/coreclr/debug/daccess/dacfn.cpp#L384-L388

Also are addresses less than ushort.MaxValue invalid in every environment?

This is true for Windows (actually, less or equal to that value are invalid). For Unix, this can differ based on the memory page size. On Linux, it is 4096 in most cases and 65536 for the case when the Linux kernel is configured for 64kB page size (some arm64 systems use that). But it can be different. For example, in WSL2 on Windows, it is 65536. The /proc/sys/vm/mmap_min_addr can be used to get the actual value on Linux.
On macOS, it is also different. On x64 it is be 4096 and on arm64 it is 4GB (0x100000000).

@timcassell
Copy link
Collaborator Author

@janvorli How can I use that to get the value on Linux? I tried /proc/sys/vm/mmap_min_addr on a GitHub runner to test, and it gave back Permission denied.

@leculver
Copy link

@janvorli How can I use that to get the value on Linux? I tried /proc/sys/vm/mmap_min_addr on a GitHub runner to test, and it gave back Permission denied.

You are trying to run it, if that's the command you gave.

cat /proc/sys/vm/mmap_min_addr
or
sudo cat /proc/sys/vm/mmap_min_addr
if it's protected

@timcassell timcassell force-pushed the dac-crash branch 2 times, most recently from 8d125d0 to a5dc597 Compare August 23, 2023 23:49
@timcassell timcassell force-pushed the dac-crash branch 3 times, most recently from 54fae10 to 3b51864 Compare August 24, 2023 21:11
@timcassell
Copy link
Collaborator Author

@adamsitnik This should be good to go, if you could take a look.

Copy link
Member

@adamsitnik adamsitnik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The changes look OK, but we have lost a possibility to map certain addresses.

Every time we introduce changes to the disassembler I run a sample benchmark with --disasmFilter * to get the disassembly of everything that we can on Win x64, Lin x64 and arm64.

dotnet run -c Release -f net7.0 --filter BenchmarkDotNet.Samples.IntroDisassembly.SumLocal --disasmFilter '*'

I've run this command on master branch, created a copy of the exported asm file and then run it against your branch. When eyeballing the diff I've found:

image

The smallest repro is:

dotnet run -c Release -f net7.0 --filter BenchmarkDotNet.Samples.IntroDisassembly.SumLocal --disasmFilter "BenchmarkDotNet.Characteristics.CharacteristicSetPresenter..ctor()"

@timcassell PTAL

src/BenchmarkDotNet/Disassemblers/ClrMdV2Disassembler.cs Outdated Show resolved Hide resolved
@timcassell
Copy link
Collaborator Author

@adamsitnik I spent a few hours trying to figure out why that was happening. I finally found that if I delete my BenchmarkDotNet.Samples\bin directory before running, it works as expected. If I run it twice in a row without deleting it, I get the missing ctor method. I see the same behavior on master, so it's unrelated to my changes.

Copy link
Member

@adamsitnik adamsitnik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thank you @timcassell

I spent a few hours trying to figure out why that was happening. I finally found that if I delete my BenchmarkDotNet.Samples\bin directory before running, it works as expected. If I run it twice in a row without deleting it, I get the missing ctor method. I see the same behavior on master, so it's unrelated to my changes.

@janvorli it would be great if we could debug it one day

@timcassell timcassell merged commit 46b3c01 into master Sep 22, 2023
@timcassell timcassell deleted the dac-crash branch September 22, 2023 21:34
@timcassell timcassell added this to the v0.13.9 milestone Sep 22, 2023
@timcassell timcassell added the bug label Sep 22, 2023
@janvorli
Copy link
Member

I have experimented with this a bit - just changing the runtime version in the test command line to 8.0. I cannot reproduce the behavior that @timcassell has seen, it fails to discover the method name for the constructor call all the time, even if I deleted the bin/obj folders before.
However, I've re-read some of my notes from the time I have implemented decoding the target from the stubs it calls and I've found that a case that doesn't work is when the target is called via a ready to run delayload helper. So, I've tried to disable ready to run by setting DOTNET_ReadyToRun=0 and the System.Object..ctor got decoded correctly. And then I've set it to 1 - and it was not decoded. So, I guess that is the culprit.
My notes say that real target extraction for calls via the ready to run delayload helpers would not be reasonably feasible from the disassembler code.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Using DisassemblyDiagnoser in GitHub Actions
4 participants