Fill freed loader heap chunk with non-zero value #12731

parjong · 2017-07-11T01:44:12Z

This commit implements a new feature that fills freed global loader heap chunk with some non-zero value (currently 0xCC) in order to make it easy to diagnose dangling Reverse P/Invoke callback issues.

jkotas · 2017-07-12T19:53:52Z

This feature make it easy to detect issues caused by dangling Reverse P/Invoke callbacks.

Requiring a private build of the runtime won't qualify as "easy" for most users. We used to have MDA (managed debugging assistants) in full .NET Framework for similar debugging aids.

This particular one is CallbackOnCollectedDelegate . It is more robust than what you are proposing, gives better diagnostic message and does not require a custom build of the runtime. The code for it still there, but it is disable. We may want to rather look into adding a switch to enable it for CoreCLR. In full framework, it is configured via pretty complex xml config file. It should be more lightweight in CoreCLR.

cc @yizhang82

yizhang82 · 2017-07-12T20:43:26Z

Agree with @jkotas with the delegate scenario. However, this feature sounds like goodness in general in debug builds. We should probably enable it in debug build by default. I don't think the feature switch is required at all.

jkotas · 2017-07-12T20:47:25Z

We should probably enable it in debug build by default

What kind of debug builds do you have in mind?

yizhang82 · 2017-07-12T20:51:26Z

What kind of debug builds do you have in mind?

Our internal dbg/chk builds, to catch issues in our dbg/chk test runs. Not the retail ones.

parjong · 2017-07-13T10:52:22Z

@jkotas MDA will be very useful, but this feature is intended for a bit different use cases.

This feature aims to provide some hints when only coredump is available. I'm not sure, but it seems that MDA is not intended for such cases (Please correct me if I'm wrong).

jkotas · 2017-07-13T11:51:20Z

With your proposed change, you need a custom build of the runtime. With MDAs, you need to set an environment variable. Either way, it is not on by default.

I think it is good idea to make the basic diagnostic - that crashes more predictably for recycled thunks - on by default. You change is trying to achieve this by overwriting the recycled delegate thunks with a different pattern, not with zeros.

The problem is that it negative perf consequences because of it makes us memset all loader heap memory to 0 as side-effect. It is unnecessary because of this memory is zero filled by the OS.

Could you please look into coding this such that the memset kicks in for the recycled thunks only? memseting the recycled thunks has neglible cost - it can be on by default. You actually just need to set thunk entrypoint to a breakpoint instruction, no need to memset the whole thing.

parjong · 2017-07-17T23:47:02Z

@dotnet-bot test Windows_NT x64 Formatting please

parjong · 2017-07-18T01:56:03Z

@jkotas Sorry for late response (I was out of office). I updated PR to invoke memset only for reclaimed regions.

Reclaimed regions could be distinguished from fresh regions by reading the first byte as fresh regions will be filled with zero (by OS) and reclaimed regions will be filled with non-zero (by CLR), and thus simple check before memset will be enough in order to prevent memset for fresh regions.

Unfortunately, the whole reclaimed regions should be filled with non-zero value under this scheme.

jkotas · 2017-07-18T05:12:10Z

I do not think we want to make this a yet another build option. It is not that hard to fix this up to make it enabled by default without negative perf consequence.

jkotas · 2017-07-18T11:12:08Z

clrdefinitions.cmake

@@ -136,6 +136,7 @@ if(FEATURE_INTERPRETER)
  add_definitions(-DFEATURE_INTERPRETER)
 endif(FEATURE_INTERPRETER)
 add_definitions(-DFEATURE_ISYM_READER)
+add_definitions(-DFEATURE_LOADER_HEAP_GUARD)


Delete this

jkotas · 2017-07-18T11:19:00Z

src/utilcode/loaderheap.cpp

@@ -723,6 +729,9 @@ struct LoaderHeapFreeBlock
            }
 #endif

+#if LOADER_HEAP_REDZONE_VALUE != 0


Instead of this:

Add flag to UnlockedLoaderHeap that suppresses all filling of the memory by LoaderAllocator (you can steal some bits from m_flProtect), and set this flag for flag for SystemDomain::GetGlobalLoaderAllocator().

Change UMEntryThunk::Terminate to overwrite the entrypoint with jump to a method that reports the collected delegate problem and makes it easy to diagnose it.

While you are on this, you can delete FEATURE_WINDOWSPHONE from UMEntryThunk::CreateUMEntryThunk/Terminate. FEATURE_WINDOWSPHONE has been always defined for CoreCLR

It means that zero-filling is not necessary for GlobalLoaderHeap. Am I right? I though that it is a part of LoaderHeap semantics.

The zero filling should not be required for GetGlobalLoaderAllocator()->GetExecutableHeap(). This method is used in a very few places, and none of them need it.

jkotas · 2017-07-19T03:36:07Z

src/inc/loaderheap.h

@@ -288,7 +288,8 @@ class UnlockedLoaderHeap
                       SIZE_T dwReservedRegionSize,
                       size_t *pPrivatePerfCounter_LoaderBytes = NULL,
                       RangeList *pRangeList = NULL,
-                       BOOL fMakeExecutable = FALSE);
+                       BOOL fMakeExecutable = FALSE,
+                       BOOL fMakeRelaxed = FALSE);


I would call this flag fZeroInit = TRUE to make it clear what it does.

jkotas · 2017-07-19T03:37:05Z

src/vm/dllimportcallback.cpp

-#endif
+
+    _ASSERTE(SystemDomain::GetGlobalLoaderAllocator()->GetExecutableHeap()->IsRelaxed());
+    FillMemory(&m_code, sizeof(m_code), 0xcc);


Other thread can allocate the block at this point. This needs to be done before the call to BackoutMem.

jkotas · 2017-07-19T03:39:54Z

src/vm/loaderallocator.cpp

@@ -893,7 +893,7 @@ void LoaderAllocator::ActivateManagedTracking()
 #define COLLECTIBLE_CODEHEAP_SIZE                  (7 * GetOsPageSize())
 #define COLLECTIBLE_VIRTUALSTUBDISPATCH_HEAP_SPACE (5 * GetOsPageSize())

-void LoaderAllocator::Init(BaseDomain *pDomain, BYTE *pExecutableHeapMemory)
+void LoaderAllocator::Init(BaseDomain *pDomain, BYTE *pExecutableHeapMemory, BOOL fRelaxed)


This method should not take the flag.

jkotas · 2017-07-19T03:40:13Z

src/vm/loaderallocator.cpp

@@ -1005,7 +1005,8 @@ void LoaderAllocator::Init(BaseDomain *pDomain, BYTE *pExecutableHeapMemory)
                                                                      dwExecutableHeapReserveSize,
                                                                      LOADERHEAP_PROFILE_COUNTER,
                                                                      NULL,
-                                                                      TRUE /* Make heap executable */);
+                                                                      TRUE /* Make heap executable */,
+                                                                      fRelaxed);


This should be false unconditionally.

jkotas · 2017-07-19T03:41:47Z

src/vm/dllimportcallback.cpp

-#endif
+
+    _ASSERTE(SystemDomain::GetGlobalLoaderAllocator()->GetExecutableHeap()->IsRelaxed());
+    FillMemory(&m_code, sizeof(m_code), 0xcc);


0xCC works well for x86 because of it is code for the breakpoint instruction. What does it do on arm?

Use fZeroInit (instead of fMakeRelazed)

jkotas · 2017-07-19T10:55:31Z

src/vm/dllimportcallback.cpp


    RETURN p;
 }

+#if defined(_TARGET_X86_) || defined(_TARGET_AMD64_)
+#define REDZONE_VALUE 0xcc


Red zone is area under stack pointer: https://en.m.wikipedia.org/wiki/Red_zone_(computing)

Could you please find better name for it?

It may be best to move this into a platform specific method on UMEntryThunkCode . memset may not be always appropriate to neutralize the entrypoint.

How about poison pill?

We may introduce Spoil method as you suggested, which allows us to insert a breakpoint to the exact entry point (instead of filling the whole region).

Sounds like a plan. Poison or Neutralize may be a better name than Spoil.

parjong · 2017-07-19T23:24:08Z

@jkotas I updated PR as suggested, and confirmed that it is working for x86/x64/arm. Unfortunately, I have no access to arm64 environment. Could you let me know if there is any way to build and test arm64 binaries from CI?

jkotas · 2017-07-20T02:43:27Z

src/vm/loaderallocator.cpp

@@ -1005,7 +1005,8 @@ void LoaderAllocator::Init(BaseDomain *pDomain, BYTE *pExecutableHeapMemory)
                                                                      dwExecutableHeapReserveSize,
                                                                      LOADERHEAP_PROFILE_COUNTER,
                                                                      NULL,
-                                                                      TRUE /* Make heap executable */);
+                                                                      TRUE /* Make heap executable */,
+                                                                      FALSE /* Do NOT zero-initialize */);


Maybe useful to describe why - that it is needed by the UMEntryThunkCode::Poison.

jkotas · 2017-07-20T02:48:50Z

src/vm/arm/stubs.cpp

+void UMEntryThunkCode::Poison()
+{
+    m_code[0] = 0xbebe;
+    FlushInstructionCache(GetCurrentProcess(),&m_code,sizeof(m_code));


I do not think FlushInstructionCache is needed here - it is not free. This aid is not 100% reliable because of it depends on the GC timings, making it a bit less reliable by skipping the FlushInstructionCache here should be ok.

jkotas · 2017-07-20T02:50:02Z

Unfortunately, I have no access to arm64 environment. Could you let me know if there is any way to build and test arm64 binaries from CI?

@sdmaclea Could you please help with filling in/testing the arm64 implementation of this?

sdmaclea · 2017-07-20T14:39:51Z

@dotnet-bot test Windows_NT arm64 Checked

sdmaclea · 2017-07-20T14:41:26Z

@parjong I assume Windows testing will be sufficient. Let me know if you need it tested on Ubuntu, I make a local build. if needed.

parjong · 2017-07-20T22:11:56Z

@sdmaclea Windows testing is sufficient for me. Thank you for your help!

parjong · 2017-07-30T23:15:12Z

@jkotas Could you please take a look? Given the result, this PR does not introduce any regression on ARM64.

jkotas · 2017-07-31T05:41:37Z

Could you please open an issue to get the implementation filled on ARM64 and add link to the issue from the code?

jkotas · 2017-07-31T05:43:40Z

src/vm/arm/stubs.cpp

@@ -2522,6 +2522,11 @@ void UMEntryThunkCode::Encode(BYTE* pTargetCode, void* pvSecretParam)
    FlushInstructionCache(GetCurrentProcess(),&m_code,sizeof(m_code));
 }

+void UMEntryThunkCode::Poison()
+{


Could you please add a comment with instruction that this translates so

jkotas · 2017-07-31T05:45:03Z

src/vm/amd64/cgenamd64.cpp

+    }
+    CONTRACTL_END;
+
+    m_movR10[0] = 0xCC;


Nit: You can use X86_INSTR_INT3 here - it is defined for x64 as well.

jkotas · 2017-07-31T05:45:24Z

LGTM modulo comments. Thanks

jkotas

Thanks

Add FEATURE_LOADER_HEAP_GUARD feature

3384270

dnfclas added the cla-already-signed label Jul 11, 2017

parjong changed the title ~~FEATURE_LOADER_HEAP_GUARD: Fill Freed Loader Heap with non-zero value~~ FEATURE_LOADER_HEAP_GUARD: Fill freed loader heap chunk with non-zero value Jul 11, 2017

jkotas added the area-Interop label Jul 12, 2017

Invoke memset only for reclaimed regions

48e667b

Enable FEATURE_LOADER_HEAP_GUARD by default

87b2287

jkotas suggested changes Jul 18, 2017

View reviewed changes

parjong changed the title ~~FEATURE_LOADER_HEAP_GUARD: Fill freed loader heap chunk with non-zero value~~ Fill freed loader heap chunk with non-zero value Jul 19, 2017

Insert trap inside UMEntryThunk::Terminate

66939d1

jkotas reviewed Jul 19, 2017

View reviewed changes

parjong added 3 commits July 19, 2017 14:29

Make all exectuable heaps not to zero-initialize itself

0c13959

Use fZeroInit (instead of fMakeRelazed)

Add comment

7c63a69

Revert unnecessary changes

cd6d5ec

jkotas reviewed Jul 19, 2017

View reviewed changes

Add and use 'Poison' method to insert a trap

63dcb89

jkotas reviewed Jul 20, 2017

View reviewed changes

parjong added 2 commits July 20, 2017 12:59

Do NOT invoke FlushInstructionCache

03e2ed1

Update comment

4029d1c

jkotas reviewed Jul 31, 2017

View reviewed changes

parjong added 2 commits July 31, 2017 15:04

Add comment on ARM Poisoning instruction

c986ba4

Use X86_INSTR_INT3 instead of 0xCC

9c97b74

jkotas approved these changes Jul 31, 2017

View reviewed changes

jkotas merged commit 1c540c5 into dotnet:master Jul 31, 2017

parjong deleted the fix/FEATURE_LOADER_HEAP_GUARD branch July 31, 2017 07:53

jkotas mentioned this pull request Jul 31, 2017

[Arm64] Implement Poison() #13125

Merged

karelz modified the milestone: 2.1.0 Aug 28, 2017

parjong mentioned this pull request Jan 31, 2020

[ARM64/Linux] Implement UMEntryThunkCode::Poison dotnet/runtime#8658

Closed

Fill freed loader heap chunk with non-zero value #12731

Fill freed loader heap chunk with non-zero value #12731

Conversation

parjong commented Jul 11, 2017 • edited Loading

jkotas commented Jul 12, 2017

yizhang82 commented Jul 12, 2017

jkotas commented Jul 12, 2017

yizhang82 commented Jul 12, 2017

parjong commented Jul 13, 2017

jkotas commented Jul 13, 2017 • edited Loading

parjong commented Jul 17, 2017

parjong commented Jul 18, 2017 • edited Loading

jkotas commented Jul 18, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

parjong commented Jul 19, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jkotas commented Jul 20, 2017

sdmaclea commented Jul 20, 2017

sdmaclea commented Jul 20, 2017

parjong commented Jul 20, 2017

parjong commented Jul 30, 2017

jkotas commented Jul 31, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jkotas commented Jul 31, 2017

jkotas left a comment

Choose a reason for hiding this comment

parjong commented Jul 11, 2017 •

edited

Loading

jkotas commented Jul 13, 2017 •

edited

Loading

parjong commented Jul 18, 2017 •

edited

Loading

parjong commented Jul 19, 2017 •

edited

Loading