Fix CLR startup issues provided by Clang sanitizer

[danmoseley]

@steveharter commented on Thu Dec 03 2015

As part of #5236, attempt fix the clr startup issues so that test-specific sanitizer output can be determined and evaluated.

Currently, when the sanitizers are enabled, several messages from the undefined behavior sanitizer are displayed when any program is run:
/src/inc/clrhost.h:137:30: runtime error: call to function CExecutionEngine::GetTlsData() through pointer to incorrect function type 'void ()()'

/src/coreclr/hosts/unixcoreruncommon/coreruncommon.cpp:336:22: runtime error: call to function coreclr_initialize through pointer to incorrect function type 'int ()(const char *, const char *, int, const char *, const char *, void *, unsigned int *)'
/src/dlls/mscoree/unixinterface.cpp:187: note: coreclr_initialize defined here

/src/inc/clrhost.h:137:30: runtime error: call to function ClrFlsGetBlockGeneric() through pointer to incorrect function type 'void *()()'
/src/utilcode/clrhost_nodependencies.cpp:768: note: ClrFlsGetBlockGeneric() defined here

/src/inc/clrhost.h:184:30: runtime error: call to function ClrFlsGetBlockGeneric() through pointer to incorrect function type 'void *()()'

/src/vm/object.cpp:2633:5: runtime error: member call on null pointer of type 'Object'

/src/vm/object.cpp:2712:5: runtime error: member call on null pointer of type 'Object'

/src/gc/handletablecore.cpp:1987:42: runtime error: shift exponent 64 is too large for 32-bit type 'unsigned int'

/src/debug/ee/debugger.cpp:2351:24: runtime error: reference binding to null pointer of type 'const InteropSafe'

/src/debug/ee/controller.cpp:894:26: runtime error: reference binding to null pointer of type 'const InteropSafe'

/src/inc/random.h:108:30: runtime error: signed integer overflow: 1855390761 - -1417746578 cannot be represented in type 'int'

/src/gc/gc.cpp:9922:14: runtime error: member access within null pointer of type 'WKS::gc_heap'

/src/inc/sstring.inl:2090:20: runtime error: left shift of negative value -1

/src/pal/src/map/map.cpp:1588:35: runtime error: member access within null pointer of type 'MAPPED_VIEW_LIST' (aka 'CorUnix::_MVL')

There is also an address sanitizer error (these errors stop processing so it is blocking)
==51212==ERROR: AddressSanitizer: stack-buffer-underflow on address 0x7ffd252450df at pc 0x7fe3711c2a68 bp 0x7ffd25244f70 sp 0x7ffd25244f68
READ of size 1 at 0x7ffd252450df thread T0
#0 0x7fe3711c2a67 in CMiniMdBase::UsesAllocatedMemory(CMiniColDef*) /src/md/runtime/metamodel.cpp:1240:5

---

@danmosemsft commented on Tue Oct 11 2016

Shouldn't this move to a coreclr issue

---

@steveharter commented on Tue Oct 11 2016

Sure it could be moved as the bulk of the work is in coreclr. However, it is enabled for corefx native code as well.

---

@danmosemsft commented on Mon Oct 17 2016

Clearing assignee per our policy that assignment is for work that is imminent. UP for grabs

[yowl]

There's a few address sanitizer (ASan) problems also in gc.cpp around the use of the delete operator where delete [] should be used.e.g.

CFinalize::GrowArray()
{
    size_t oldArraySize = (m_EndArray - m_Array);
    size_t newArraySize =  (size_t)(((float)oldArraySize / 10) * 12);

    Object** newArray = new (nothrow) Object*[newArraySize];
    if (!newArray)
    {
        return FALSE;
    }
    memcpy (newArray, m_Array, oldArraySize*sizeof(Object*));

    //adjust the fill pointers
    for (int i = 0; i < FreeList; i++)
    {
        m_FillPointers [i] += (newArray - m_Array);
    }
    delete m_Array;

Are you accepting PRs for these?

[danmoseley]

@mangod9 for above question

[mangod9]

@yowl Sure please go ahead and create a PR.