enable redzones in Debug CRT

[derekbruening]

From [email protected] on March 12, 2012 23:20:52

pattern mode is currently not detecting over/underflows w/ debug CRT b/c
there are no redzones

redzones are disabled because of issue #26 where debug operator delete goes and reads the heap header at a hardcoded negative offset.

originally they were disabled only for the _dbg routines themselves, but
there are asymmetries and abstraction violations seen in issue #500 that
resulted in having to disable redzones for all allocators in a module that contains _dbg routines

if we can handle redzones for issue #26 then the issue #500 mismatches shouldn't matter b/c those were all about redzone vs no redzone.

issue #26 alternative solutions to disabling redzones:

1. adjust on operator new:
   note that this was the first thing I considered (from looking at my
   notes) but at the time I was concerned about being able to find all
   the operators via the drsyms interfaces of the time.

   later I did write code having the operators as a first-class adjusting
   layer in my initial issue detect mismatched use of malloc/new/new[] vs free/delete/delete[] #123 implementation but I reverted it b/c
   I had trouble with placement new and placement delete.

   to identify placement operators we need func params: we have that on
   windows (DRi#605) but not on linux. and, there's no standard signature
   for placement delete. we could make some assumptions, but are we sure
   they'd be bulletproof? perhaps a 2nd param to operator delete might
   be used for some other purpose in a custom usage.

2. hack: make location in our redzone where operator delete will find
   header bit have the right bit set

it may well be that #2 will end up being more robust (as well as more
efficient, since a full layer will require post-call interception of
operator new which we avoid today)

Original issue: http://code.google.com/p/drmemory/issues/detail?id=832

[derekbruening]

From [email protected] on March 15, 2012 09:06:34

Option 3: only have redzone at the end.

#2 is easier to implement in general:
Any attempt to read the header can be discovered and handled in the fault handler.
However, there are two potential problems:

1. Some header read is not discovered, e.g. unaligned read or read at boundary.
2. Performance, if every memory deletion will cause a fault, the overhead might not be acceptable.

[derekbruening]

From [email protected] on March 15, 2012 09:10:52

by "fault handler" I assume you mean "unaddr handler" b/c only light mode (both types) uses faults. the ignore-unaddr optimization would have to be tweaked.