Add 64-bit Mac OSX support #1979
Comments
wtfbbqhax
changed the title
derekbruening
added
Component-Build
OpSys-x64
OpSys-OSX
Priority-Low
help wanted
and removed
Priority-Low
labels
derekbruening
changed the title
derekbruening
added a commit
that referenced
this issue
Apr 7, 2019
A series of small changes to build with XCode 10.1 64-bit: + Updates -max_supported_os_version on Mac to 18. + Updates 64-bit Mac frame field. + Removes , from runtime options in template defines. + Removes 32-bit-only tests from the OSX label list for 64-bit. + Increases the core's preferred base to 0x1'71000000 to stay above the default 0x1' _PAGEZERO. + Adds missing DECL_EXTERNs in mangle_suspend and mangle_asynch tests. + Adds missing GLOBAL_REFs in common.decode test. + Re-defines sigcontext_t and uc_mcontext in tools.h to match this toolchain. + Adds 64-bit libelftc.a and libdwarf.a Mach-O libraries along with instructions on how they were built. + Adds -mmacosx-version-min=10.9 to drcachesim to match core/. + Fixes a format string warning in drcachesim. + Enables Mac testing on Travis for PR's for project-mac64. Issue: #1979
derekbruening
added a commit
that referenced
this issue
Apr 7, 2019
A series of small changes to build with XCode 10.1 64-bit: + Updates -max_supported_os_version on Mac to 18. + Updates 64-bit Mac frame field. + Removes , from runtime options in template defines. + Removes 32-bit-only tests from the OSX label list for 64-bit. + Increases the core's preferred base to 0x1'71000000 to stay above the default 0x1' _PAGEZERO. + Adds missing DECL_EXTERNs in mangle_suspend and mangle_asynch tests. + Adds missing GLOBAL_REFs in common.decode test. + Re-defines sigcontext_t and uc_mcontext in tools.h to match this toolchain. + Adds 64-bit libelftc.a and libdwarf.a Mach-O libraries along with instructions on how they were built. + Adds -mmacosx-version-min=10.9 to drcachesim to match core/. + Fixes a format string warning in drcachesim. + Enables Mac testing on Travis for PR's for project-mac64. + Includes project-mac64 on the Travis and Appveyor branch lists. + Fixes clang-format issues from prior commits 7aef272 and 4975170 Issue: #1979
derekbruening
added a commit
that referenced
this issue
Apr 8, 2019
A series of small changes to build with XCode 10.1 64-bit: + Updates -max_supported_os_version on Mac to 18. + Updates 64-bit Mac frame field. + Removes , from runtime options in template defines. + Removes 32-bit-only tests from the OSX label list for 64-bit. + Increases the core's preferred base to 0x1'71000000 to stay above the default 0x1' _PAGEZERO. + Adds missing DECL_EXTERNs in mangle_suspend and mangle_asynch tests. + Adds missing GLOBAL_REFs in common.decode test. + Re-defines sigcontext_t and uc_mcontext in tools.h to match this toolchain. + Adds 64-bit libelftc.a and libdwarf.a Mach-O libraries along with instructions on how they were built. + Adds -mmacosx-version-min=10.9 to drcachesim to match core/. + Fixes a format string warning in drcachesim. + Enables Mac testing on Travis for PR's for project-mac64. Issue: #1979
derekbruening
pushed a commit
that referenced
this issue
Apr 8, 2019
For 64-bit MacOS, there is no way to set the %fs base which stops us from using DR's scheme used on other unix platforms. This commit provides initial support to MacOS 64-bit by stealing a TLS slot from the app for DR's TLS base. + implement is_thread_tls_initialized for MacOS 64-bit + implement tls_thread_init and tls_thread_free + set MACOS64 define in cmake script + add WRITE_TLS_SLOT_IMM etc. for MacOS 64-bit + add read_thread_register for MacOS 64-bit to get pthread_t base Issue: #1979
derekbruening
added a commit
that referenced
this issue
Apr 15, 2019
A series of small changes to build with XCode 10.1 64-bit: + Updates -max_supported_os_version on Mac to 18. + Updates 64-bit Mac frame field. + Removes , from runtime options in template defines. + Removes 32-bit-only tests from the OSX label list for 64-bit. + Increases the core's preferred base to 0x1'71000000 to stay above the default 0x1' _PAGEZERO. + Adds missing DECL_EXTERNs in mangle_suspend and mangle_asynch tests. + Adds missing GLOBAL_REFs in common.decode test. + Re-defines sigcontext_t and uc_mcontext in tools.h to match this toolchain. + Adds 64-bit libelftc.a and libdwarf.a Mach-O libraries along with instructions on how they were built. + Adds -mmacosx-version-min=10.9 to drcachesim to match core/. + Fixes a format string warning in drcachesim. Includes a number of similar small changes from Shawn Denbow's first 2 commits on the project-mac64 branch in PR #2269 and PR #2273, updated for clang-format: + Move -vm_base above 4GB since __PAGEZERO takes up first 4GB by default + Increase MEMQUERY_INTERNAL_DATA_LEN + Update Mach-O parsing to check for LC_SEGMENT_64 + Change error notifications to warning to allow build to continue + Define SC_FIELD for 64-bit registers r8-r15 and update references + Update dynamorio_mach_dep_syscall to use syscall for 64-bit + Update dynamorio_mach_syscall to use syscall for 64-bit + Change code referencing eflags to xflags for cross platform compat + Misc code fix-ups Issue: #1979
derekbruening
added a commit
that referenced
this issue
Apr 15, 2019
A series of small changes to build with XCode 10.1 64-bit (from PR #3514 on project-mac64): + Updates -max_supported_os_version on Mac to 18. + Updates 64-bit Mac frame field. + Removes , from runtime options in template defines. + Removes 32-bit-only tests from the OSX label list for 64-bit. + Increases the core's preferred base to 0x1'71000000 to stay above the default 0x1' _PAGEZERO. + Adds missing DECL_EXTERNs in mangle_suspend and mangle_asynch tests. + Adds missing GLOBAL_REFs in common.decode test. + Re-defines sigcontext_t and uc_mcontext in tools.h to match this toolchain. + Adds 64-bit libelftc.a and libdwarf.a Mach-O libraries along with instructions on how they were built. + Adds -mmacosx-version-min=10.9 to drcachesim to match core/. + Fixes a format string warning in drcachesim. Includes a number of similar small changes from Shawn Denbow's first 2 commits on the project-mac64 branch in PR #2269 and PR #2273, updated for clang-format: + Move -vm_base above 4GB since __PAGEZERO takes up first 4GB by default + Increase MEMQUERY_INTERNAL_DATA_LEN + Update Mach-O parsing to check for LC_SEGMENT_64 + Change error notifications to warning to allow build to continue + Define SC_FIELD for 64-bit registers r8-r15 and update references + Update dynamorio_mach_dep_syscall to use syscall for 64-bit + Update dynamorio_mach_syscall to use syscall for 64-bit + Change code referencing eflags to xflags for cross platform compat + Misc code fix-ups Issue: #1979
derekbruening
pushed a commit
that referenced
this issue
Apr 15, 2019
For 64-bit MacOS, there is no way to set the %fs base which stops us from using DR's scheme used on other unix platforms. This commit provides initial support to MacOS 64-bit by stealing a TLS slot from the app for DR's TLS base. + implement is_thread_tls_initialized for MacOS 64-bit + implement tls_thread_init and tls_thread_free + set MACOS64 define in cmake script + add WRITE_TLS_SLOT_IMM etc. for MacOS 64-bit + add read_thread_register for MacOS 64-bit to get pthread_t base Issue: #1979
derekbruening
added a commit
that referenced
this issue
Apr 15, 2019
For 64-bit MacOS, there is no way to set the %fs base which stops us from using DR's scheme used on other unix platforms. This commit provides initial support to MacOS 64-bit by stealing a TLS slot from the app for DR's TLS base. + implement is_thread_tls_initialized for MacOS 64-bit + implement tls_thread_init and tls_thread_free + set MACOS64 define in cmake script + add WRITE_TLS_SLOT_IMM etc. for MacOS 64-bit + add read_thread_register for MacOS 64-bit to get pthread_t base Issue: #1568, #1979
hgreving2304
pushed a commit
that referenced
this issue
Apr 22, 2019
A series of small changes to build with XCode 10.1 64-bit (from PR #3514 on project-mac64): + Updates -max_supported_os_version on Mac to 18. + Updates 64-bit Mac frame field. + Removes , from runtime options in template defines. + Removes 32-bit-only tests from the OSX label list for 64-bit. + Increases the core's preferred base to 0x1'71000000 to stay above the default 0x1' _PAGEZERO. + Adds missing DECL_EXTERNs in mangle_suspend and mangle_asynch tests. + Adds missing GLOBAL_REFs in common.decode test. + Re-defines sigcontext_t and uc_mcontext in tools.h to match this toolchain. + Adds 64-bit libelftc.a and libdwarf.a Mach-O libraries along with instructions on how they were built. + Adds -mmacosx-version-min=10.9 to drcachesim to match core/. + Fixes a format string warning in drcachesim. Includes a number of similar small changes from Shawn Denbow's first 2 commits on the project-mac64 branch in PR #2269 and PR #2273, updated for clang-format: + Move -vm_base above 4GB since __PAGEZERO takes up first 4GB by default + Increase MEMQUERY_INTERNAL_DATA_LEN + Update Mach-O parsing to check for LC_SEGMENT_64 + Change error notifications to warning to allow build to continue + Define SC_FIELD for 64-bit registers r8-r15 and update references + Update dynamorio_mach_dep_syscall to use syscall for 64-bit + Update dynamorio_mach_syscall to use syscall for 64-bit + Change code referencing eflags to xflags for cross platform compat + Misc code fix-ups Issue: #1979
hgreving2304
pushed a commit
that referenced
this issue
Apr 22, 2019
For 64-bit MacOS, there is no way to set the %fs base which stops us from using DR's scheme used on other unix platforms. This commit provides initial support to MacOS 64-bit by stealing a TLS slot from the app for DR's TLS base. + implement is_thread_tls_initialized for MacOS 64-bit + implement tls_thread_init and tls_thread_free + set MACOS64 define in cmake script + add WRITE_TLS_SLOT_IMM etc. for MacOS 64-bit + add read_thread_register for MacOS 64-bit to get pthread_t base Issue: #1568, #1979
derekbruening
added a commit
that referenced
this issue
Sep 14, 2019
Uses pthread_key_create() to allocate enough contiguous and aligned TLS slots to fit our os_local_state_t struct. This makes it easier to share Linux code for Mac64. Keeps the scheme from ce8e803 of storing a pointer to the base of os_local_state_t in TLS slot 6. This is indirection we don't need with the entire os_local_state_t struct in TLS but it is not clear we can take that many TLS slots for large applications, so I'm leaving this mixture until we're sure which direction to go in. Disables the options -mangle_app_seg and -safe_read_tls_init for Mac64. Issue: #1568, #1979
derekbruening
added a commit
that referenced
this issue
Sep 14, 2019
Uses pthread_key_create() to allocate enough contiguous and aligned TLS slots to fit our os_local_state_t struct. This makes it easier to share Linux code for Mac64. Keeps the scheme from ce8e803 of storing a pointer to the base of os_local_state_t in TLS slot 6. This is indirection we don't need with the entire os_local_state_t struct in TLS but it is not clear we can take that many TLS slots for large applications, so I'm leaving this mixture until we're sure which direction to go in. Disables the options -mangle_app_seg and -safe_read_tls_init for Mac64. Issue: #1568, #1979
derekbruening
added a commit
that referenced
this issue
Sep 16, 2019
Fixes several issues to get small 64-bit apps to run on Mojave:
+ Fixes library discovery problems with the dyld shared cache.
On Mojave, SYS_shared_region_check_np returns the whole lib region,
while the existing code assumed its return value was the shared
__LINKEDIT segment.
+ Demotes a DOCHECK at level 1 in check_thread_vm_area to 3 for
Mac64 as it shows up as a large performance hit apparently due
to slow memory queries.
+ Segment fixes: fixes two issues hit due to DR and lib segments being
equal: a bb building assert for -no_mangle_app_seg and a
dr_insert_get_seg_base() assert.
+ Attempts to reduce overhead in memquery_from_os() where it needs extra
iterations to find the surrounding bounds of a free area, by walking
backward instead of starting from 0.
+ Uses SYS_stat64 and SYS_fstat64 for stat syscalls.
+ Basic signal support:
+ dynamorio_sigreturn() syscall number.
+ master_signal_handler() setup to invoke sigreturn with the proper arguments.
+ Accessing the frame within master_signal_handler_C().
+ No support yet for AVX512 but put some pieces in place and comments for
future work.
+ fixup_rtframe_pointers() and copy_frame_to_stack().
+ execute_handler_from_dispatch(): set up arguments for app handler,
including the style.
+ handle_sigreturn(): obtain signal number from siginfo.
Issue: #1979
derekbruening
added a commit
that referenced
this issue
Sep 16, 2019
Fixes several issues to get small 64-bit apps to run on Mojave:
+ Fixes library discovery problems with the dyld shared cache.
On Mojave, SYS_shared_region_check_np returns the whole lib region,
while the existing code assumed its return value was the shared
__LINKEDIT segment.
+ Demotes a DOCHECK at level 1 in check_thread_vm_area to 3 for
Mac64 as it shows up as a large performance hit apparently due
to slow memory queries.
+ Segment fixes: fixes two issues hit due to DR and lib segments being
equal: a bb building assert for -no_mangle_app_seg and a
dr_insert_get_seg_base() assert.
+ Attempts to reduce overhead in memquery_from_os() where it needs extra
iterations to find the surrounding bounds of a free area, by walking
backward instead of starting from 0.
+ Uses SYS_stat64 and SYS_fstat64 for stat syscalls.
+ Basic signal support:
+ dynamorio_sigreturn() syscall number.
+ master_signal_handler() setup to invoke sigreturn with the proper arguments.
+ Accessing the frame within master_signal_handler_C().
+ No support yet for AVX512 but put some pieces in place and comments for
future work.
+ fixup_rtframe_pointers() and copy_frame_to_stack().
+ execute_handler_from_dispatch(): set up arguments for app handler,
including the style.
+ handle_sigreturn(): obtain signal number from siginfo.
Issue: #1979
derekbruening
added a commit
that referenced
this issue
Sep 16, 2019
Adds 64-bit Mach-O support to drsyms, which consists simply of looking for LC_SEGMENT_64 in addition to LC_SEGMENT in drsyms_macho.c and in a repatched version of libelftc. Issue: #1979
derekbruening
added a commit
that referenced
this issue
Sep 16, 2019
Adds 64-bit Mach-O support to drsyms, which consists simply of looking for LC_SEGMENT_64 in addition to LC_SEGMENT in drsyms_macho.c and in a repatched version of libelftc. Issue: #1979
|
I can now run small apps, including simple signal use and including with clients (tested with memtrace_simple), on Mojave. Threads are still not working: somehow our bsdthread_create interception is failing. |
derekbruening
added a commit
that referenced
this issue
Sep 17, 2019
When querying into a submap, we can go straight to the middle, avoiding overhead of tons of syscalls querying from the start. And when we walk backward finding the prior on a free, a smaller starting step is a win for common queries. This drops Dr. Memory from taking 25s (!) on a tiny app down to 8s. Most of the 8s is still memqueries on the leak scan but that's for another day. Issue: #1979
derekbruening
added a commit
that referenced
this issue
Sep 17, 2019
When querying into a submap, we can go straight to the middle, avoiding overhead of tons of syscalls querying from the start. And when we walk backward finding the prior on a free, a smaller starting step is a win for common queries. This drops Dr. Memory from taking 25s (!) on a tiny app down to 8s. Most of the 8s is still memqueries on the leak scan but that's for another day. Issue: #1979
derekbruening
added a commit
that referenced
this issue
Feb 13, 2020
We no longer support 32-bit Mac as it does not really exist anymore. Issue: #1979
|
We're going to abandon 32-bit Mac support since Apple doesn't support it anymore. To make progress I will probably remove those tests from the list for now. |
|
#1815 covers fixing the failing tests. |
derekbruening
added a commit
that referenced
this issue
Feb 13, 2020
|
There is a non-deterministic assert hitting 38 of the 45 tests: https://travis-ci.com/DynamoRIO/dynamorio/jobs/287035442. I can't reproduce it locally. It is not the start==end case (xref #4031) b/c I ran a build with that fixed and it still fired: |
This was referenced Feb 14, 2020
derekbruening
added a commit
that referenced
this issue
Feb 14, 2020
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Build fails on Mac OS X,
makeoutput follows.The text was updated successfully, but these errors were encountered: