Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

add end-user support for updating syscall #'s from pdb's #1848

Closed
derekbruening opened this issue Jan 20, 2016 · 2 comments
Closed

add end-user support for updating syscall #'s from pdb's #1848

derekbruening opened this issue Jan 20, 2016 · 2 comments
Assignees
@derekbruening
Labels
Hotlist-Release Maintainability OpSys-Windows Type-Feature

Comments

@derekbruening
Copy link
Contributor

derekbruening commented Jan 20, 2016 •
edited
Loading

The goal is to future-proof Dr. Memory: make it more adaptive to avoid requiring manual updates to fix breakages on each new Windows change. Xref #1826.

The plan is:

  • Detect unknown version by looking at particular syscall #'s (as we can't rely on PEB versions anymore): xref more adaptive new Windows version support dynamorio#1598
  • Create utility that downloads pdb's for the core dll's, does sthg like what winsysnums does, and comes up with new syscall numbers. We should be able to automate everything except for the usercall stuff.
  • Can we launch the helper process from our online client? Even if so, we'll need to cache the results, so we could ask the user to run the utility standalone?
  • Cache the results and load them in.

Things can still break if the syscall wrappers change (xref DynamoRIO/dynamorio#1854) or other things besides numbers change, but this would be an improvement and could help future-proof Dr. Memory.
@derekbruening derekbruening mentioned this issue Jan 20, 2016
Open
@derekbruening derekbruening mentioned this issue Jun 2, 2016
Closed
@derekbruening derekbruening mentioned this issue Aug 12, 2016
Closed
@derekbruening derekbruening mentioned this issue Aug 22, 2016
Closed
@derekbruening
Copy link
Contributor Author

derekbruening commented Aug 27, 2016 •
edited
Loading

#1908 gets us part of the way there: DrM now supports loading the numbers from a separate file. This issue is about automating generation of that file on the end-user's machine.

@derekbruening
Copy link
Contributor Author

We should remember to include drstrace and drltrace in the solution here and not limit it to drmemory (xref #1983).

@derekbruening derekbruening mentioned this issue Apr 25, 2017
Closed
@derekbruening derekbruening mentioned this issue Nov 24, 2017
Open
@derekbruening derekbruening self-assigned this Nov 25, 2017
@derekbruening derekbruening mentioned this issue May 18, 2018
Closed
@derekbruening derekbruening mentioned this issue Aug 29, 2018
Closed
@derekbruening derekbruening mentioned this issue Sep 23, 2018
Closed
@derekbruening derekbruening mentioned this issue Jan 14, 2019
Open
derekbruening added a commit that referenced this issue Feb 19, 2019
@derekbruening
i#1848 gensysnums: Fix x64 symbol fetch error
93ffe22 
Updates DR to 191c479 to fix an error in
drfront_fetch_module_symbols() when loading 64-bit symbols.

Issue: #1848
@derekbruening derekbruening mentioned this issue Feb 19, 2019
Merged
derekbruening added a commit that referenced this issue Feb 19, 2019
@derekbruening
i#1848 gensysnums: Fix x64 symbol fetch error (#2153)
8bcd285 
Updates DR to 191c479 to fix an error in
drfront_fetch_module_symbols() when loading 64-bit symbols.

Updates the Windows test asm line numbers for another
asm_defines shift.

Issue: #1848
derekbruening added a commit that referenced this issue Feb 24, 2019
@derekbruening
i#1848: auto-generate a syscall file on a new OS
45401e9 
Moves the default syscall file location into logs/symbols instead of the
bin/ directory, as we need a writable location.  Adds a new option
-syscall_number_path to allow specifying a custom location.

Adds functionality to drsyscall to fetch debug info for system libraries
and parse every symbol in each library looking for syscall wrappers.  Two
interfaces are added: drsys_find_sysnum_dlls() and
drsys_generate_sysnum_file().  Includes usercall identification support via
a list of target wrappers for key usercalls plus interpolation between
known numbers to include speculative results for all usercalls.  The
syscall wrapper parsing code is based on DR's winsysnums.c, expanded to
support 64-bit.

The resulting syscall file only supports the current machine, unlike the
general files we have posted for downloading.

Adds auto-triggering of the new functionality when an unknown OS version is
detected.  A special exit code is used, and when the frontend sees it, it
invokes the drsyscall functions for generating a syscall file.  If that is
successful, it re-launches the app.

Adds new options -vv and -vvv to support raising verbosity for the
drsyscall code invoked directly from the frontend.

Changes the existing mksystable.pl-based test to test the new auto-gen
functionality.

Fixes #1848
This was referenced Feb 24, 2019
Merged
Closed
derekbruening added a commit that referenced this issue Feb 24, 2019
@derekbruening
i#1848: auto-generate a syscall file on a new OS (#2155)
313de07 
Moves the default syscall file location into logs/symbols instead of the
bin/ directory, as we need a writable location.  Adds a new option
-syscall_number_path to allow specifying a custom location.

Adds functionality to drsyscall to fetch debug info for system libraries
and parse every symbol in each library looking for syscall wrappers.  Two
interfaces are added: drsys_find_sysnum_dlls() and
drsys_generate_sysnum_file().  Includes usercall identification support via
a list of target wrappers for key usercalls plus interpolation between
known numbers to include speculative results for all usercalls.  The
syscall wrapper parsing code is based on DR's winsysnums.c, expanded to
support 64-bit.

The resulting syscall file only supports the current machine, unlike the
general files we have posted for downloading.

Adds auto-triggering of the new functionality when an unknown OS version is
detected.  A special exit code is used, and when the frontend sees it, it
invokes the drsyscall functions for generating a syscall file.  If that is
successful, it re-launches the app.

Adds new options -vv and -vvv to support raising verbosity for the
drsyscall code invoked directly from the frontend.

Changes the existing mksystable.pl-based test to test the new auto-gen
functionality.

Fixes #1848
derekbruening added a commit that referenced this issue Feb 25, 2019
@derekbruening
i#2126 win10-1809: Fix 32-bit crash
80c14e0 
Updates DR to dbccde9d for two features:
+ DRi#3391: dynamically identify APC CONTEXT offset, to fix an assert in
  every app and crashes in apps with APC's on win10-1809.
+ The addition of drfront_set_verbose() which we now use here when
  -v* is passed for symbol fetching diagnostics.

Issue: #2126, #1848
@derekbruening derekbruening mentioned this issue Feb 25, 2019
Merged
derekbruening added a commit that referenced this issue Feb 25, 2019
@derekbruening
i#2126 win10-1809: Fix 32-bit crash (#2157)
e40f342 
Updates DR to dbccde9d for two features:
+ DRi#3391: dynamically identify APC CONTEXT offset, to fix an assert in
  every app and crashes in apps with APC's on win10-1809.
+ The addition of drfront_set_verbose() which we now use here when
  -v* is passed for symbol fetching diagnostics.

Issue: #2126, #1848
derekbruening added a commit that referenced this issue Feb 25, 2019
@derekbruening
i#2156: symbol fetch failures on Appveyor (#2158)
f7eb0ad 
Update DR to d11b7a99 for the drfront_fetch_module_symbols() workaround for
older dbghelp versions.

Remove the flaky markers for drstrace_unit_tests and syscall_file_gen as
these tests now work on Appveyor.

Adjust #1848's drsys_find_sysnum_libs() to check readability to skip dlls that do not exist on this OS version (e.g., win32u.dll)
Adjusts drsys_generate_sysnum_file() to avoid duplicates in interpolated usercalls.

Issue: #1848 
Fixes #2156
This was referenced Apr 2, 2019
Closed
Closed
@derekbruening derekbruening mentioned this issue Mar 9, 2020
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@derekbruening derekbruening

Labels
Hotlist-Release Maintainability OpSys-Windows Type-Feature
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@derekbruening