Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix use-after-free bugs in debuginfo #45016

Merged
merged 24 commits into from
Apr 19, 2022
Merged

Fix use-after-free bugs in debuginfo #45016

merged 24 commits into from
Apr 19, 2022

Conversation

pchintalapudi
Copy link
Member

@pchintalapudi pchintalapudi commented Apr 18, 2022

Fixes #44562
Fixes #44947

This is achieved by making the debug info global variables a part of the JIT engine, which should leak their memory with our JIT rather than being destroyed on process exit.

We also do the following as part of the move:

  1. Consolidate all of the debug info global variables into one large struct
    1. This sequences all of the destructor orderings, and ties their lifetimes together for easier analysis.
  2. Make debug info threadsafe
    1. This is achieved by locking around the data structures within the debug info registry

@@ -181,212 +438,11 @@ static void create_PRUNTIME_FUNCTION(uint8_t *Code, size_t Size, StringRef fnnam
}
#endif

struct revcomp {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We now use std::greater<size_t> instead of revcomp

@DilumAluthge
Copy link
Member

@pchintalapudi Does this fix #44562?

@pchintalapudi
Copy link
Member Author

I'm hoping for that, but I want to get results for macos before confirming that.

@DilumAluthge
Copy link
Member

DilumAluthge commented Apr 18, 2022

Once you have the builders and testers passing, we can check the Buildkite CI logs for both the macOS x86_64 tester and the macOS Apple Silicon tester, and see if there are still any segfaults in the logs.

@DilumAluthge
Copy link
Member

DilumAluthge commented Apr 18, 2022

Looks like the Windows builders are failing with: /cygdrive/c/buildbot/worker/package_win32/build/src/debuginfo.cpp:246:9: error: 'create_PRUNTIME_FUNCTION' was not declared in this scope

@pchintalapudi
Copy link
Member Author

I'll tackle that and the analyzegc failures once the mac tester finishes, just to see if it works.

@DilumAluthge
Copy link
Member

You'll find the macOS x86_64 tester in the Test group - look for the job named test x86_64-apple-darwin.

The macOS Apple Silicon tester is in the Allow Fail group - look for the job named test aarch64-apple-darwin.

@pchintalapudi
Copy link
Member Author

Macos tests look successful for 07eb7bf

@pchintalapudi pchintalapudi marked this pull request as ready for review April 18, 2022 03:41
@DilumAluthge DilumAluthge changed the title Try to fix use-after-free bugs in debuginfo Fix use-after-free bugs in debuginfo Apr 18, 2022
@DilumAluthge DilumAluthge added bugfix This change fixes an existing bug system:mac Affects only macOS labels Apr 18, 2022
@DilumAluthge
Copy link
Member

@pchintalapudi
Copy link
Member Author

Looks like Mac+aarch64 is also clean: https://buildkite.com/julialang/julia-master/builds/11193#7749e3f8-a7fd-4f3f-bed1-30a0496e6540

src/jitlayers.cpp Outdated Show resolved Hide resolved
@vtjnash
Copy link
Member

vtjnash commented Apr 18, 2022

This seems a good approach. Thanks!

@pchintalapudi
Copy link
Member Author

@vtjnash Do you have suggestions on how to fix the analyzegc failures? I tried to mark getDebugInfoRegistry with JL_NOTSAFEPOINT, but that didn't stop the analyzer from complaining about the call.

@vtjnash
Copy link
Member

vtjnash commented Apr 18, 2022

getDebugInfoRegistry should be marked JL_NOTSAFEPOINT, but you might need to teach the analyzer how to find properties of CxxMember function calls, since finding their decl site is different from finding the decl target for a basic C function call.

@pchintalapudi
Copy link
Member Author

Now that the analyzegc check is passing, I think we can merge this into master if tests pass and no segfaults are observed on macos?

@pchintalapudi pchintalapudi added the merge me PR is reviewed. Merge when all tests are passing label Apr 18, 2022
@pchintalapudi
Copy link
Member Author

@DilumAluthge do we need to rerun the windows testers? I haven't seen these kinds of errors before.

@DilumAluthge
Copy link
Member

Let me try rebooting the Windows VMs.

@DilumAluthge
Copy link
Member

@vtjnash If CI passes, is this good to merge from your point of view?

@vtjnash
Copy link
Member

vtjnash commented Apr 19, 2022

yep, seemed to be good to go

@DilumAluthge DilumAluthge merged commit 3fc6eac into master Apr 19, 2022
@DilumAluthge DilumAluthge deleted the pc/debuginfo branch April 19, 2022 07:02
@DilumAluthge DilumAluthge removed the merge me PR is reviewed. Merge when all tests are passing label Apr 19, 2022
@maleadt
Copy link
Member

maleadt commented May 5, 2022

Doesn't this need a backport? Otherwise we'll still have #44562 on 1.8. (cc @KristofferC)

@DilumAluthge
Copy link
Member

It would be great to get this backported.

@KristofferC KristofferC added the backport 1.8 Change should be backported to release-1.8 label May 5, 2022
@KristofferC KristofferC mentioned this pull request May 16, 2022
67 tasks
maleadt pushed a commit that referenced this pull request May 24, 2022
KristofferC pushed a commit that referenced this pull request May 25, 2022
@KristofferC KristofferC removed the backport 1.8 Change should be backported to release-1.8 label May 26, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bugfix This change fixes an existing bug system:mac Affects only macOS
Projects
None yet
5 participants