WeakKeyDict is unsound

[Keno]

This is reduced from #38712 (comment). See that analysis for what happens in the original case. The short version is that there is a window of opportunity where the only reference to a value is weak, so finalizers can get scheduled, and then then a root can be re-established which causes much confusion. The below code illustrates the issue. This will return a foo(-1) because of this issue, which is illegal. It should either return foo(1) or missing.

mutable struct foo
	i::Int
end
const armed = Ref{Bool}(true)
@noinline fwdab(a, b) = invoke(Base.isequal, Tuple{Any, WeakRef}, a, b)
function Base.isequal(a::foo, b::WeakRef)
	# This GC.gc() here simulates a GC during compilation in the original issue
	armed[] && GC.gc()
        armed[] = false
	fwdab(a, b)
end
Base.isequal(a::WeakRef, b::foo) = isequal(b, a)
Base.:(==)(a::foo, b::foo) = a.i == b.i
Base.hash(a::foo, u::UInt) = Base.hash(a.i, u)

function make_foo(wkd, i)
	f = foo(i)
	function fin(f)
		f.i = -1
	end
	finalizer(fin, f)
	f
end

@noinline mkfoo(wkd) = wkd[make_foo(wkd, 1)] = nothing
function bar()
	wkd = WeakKeyDict{Any, Nothing}()
	mkfoo(wkd)
	armed[] = true
	z = getkey(wkd, foo(1), missing)
end
bar()

[Keno]

If trying the reproducer, make sure to try after 59aedd1. Without that, the reproducer needs to be written somewhat differently.

[StefanKarpinski]

I can't reproduce on master (macOS).

[Keno]

Fails reliably for me on both linux and macOS master. What output of bar() do you see?

[StefanKarpinski]

missing

[StefanKarpinski]

Do I need to run it in a file?

[Keno]

No, I just paste it into the REPL and it fails. What commit are you on exactly?

[StefanKarpinski]

Tried it loaded from a file, same result.

[StefanKarpinski]

Maybe something specific about the macOS system you're testing on?

[Keno]

Shouldn't be, although it's somewhat sensitive to collection intervals. What happens if you runs bar twice and also what exact commit did you try?

[jmkuhn]

I get foo(-1) on macOS with commit 65382c7.

[KristofferC]

I also get foo(-1) on mac (every time).

[StefanKarpinski]

My bad — I was on a branch so when I did git pull and rebuilt Julia, I was not including 59aedd1. On master I get the same thing, so this seems like a robust reproducer after all. Sorry for the false alarm!