Behavior of the GC regarding finalization of objects at program termination

[dodexahedron]

Here's a fun one about dotnet that I only learned about in the last year or so, but which came up again today and I wanted to mention it, to raise awareness...

Dotnet 5 and up GC does not process remaining items in the finalizer queue at program termination.

Huh?

The GC, when it runs, does a whole bunch of stuff, but I'll quote this MS Learn document from the C# programming guide first:

In most cases, you can avoid writing a finalizer by using the System.Runtime.InteropServices.SafeHandle or derived classes to wrap any unmanaged handle.

Now why is that?

Well, the GC tries to directly free heap-allocated objects without a reference path back to the thread's root (well...actually a one-way path from a root to that leaf object - in the other direction, the GC doesn't care and nukes things anyway).
But, if a finalizer is declared on a type, the object automatically will live at least to gen 1 (goal is gen 0 or even no heap allocations, in the ideal case, if possible).

And why is that?

Having a finalizer, even if it is literally empty (~SomeClass {}), results in that object being added to the GC's finalizer queue, unless you have previously called GC.SuppressFinalize(this); on that instance.¹

You can manually invoke GC.Collect(), but that's nearly always an awful idea in production code. You also have no guarantee a finalizer on any given object instance will actually ever be called, even when the program terminates.

I'll say that again... When the program exits, whether "cleanly" or due to an unhandled exception, the GC's finalizer queue WILL NOT be processed, and any unmanaged resources or even managed IDisposable resources that weren't cleaned up properly are now in an undefined state.

Why should I care?

The impact of this is that handles/pointers to system objects can be leaked and still be in existence, even after the program exists. For something like a file stream, that's usually no big deal and windows will usually release the exclusive lock on the file after a time, if one exists.² Linux has no such protection, so it's an even bigger potential problem there, if/when native code is called and things aren't cleaned up properly. Either situation, at minimum, eats up memory or other resources, but can even be exploited by an attacker to elevate permissions at least to those of the original process owner of the leaked resources, but potentially to root or even system, depending on how it was launched and a whole bunch of other variables that all end up being moot if native resources are properly cleaned up (whether managed by built-in types, managed by SafeHandle wrappers, or unmanaged and only referenced via HANDLEs or other pointer-esque constructs.

This is a bad thing for any program to do, but it's especially bad for a library to do, because it's not the consumer's responsibility to clean up after the library, nor is it likely to be possible, especially without spending more effort than fixing it in the library's source would take, at minimum.

Hang on. But we have finalizers that don't do much. Isn't that bad?

Depends. For anything that owns an unmanaged resource or a managed resource that itself does not explicitly clean up without being told to by the library's code, anywhere from zero to all of the following could happen (and more), in any combination and any order, and potentially multiple times:

• Resources can be leaked, and some could even live until reboot or even across reboots, depending on the implementation of the resource being called.
  • We're dealing with potentially durable system objects that have the possibility of outliving the application, even if everything goes well, so it's pretty critical to clean things up properly, or we could end up causing stuck terminal sessions in random and not easily reproducible ways and times, among other undesirable behaviors that can last after the application terminates.
• Events can continue to be raised unexpectedly, which even means the instance may be ineligible for ANY GC activity, in certain situations
  • Events on other objects that the unreferenced object has subscribed to do not count and can be ignored (though the lifetime issues mentioned mean it's still a good idea to unsubscribe in Dispose())
  • Events published by this object that other objects have subscriptions to can keep the instance alive, as that is a valid path to root from a currently live object, making this leaf object not disconnected and thus not subject to GC.
  • Again, that is a ONE-WAY, INTRANSITIVE relationship, and only the instance being disposed is susceptible, via its published events. Just set them null in Dispose and all is well (but don't do that to the Disposed event until ALL subscribers are done)
• Any of the above makes it easier to hit other issues, including but not limited to:
  • Deadlocks on access to synchronized code that gets blocked by the invalid and unreferenced object.
  • Null pointer dereferences by unmanaged code (which won't be a NullReferenceException and could be exploited for arbitrary code execution)
  • Heap corruption both for the current process and possibly even others
  • Memory access violations potentially up to kernel space, which could cause a crash of the whole machine or myriad other bad and potentially dangerous consequences

There are plenty of other potential issues, but these are what occurred to me before I cut myself off. 😅

So what do we do about it?

In general, the recommended solution is pretty simple: Wrap access to/the pointers for unmanaged native resources (so, typically HANDLE/nint/nuint) in a System.Threading.SafeHandle-derived type, which enables idiomatic C# semantics, like using blocks and the IDisposable interface. Those types do some of the stuff that is sometimes not done (or not even realized that it needs to be done), such as reference counting and that sort of thing, to ensure that release and disposal of the managed object results in full, complete, and at least relatively certain freeing of the native unmanaged resources.

Wrapping them like that also helps the compiler help us by enabling static analysis, thanks to the IDisposable interface and associated annotations, which it can't do for nint or others.

It also provides a clear and clean boundary between our code and native interop, which we did already have to a degree, but only insofar as it was just in its own class that directly contained the PInvoke calls. The new way provides the wrapping as mentioned above, which can also lead to us being able to unify the internal API around a common interface, to keep features/functionality equivalent for all platforms. The wrappers can be turned into something that the various driver classes call via a common interface, injecting themselves into the wrapper method calls when and where necessary.

That also enables various platform-specific optimizations, but mostly allows code not needed by the current platform to be excluded.

Consider:

• An interface defined for IConsoleDriver
• Specific console drivers extracted out to their own projects, both de-coupling them and making TG more extensible for anyone who wants to provide their own driver implementation.
• The assembly for Terminal.Gui could be provided in several flavors:
  • A Full flavor that is essentially as it is now, with all code compiled in statically, which anyone can continue to use on any platform, as always
  • A minimal flavor, which is just Terminal.Gui without any compiled-in drivers
  • A Linux version compiled with only the linux-relevant drivers
  • A Windows version, compiled with only the Windows console driver
• Any flavor would still first look for user-provided assemblies to support arbitrary extensibility of console drivers and then load them into the appdomain.
• Use of an interface inside Terminal.Gui itself will improve our own code in several ways, not the least of which is the previously-mentioned de-coupling, better testability, internal consistency, and easier maintenance and improvements for the future.

That sounds like a lot...

It's not so bad, really. The wrappers end up looking a lot like what we already have, and most usages only need to update types or the static class they call methods from and be sure the handles are disposed (which just replaces the CloseHandle calls, mostly). Also, it's closely linked to the work I'm doing for the events, and naturally fits in that body of work, so I'll be doing it during that.

Cool, so what can we expect?

I'm tracking work on that and the various other issues related to events on my fork, via this project

Why isn't this just an issue?

Because there is already an issue for the parent project, and the concepts are just about the language itself, even if it applies to bits and pieces of our code, and this would be noise on any of those.

Footnotes

1. Big caveat there. If you don't actually properly release resources before you do that, you WILL leak handles/other resources. ↩

2. It does, but it is known to be broken and useless in the kernel and it can't be relied upon) ↩