Port signals to Windows

[malte-v]

This issue is about porting Signal to Windows and therefore making it possible to implement other relevant features like Process.

Short introduction (moved from #5430):

Microsft doc about signals
I already looked at this page, and I think these signals are only there because they are part of the C standard library. From wikipedia:

The C standard defines only 6 signals. They are all defined in signal.h header (csignal header in C++):

SIGABRT - "abort", abnormal termination.
SIGFPE - floating point exception.
SIGILL - "illegal", invalid instruction.
SIGINT - "interrupt", interactive attention request sent to the program.
SIGSEGV - "segmentation violation", invalid memory access.
SIGTERM - "terminate", termination request sent to the program.

Additional signals may be specified in the signal.h header by the implementation. For example, Unix and Unix-like operating systems (such as Linux) define more than 15 additional signals; see Unix signal.

And these 6 signals are the only ones "supported" by Windows.

There is already an issue about this topic on the Node.js repo.
sam-github commented there:

[...] Windows as an O/S has absolutely no signal support. [...]

If SIGTERM was supported on Windows, you could listen for it and receive it... but you cannot. You can listen..., but you'll never receive it.

And I guess this is exactly the case. You have a function for listening to signals (signal()), but there is nothing like kill() for sending signals on Windows.

---

So this is about finding replacement functions to "fake" signals on Windows. I already found some replacements for signals one might want to send:

• SIGTERM: SendMessage(WM_CLOSE) <- this is also what taskkill.exe uses; works for windowed processes
• SIGKILL: TerminateProcess() <- used by taskkill.exe /f; works for windowed and console processes
• SIGINT: GenerateConsoleCtrlEvent() <- works for console processes, simulates Ctrl + C

List of UNIX signals: https://en.wikipedia.org/wiki/Signal_(IPC)
Another microsoft doc that mentions replacements for UNIX signals: https://docs.microsoft.com/en-us/previous-versions/ms811896(v=msdn.10)#signals-and-signal-handling

[malte-v]

Possible replacements for SIGSTOP and SIGCONT:

• SIGSTOP: DebugActiveProcess() (is actually for enabling a debugger to attach to the process, but also stops the process)
• SIGCONT: DebugActiveProcessStop() (same thing as above; resumes the process)

[sancarn]

Bear in mind, WM_CLOSE is a windows message and thus does not work for console applications as far as I know.

[malte-v]

Good news: WM_CLOSE, TerminateProcess and DebugActiveProcess work like a charm. I wrote a simple C++ console app for testing purposes: https://github.com/maltevoos/SignalFaker

Bad news: GenerateConsoleCtrlEvent doesn't work at all. It even says in the docs:

Sends a specified signal to a console process group that shares the console associated with the calling process.

Probably should have looked at this closer.

@sancarn I know this only works for windowed processes, but considering that it works very well for those and there is no real alternative that works for any process (as far as I know) I think this is the way to go. Of course we need to make it clear in the docs that SIGTERM doesn't work for console processes on Windows. Or we could call our SIGINT replacement instead when the user tries to send a SIGTERM to a console app.

[sancarn]

@maltevoos If so then you might consider using WM_QUIT. I personally don't know the difference between most of these signals to be honest (not a native linux user... 😛 )

[malte-v]

@sancarn So what exactly is the difference between WM_CLOSE and WM_QUIT and what makes one better? Btw, a short explanation of the signals I mentioned:

• SIGKILL is like taskkill /f
• SIGTERM is like taskkill without /f
• SIGSTOP suspends the process
• SIGCONT resumes the process
• SIGINT is like Ctrl-C

[sancarn]

WM_CLOSE is sent to the window when the "X" is pressed on a window.

Turns out that I was mistaken and WM_QUIT is not related to the window. The message indicates that a thread should quit. But it is only called from within the thread itself using PostQuitMessage(). It's used to tell windows that the Message loop should be stopped. It doesn't look like you can easily quit a remotely running thread with it at least.

Some docs on terminating applications

[malte-v]

I think we should stick with WM_CLOSE because this is what taskkill uses (at least multiple stackoverflow posts say so).
Also I can confirm that WM_QUIT does nothing when sending it to a window.

[sancarn]

Are you sure GenerateConsoleCtrlEvent() doesn't work?

    // It's impossible to be attached to 2 consoles at the same time,
    // so release the current one.
    FreeConsole();
 
    // This does not require the console window to be visible.
    if (AttachConsole(pid))
    {
        // Disable Ctrl-C handling for our program
        SetConsoleCtrlHandler(null, true);
        GenerateConsoleCtrlEvent(CtrlTypes.CTRL_C_EVENT, 0);
 
        // Must wait here. If we don't and re-enable Ctrl-C
        // handling below too fast, we might terminate ourselves.
        Thread.Sleep(2000);
 
        FreeConsole();
 
        // Re-enable Ctrl-C handling or any subsequently started
        // programs will inherit the disabled state.
        SetConsoleCtrlHandler(null, false);
    }

or for the current process use AllocConsole instead of AttachConsole.

Edit: Also found an article about it which provides full example.

[rdp]

https://github.com/jruby/jruby/blob/c3385b8feac7c05675a5f430b0d9b79f0edc5d37/core/src/main/java/org/jruby/RubyProcess.java#L1286 is how jruby does it, well for 0 and 9 anyway.

0       liveness
 1       HUP (hang up)
 2       INT (interrupt)
 3       QUIT (quit)
 6       ABRT (abort)
 9       KILL (non-catchable, non-ignorable kill)
 14      ALRM (alarm clock)
 15      TERM (software termination signal)

And here's how ruby does it https://github.com/ruby/ruby/blob/6a65f2b1e479f268489b51a004b6c153c634c68a/win32/win32.c#L4881 0,2,9

Maybe 15 could be WM_CLOSE if it's windowed, else OpenProcess(PROCESS_TERMINATE | PROCESS_QUERY_INFORMATION, FALSE, (DWORD)pid);
?
FWIW :)

[malte-v]

@sancarn That looks pretty promising indeed. Will try it out later today :)

@rdp OpenProcess just returns a handle to the process object with the given access rights so that would be suitable to check whether a process exists (signal 0).
We already have 0 and 9 anyway. But do you understand ruby's implementation for SIGINT?

Edit: Apparently the PID != 0 check Ruby uses ensures that when a PID is specified a Ctrl-Break event is generated instead of a Ctrl-C event. It says so in the comment:

/* CTRL+C signal cannot be generated for process groups.
* Instead, we use CTRL+BREAK signal. */

But AFAIK the Ctrl-C event should work just fine in any case so I'll try that first.

[rdp]

My hunch is they always send ctrl+break so that it works for "both pids and process groups" in case they happen to be targeting a process group? Kind of weird...

Also, appears one needs to call AttachConsole before sending the ctrl+break but...then you lose your current console, not to mention temporarily losing your own ctrl+c handler (see "cons" listed here https://stackoverflow.com/questions/813086/can-i-send-a-ctrl-c-sigint-to-an-application-on-windows ). Maybe we should just shell out to taskkill ... ? :/

[sancarn]

Also, appears one needs to call AttachConsole before sending the ctrl+break but...then you lose your current console, not to mention temporarily losing your own ctrl+c handler

@rdp See #7339 (comment)

[malte-v]

Maybe we should just shell out to taskkill ... ? :/

@rdp taskkill uses the window message WM_CLOSE so that won't work for console processes. If I try to kill a console process like node.exe it prints out that the process could not be terminated and I have to use taskkill /f /im node.exe. That on the other hand is not what we're looking for because it abruptly terminates the process like SIGKILL.

[rdp]

@sancarn I think one of the "cons" is that after running code like that you would no longer be bound to your initial console (as in...printf would now go no where...)? @maltevoos dang...wonder if there is any built-in way to "shell out" to something that sends a ctrl+c signal? I couldn't find any...I guess we could tell people "if you want this to work, you have to bundle it with this helper exe" :|

[rdp]

Unrelated to the standard linux signals, I wonder if there's a way to add handlers for other windows signals, like CTRL_LOGOFF_EVENT https://docs.microsoft.com/en-us/windows/console/handlerroutine or maybe it's already possible dunno. Might be useful long term anyway, FWIW :)

[malte-v]

@rdp I agree. We can't accept losing our console window just for terminating another process. Additionally, GenerateConsoleCtrlEvent doesn't do the same as a real Ctrl-C user input. For example when Ctrl-C-ing node, it always prompts the user to press Ctrl-C again. When using GenerateConsoleCtrlEvent, it prints out "^C", but node stops immediately.

[neatorobito]

I meant to comment on this issue a few days ago, but got bogged down. I had been looking into this because I wanted to help port Signal. I am by no means an expert on this stuff, so if I get something wrong or I'm way off base here, please let me know. Most of the stuff mentioned above should work, so I feel like we're on the right track. We can use abort() for SIGABORT.

My one suggestion is that we provide support only for the the signals that Windows itself "supports" based on the latest docs.

SIGABRT, SIGFPE, SIGILL, SIGINT, SIGSEGV, SIGTERM, SIGBREAK, CTRL_C_EVENT and CTRL_BREAK_EVENT .

Windows uses Structured Exception Handling, and so we can't really catch stuff like SIGFPE and SIGSEGV unless we wrap main in the __try__/__except__ syntax and then create a handler to raise the correct POSIX signal from the corresponding Windows exception. There's also Vectored Exception Handling which would allow us to just setup handlers, but it's not available on 64-bit Windows. Maybe those are outside of the purview of Signal but they aren't handled right now on Windows. This book has some more information on that.

If we take the route of only supporting the above signals, we can be very clear about this in the documentation and set expectations early. The Windows story for signals across newer languages seems shaky. It isn't clear what exactly Go implements for Windows. It seems to only implement the CTRL-C stuff. It does not implement SIGINT for Windows.

Rust doesn't seem to implement anything for Windows in the standard library. The tutorial suggests using a 3rd party library, and there's an RFC to add signal handling to the stdlib, but it's still open, and the latest reply suggests using a 3rd party library.

Python handles things like I've mentioned above (and this seems like the sanest approach). It's much less work than trying to find replacements for all the UNIX signals, especially when we can't necessarily guarantee that they'll always behave the same way (through changing system APIs, etc.) It's less maintenance going forward, and I'd argue that the Python signal handling presents a more concise and coherent API surface for someone using the language.

Edit: Wording.

[malte-v]

@markrjr Catching signals on Windows works the same way as on UNIX, namely through signal(). Our main problem is sending signals to other processes. On UNIX we have kill() where we just need to specify the process id and the signal we want to send. On Windows, such a thing doesn't exist (at least from my understanding) so we need to find replacements. Apart from that Windows doesn't seem to use its signals that much, e.g. when closing a window, there is a windows-specific window message being sent rather than a SIGTERM signal which makes listening to signals quite pointless (see first post).

Edit: Apparently Windows does use the signals it supports (like SIGINT or SIGABRT). Looks like I was mistaken there.

[malte-v]

@markrjr The Python devs only did the easy part, namely registering signal handlers for the signals supported by windows when the user calls signal.signal(). When it comes to sending signals (what this issue is mainly about), Pythons implementation is rather poor:

Send signal sig to the process pid. Constants for the specific signals available on the host platform are defined in the signal module.
Windows: The signal.CTRL_C_EVENT and signal.CTRL_BREAK_EVENT signals are special signals which can only be sent to console processes which share a common console window, e.g., some subprocesses. Any other value for sig will cause the process to be unconditionally killed by the TerminateProcess API, and the exit code will be set to sig. The Windows version of kill() additionally takes process handles to be killed.

Why Python's way of doing this is bad:

• signal.CTRL_C_EVENT only works when the target process and the calling process share the same console.
• No matter what signal you send, the target process will be unconditionally killed by TerminateProcess(). Even if you send something like SIGCONT (which is for resuming a suspended process), the target process will be unconditionally killed. It's like every signal was replaced by SIGKILL. This is just confusing for people working with Python.
• Minor thing: The windows version of os.kill takes a HANDLE to the process, which is unnecessarily
  complicated.

Sadly, most languages with signal "support" on Windows do it this way. Crystal should do better 🥇

[sancarn]

@rdp > I think one of the "cons" is that after running code like that you would no longer be bound to your initial console (as in...printf would now go no where...)?

I don't think that is true, because I assume Crystal will implement echo etc by writing to the STDOUT pipe directly but may be mistaken. But even if it were, you can always call AllocConsole afterwards to re-join the existing processes console.

[malte-v]

@sancarn I tested it yesterday. It doesn't work.
Edit: Maybe I missed the AllocConsole(). I'll try it again and if it works, we could see how low we can go with the Sleep duration.

Edit 2: I tested it again with AllocConsole but this opens an entirely new console window which looks very weird. Another approach that came to my mind:

• Spawn a dummy process in the current console and capture its PID
• Go to the console of the target process and call GenerateConsoleCtrlEvent
• Go back to the home console using AttachConsole with the pid of the dummy process
• Kill the dummy process

Another nice thing about this would be that we probably don't need a Sleep call.

[asterite]

I didn't read the entire conversation, but in my opinion signals in Crystal should be done like in Go: only SIGKILL is supported in Windows, trying to use SIGINT gives an error in Windows, and other signals are only available in Unix.

References:

• https://golang.org/pkg/os/signal/
• https://golang.org/pkg/os/#Signal

[malte-v]

@asterite I disagree. In my opinion we should support any signal that can be elegantly replaced with a WinApi call.
Currently we have:

• SIGTERM: SendMessage(WM_CLOSE) for GUI processes, TerminateProcess for console processes
• SIGSTOP: DebugActiveProcess
• SIGCONT: DebugActiveProcessStop
• SIGKILL: TerminateProcess

The functions above are tested and (almost) work like real signals on UNIX.
Why should we only support SIGKILL/TerminateProcess if we have good replacements for other signals too? I would be fine with just supporting the four signals above since these are the ones you would really want to use.

[Sija]

@maltevoos There's an argument about maintenance overhead in supporting these replacements - how big would it be (LOC-wise), what's the complexity of OS-support matrix, how many edge-cases (like SIGTERM) we need to handle - all in all, is it worth it?

I'd go for sure for all of the cases we can support transparently, and weighted the rest against above argument.

[malte-v]

@Sija IMO it is worth it:

• SIGTERM: around 30 - 40 lines
• SIGSTOP: 2 lines
• SIGCONT: literally 1 line
• SIGKILL: 3 lines

(lines of naive C++ code, each signal only takes a PID like on UNIX)
See https://github.com/maltevoos/SignalFaker

Probably there is some more work to do if we want good error handling etc.

[neatorobito]

I misunderstood, the blurb about structured exception handling is only important with OS -> Process communication, but since we're looking for IPC then we'd probably want to use the Messages API.

@maltevoos It looks like you've already experimented with the Messages API for SIGTERM by sending the WM_CLOSE message. We could send arbitrary messages for the other signals (like SIGILL or SIGFPE and once received, raise the offending signal. We would want to post them to the thread. See PostThreadMessageW. Those messages then would only be understood by other Crystal programs, though. That's likely why Python developers just decided to use TerminateProcess for everything.

[neatorobito]

We could possibly use feraiseexcept for SIGFPE.

[malte-v]

@markrjr I don't think we should invest time into this. I mean, what's the point of having two Crystal processes (that must be GUI processes) running at the same time and communicating to each other? We should keep focusing on sending signals to external processes.

[rdp]

It's not much work/overhead, and makes it so you don't have to remember/use a "different way to kill process based on the current OS" as it were...I like maltevoos' roadmap (though who uses SIGCONT etc. in real life, but maybe still useful). I believe you can catch SIGFPE etc. but you can't "send" it to another process AFAICT.

[ysbaddaden]

The only signal values guaranteed to be present in the os package on all systems are os.Interrupt (send the process an interrupt) and os.Kill (force the process to exit).

AFAIK only signals that exist on the target are defined. I.e. Signal's enum should only contain valid signals, not fake ones. On POSIX systems you get all signals, on Windows you have the few supported ones. Overall, only Interrupt and Kill are guaranteed to be available. Using other signals isn't portable.

IMO this is a good compromise. It breaks the "code must compile on all targets" but using signals shouldn't be recommended, and usually not needed.

[stronny]

Overall, only Interrupt and Kill are guaranteed to be available.
Using other signals isn't portable.

So how do you portably use even these two? Is it possible to kill(pid, SIGKILL) on Windows? Is it possible to trap(SIGINT)?

[oprypin]

For the record, the value SIGKILL doesn't exist anywhere in Windows API.

[oprypin]

trap(SIGINT) is possible but I'm not sure exactly what it does.

"Sending a signal" in general is not a thing on Windows.

[stronny]

As was stated previously:

If SIGTERM was supported on Windows, you could listen for it and receive it... but you cannot.
You can listen..., but you'll never receive it.

And I guess this is exactly the case. You have a function for listening to signals (signal()),
but there is nothing like kill() for sending signals on Windows.

[straight-shoota]

So how do you portably use even these two?

That's already answered in the issue description. And the same thing as Go does: When win32 Process#signal implementation receives SIGKILL it calls TerminateProcess.
SendMessage(WM_CLOSE) for SIGTERM probably isn't worth implementing when it only works for windowed processes.
I'm not sure about SIGINT in Go it's not implemented, but a TODO in the code =)

So for a MVP I'd suggest to only implement custom handling of SIGKILL on windows for now.

[oprypin]

So for a MVP I'd suggest to only implement custom handling of SIGKILL on windows for now.

That's the obvious part.

So then, what should the value of Signal::KILL be on Windows?

[stronny]

Why not introduce like Process.terminate() rather than emulate one special posix signal with something that has nothing to do with signals?

[oprypin]

We are indeed moving in that direction, so perhaps we wouldn't need to fake-implement signals, but this is not a universal solution.

[oprypin]

(terminate was recently added and now is being decoupled from signals, look for it in https://github.com/crystal-lang/crystal/pull/9035/files)

[RX14]

I really like what go does, in that it defines a list of signals, and all the signal values are always there on every OS, even if using them is an error. That go has the same guarantees that compiling always works on every OS and any failures happen at runtime is a really good sign.

But only providing 2 enum values is not feasible in crystal. It's a simple extension of go's approach to add more signal values than just two.

[oprypin]

and all the signal values are always there on every OS, even if using them is an error

-- all two of them - Interrupt and Kill.

I'm not sure how commendable that is

[ysbaddaden]

@RX14 To use signals you must use values from the syscall package, which only defines some signals for each target.

For example, syscall.SIGHUP is available for both GOOS=linux and GOOS=windows, but syscall.SIGTTOU will only compile for linux targets. On windows you get a compilation error:

$ GOOS=linux bin/go build foo.go

$ GOOS=windows bin/go build foo.go
./foo.go:14:28: undefined: syscall.SIGTTOU

I.e. using signals easily breaks Go programs portability. It's a good demonstration that the "compiles everywhere" policy is meant as a goal that shouldn't be enforced blindly —let developers shoot themselves in some corners, but make it explicit.

In Crystal terminology, this means that there isn't a Signal enum, and to use signals, you must use LibC constants. For example LibC::SIGTTOU and handle portability, for example with {% if flag?(:unix) %} wrappers (if you care about it). I'd be fine with this.

[RX14]

I'm fine with that general approach, since it's obvious that LibC and Crystal::System are unstable APIs. Putting the constants under LibC or under Crystal::System I don't mind either.

In fact I'd like to move LibC under Crystal...

[ysbaddaden]

Neither Crystal nor Crystal::System namespaces should be accessed outside of core/stdlib. Putting signal constants there... undermines the whole point of these namespaces.

Putting LibC is also nonsensical (especially if we're meant to access signal constants from LibC). It's not unstable. It's unsafe and not portable but the C API is very stable. Let's remember that easy interaction with C API is one of the very powerful feature of Crystal...

[straight-shoota]

@ysbaddaden This goes a bit off topic, but just to clarify: The Crystal namespace actually provides some features that are definitely intended for usage outside stdlib. Most inside that namespace is not intended as public API and thus nodoc, though. Especially everything in Crystal::System.

[RX14]

Putting LibC is also nonsensical (especially if we're meant to access signal constants from LibC). It's not unstable.

LibC certainly is unstable - it's not identical between platforms and some future platforms may not even have it. Symbols from LibC may be removed between releases. I don't want to make any guarantees about stability on LibC at all, and as such I'd like it to be moved to Crystal. Then we can simply say "anything under Crystal that's not documented is subject to change"

[oprypin]

That suggestion could be nice, it'd expose usages of LibC which are outside Crystal::System, as only those usages would need to be prefixed Crystal::LibC.

Still, this wouldn't get us closer to the solution of the problem at hand.

[mdwagner]

Not sure it helps, but here's what Rust says about Signals.
https://rust-cli.github.io/book/in-depth/signals.html#differences-between-operating-systems

[HertzDevil]

IMO we should aim for implementating the use cases of Signal in some other platform-independent API, not the opposite. The Crystal standard library does not have to port an POSIX API for the sake of doing it when not all of the supported platforms are POSIX-compliant; otherwise, we'd simply be reinventing Cygwin's wheel. The most common use cases of Signal seem to be:

Terminating a process

Process#terminate should use TerminateProcess. No further guarantees should be made about whether the termination is graceful. Crystal code that sends SIGTERM or SIGKILL directly should use Process#terminate instead.

Handling Signal::INT

Interrupts can be trapped by SetConsoleCtrlHandler. We could have a new method Process.on_interrupt that uses this on Windows and forwards to Signal::INT on Unix-like systems. More details in #12224.

Sending them might also be done in GenerateConsoleCtrlEvent as mentioned in the OP; not sure if we need that.

Handling segmentation faults and stack overflows

This is done in #11570 already via Structured Exception Handling.

Detecting exit reason via Process::Status

The compiler's run command, for example, inspects the exit status of the temporary executable after running it. This part uses Process::Status#exit_signal, which does not exist on Windows at all, but the exit status on Windows is nonetheless sufficient to determine common causes of process termination, such as interrupt requests and access violations. More details in #12284.

---

I don't think the rest of Signal's functionality sees much utility on Windows, especially since the advent of WSL. If necessary, we could implement #6170 instead, since the MSYS runtime must emulate POSIX signals, and probably does it well enough (unlike, for example, WASI). Eventually require "signal" on Windows MSVC could be either a no-op or a macro raise.