fatal: error thrown and no exception handler available. #34184

ghost · 2019-12-23T12:58:13Z

I'm getting this in a program employing Threads.@Spawn and Channel communication between the threads when ^C is pressed. I have an InterruptException catch block in the main thread, but when the program crashes, it never gets called (It should printstyled("\rUser interrupt—finishing up.\n", bold=true, color=202) if it were.)

Normally things crash, when they do, after a single ^C press, but this time things hung and I had to press ^C a second time to cause the crash:

^Cfatal: error thrown and no exception handler available.
InterruptException()

^C
sigatomic_end at ./c.jl:425 [inlined]
task_done_hook at ./task.jl:399
jl_apply at /Users/sabae/buildbot/worker/package_macos64/build/src/./julia.h:1631 [inlined]
jl_finish_task at /Users/sabae/buildbot/worker/package_macos64/build/src/task.c:181
start_task at /Users/sabae/buildbot/worker/package_macos64/build/src/task.c:669

julia> versioninfo()
Julia Version 1.3.0
Commit 46ce4d7933 (2019-11-26 06:09 UTC)
Platform Info:
  OS: macOS (x86_64-apple-darwin19.0.0)
  CPU: Intel(R) Core(TM) i5-4308U CPU @ 2.80GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-6.0.1 (ORCJIT, haswell)
Environment:
  JULIA_NUM_THREADS = 5

The text was updated successfully, but these errors were encountered:

JeffBezanson · 2019-12-23T18:34:45Z

Could you share some example code that exhibits this?

ghost · 2019-12-23T19:18:43Z

I tried to come up with a simple example with the overall structure of the bigger code that I will not share at this time yet (maybe in a month or so I'll put it on Zenodo with the corresponding article on arXiv), but those did not exhibit the problem. This is slowing that down as half of the time when I change parameter for a computation Julia crashes and it takes quite a bit of time to reload.

PS. I verified with --check-bounds=yes and @boundscheck that @inbounds shouldn't be the problem, so it seems quite likely to be a bug in Julia itself.

ssfrr · 2020-01-02T05:35:23Z

I just saw a similar symptom in some code I'm working on right now, but without using PackageCompiler.

Here's a MWE based on my code:

try
   t = Base.Threads.@spawn begin
      sleep(10)
      42
   end
   fetch(t)
catch
   wait(t)
end

Run the try block and press Ctrl-C. In my case the thing I'm @spawning is a ccall and I want to make sure that even if the user interrupts the code we still wait for the ccall to complete before moving on.

I'd expect the wait(t) would throw an UndefVarError because t isn't in scope.

The error I get is:

fatal: error thrown and no exception handler available.
InterruptException()
jl_mutex_unlock at /buildworker/worker/package_linux64/build/src/locks.h:156 [inlined]
jl_task_get_next at /buildworker/worker/package_linux64/build/src/partr.c:450
poptaskref at ./task.jl:660
wait at ./task.jl:667
task_done_hook at ./task.jl:401
_jl_invoke at /buildworker/worker/package_linux64/build/src/gf.c:2135 [inlined]
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2305
jl_apply at /buildworker/worker/package_linux64/build/src/julia.h:1631 [inlined]
jl_finish_task at /buildworker/worker/package_linux64/build/src/task.c:181
start_task at /buildworker/worker/package_linux64/build/src/task.c:669
unknown function (ip: 0xffffffffffffffff)

with my ccall-based code I've seen the sigatomic_end stack trace that the OP posted. Originally my code had a rethrow() in the catch block, but on this MWE it seems like that is preventing the crash from happening (I just see the UndefVarError as expected).

JeffBezanson · 2020-01-02T17:07:32Z

In my case the thing I'm @spawning is a ccall and I want to make sure that even if the user interrupts the code we still wait for the ccall to complete before moving on.

Is disable_sigint the right thing here?

ssfrr · 2020-01-02T19:56:35Z

Perhaps, though I wasn't sure whether it was kosher to have a fetch inside the disable_sigint() block, which will cause a context switch (is disable_sigint() task-local?).

JeffBezanson · 2020-01-02T20:21:01Z

It's thread-local currently; it should perhaps be task-local instead. But I think it's generally ok to put disable_sigint around anything; the only ill effect is to make Ctrl-C unresponsive.

robsmith11 · 2020-05-20T15:28:31Z

Idk if it's helpful without a minimal example, but I'm getting a similar crash whenever I kill a parallel simulation that uses tmap from https://github.com/jw3126/ThreadingTools.jl

julia> rp([10910, 8305, 375609], k=10^6, T=1.03*60*60*24)
^C^C^C^C^C^Cfatal: error thrown and no exception handler available.
InterruptException()
sigatomic_end at ./c.jl:425 [inlined]
task_done_hook at ./task.jl:442
_jl_invoke at /buildworker/worker/package_linux64/build/src/gf.c:2218 [inlined]
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2402
jl_apply at /buildworker/worker/package_linux64/build/src/julia.h:1692 [inlined]
jl_finish_task at /buildworker/worker/package_linux64/build/src/task.c:198
start_task at /buildworker/worker/package_linux64/build/src/task.c:717
unknown function (ip: (nil))
[me@dev3 ~]$ JULIA_NUM_THREADS=8 ~/julia/bin/julia --optimize=3 --inline=yes --check-bounds=no --math-mode=fast
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.5.0-DEV.803 (2020-05-03)
 _/ |\__'_|_|_|\__'_|  |  Commit 8ab87d2205 (17 days old master)
|__/                   |

julia> versioninfo()
Julia Version 1.5.0-DEV.803
Commit 8ab87d2205 (2020-05-03 08:07 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
  CPU: Intel(R) Core(TM) i7-7700 CPU @ 3.60GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-9.0.1 (ORCJIT, skylake)
Environment:
  JULIA_NUM_THREADS = 8

arnaudh · 2022-12-19T06:20:35Z

This is also happening to me when calling AWS.jl functions inside asyncmap, using the default HTTP.jl backend.
Let me know if I can help debug / provide a MWE.

^Cfatal: error thrown and no exception handler available.
InterruptException()
jl_mutex_unlock at /cache/build/default-amdci5-6/julialang/julia-release-1-dot-8/src/julia_locks.h:129 [inlined]
ijl_task_get_next at /cache/build/default-amdci5-6/julialang/julia-release-1-dot-8/src/partr.c:569
poptask at ./task.jl:921
wait at ./task.jl:930
task_done_hook at ./task.jl:634
unknown function (ip: 0x7efc43211bd2)
_jl_invoke at /cache/build/default-amdci5-6/julialang/julia-release-1-dot-8/src/gf.c:2365 [inlined]
ijl_apply_generic at /cache/build/default-amdci5-6/julialang/julia-release-1-dot-8/src/gf.c:2547
jl_apply at /cache/build/default-amdci5-6/julialang/julia-release-1-dot-8/src/julia.h:1839 [inlined]
jl_finish_task at /cache/build/default-amdci5-6/julialang/julia-release-1-dot-8/src/task.c:254
start_task at /cache/build/default-amdci5-6/julialang/julia-release-1-dot-8/src/task.c:942

julia> versioninfo()
Julia Version 1.8.3
Commit 0434deb161e (2022-11-14 20:14 UTC)
Platform Info:
  OS: Linux (x86_64-linux-gnu)
  CPU: 8 × Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-13.0.1 (ORCJIT, skylake-avx512)
  Threads: 1 on 8 virtual cores

jibnew · 2023-06-16T11:32:29Z

This happens to me in GAP which I compiled with the option with-julia
When I open GAP:

 ┌───────┐   GAP 4.12.2 of 2022-12-18
 │  GAP  │   https://www.gap-system.org
 └───────┘   Architecture: x86_64-pc-linux-gnu-julia1.8-64-kv8
 Configuration:  gmp 6.2.1, Julia GC, Julia 1.8.5, readline
 Loading the library and packages ...
 Packages:   AClib 1.3.2, Alnuth 3.2.1, AtlasRep 2.1.6, AutPGrp 1.11,
             Browse 1.8.19, CaratInterface 2.3.4, CRISP 1.4.6, Cryst 4.1.25,
             CrystCat 1.1.10, CTblLib 1.3.4, FactInt 1.6.3, FGA 1.4.0,
             Forms 1.2.9, GAPDoc 1.6.6, genss 1.6.8, IO 4.8.0, IRREDSOL 1.4.4,
             LAGUNA 3.9.5, orb 4.9.0, Polenta 1.3.10, Polycyclic 2.16,
             PrimGrp 3.4.3, RadiRoot 2.9, recog 1.4.2, ResClasses 4.7.3,
             SmallGrp 1.5.1, Sophus 1.27, SpinSym 1.5.2, TomLib 1.2.9,
             TransGrp 3.6.3, utils 0.81
 Try '??help' for help. See also '?copyright', '?cite' and '?authors'
gap>

If I now press <Ctrl-C> twice and then say quit;, the result is

fatal: error thrown and no exception handler available.
InterruptException()
jl_gc_state_set at /cache/build/default-amdci4-2/julialang/julia-release-1-dot-8/src/julia_threads.h:340 [inlined]
maybe_collect at /cache/build/default-amdci4-2/julialang/julia-release-1-dot-8/src/julia_threads.h:333 [inlined]
jl_gc_pool_alloc_inner at /cache/build/default-amdci4-2/julialang/julia-release-1-dot-8/src/gc.c:1247 [inlined]
jl_gc_pool_alloc_noinline at /cache/build/default-amdci4-2/julialang/julia-release-1-dot-8/src/gc.c:1306 [inlined]
jl_gc_alloc_ at /cache/build/default-amdci4-2/julialang/julia-release-1-dot-8/src/julia_internal.h:368 [inlined]
jl_gc_alloc at /cache/build/default-amdci4-2/julialang/julia-release-1-dot-8/src/gc.c:3391
NewBag at /home/me/.local/share/gap-4.12.2/src/julia_gc.c:940
NEW_STRING at /home/me/.local/share/gap-4.12.2/src/stringobj.c:420
MakeStringWithLen at /home/me/.local/share/gap-4.12.2/src/stringobj.h:336 [inlined]
MakeString at /home/me/.local/share/gap-4.12.2/src/stringobj.h:343 [inlined]
GAP_rl_func at /home/me/.local/share/gap-4.12.2/src/sysfiles.c:1896
readlineFgets at /home/me/.local/share/gap-4.12.2/src/sysfiles.c:2109 [inlined]
syFgets at /home/me/.local/share/gap-4.12.2/src/sysfiles.c:2195
GetLine2 at /home/me/.local/share/gap-4.12.2/src/io.c:1035 [inlined]
GetLine at /home/me/.local/share/gap-4.12.2/src/io.c:1090
GetNextChar at /home/me/.local/share/gap-4.12.2/src/io.c:152
NextSymbol at /home/me/.local/share/gap-4.12.2/src/scanner.c:890
Match at /home/me/.local/share/gap-4.12.2/src/scanner.c:219
Match_ at /home/me/.local/share/gap-4.12.2/src/read.c:196 [inlined]
ReadEvalCommand at /home/me/.local/share/gap-4.12.2/src/read.c:2532
FuncSHELL at /home/me/.local/share/gap-4.12.2/src/gap.c:283
CALL_6ARGS at /home/me/.local/share/gap-4.12.2/src/calls.h:336 [inlined]
EvalOrExecCall at /home/me/.local/share/gap-4.12.2/src/funcs.c:162 [inlined]
ExecProccall6args at /home/me/.local/share/gap-4.12.2/src/funcs.c:248
EXEC_STAT at /home/me/.local/share/gap-4.12.2/src/stats.c:55 [inlined]
ExecSeqStatHelper at /home/me/.local/share/gap-4.12.2/src/stats.c:147 [inlined]
ExecSeqStat3 at /home/me/.local/share/gap-4.12.2/src/stats.c:170
EXEC_STAT at /home/me/.local/share/gap-4.12.2/src/stats.c:55 [inlined]
EXEC_CURR_FUNC at /home/me/.local/share/gap-4.12.2/src/stats.c:61
DoExecFunc at /home/me/.local/share/gap-4.12.2/src/funcs.c:468 [inlined]
DoExecFunc0args at /home/me/.local/share/gap-4.12.2/src/funcs.c:483
CALL_0ARGS at /home/me/.local/share/gap-4.12.2/src/calls.h:306 [inlined]
IntrFuncCallEnd at /home/me/.local/share/gap-4.12.2/src/intrprtr.c:396
ReadCallVarAss at /home/me/.local/share/gap-4.12.2/src/read.c:896
TryReadStatement at /home/me/.local/share/gap-4.12.2/src/read.c:2422
ReadStats at /home/me/.local/share/gap-4.12.2/src/read.c:2451
ReadIf at /home/me/.local/share/gap-4.12.2/src/read.c:2052
TryReadStatement at /home/me/.local/share/gap-4.12.2/src/read.c:2426
ReadEvalCommand at /home/me/.local/share/gap-4.12.2/src/read.c:2594
READ_GAP_ROOT at /home/me/.local/share/gap-4.12.2/src/streams.c:409
InitializeGap at /home/me/.local/share/gap-4.12.2/src/gap.c:1526
realmain at /home/me/.local/share/gap-4.12.2/src/gap.c:397
unknown function (ip: 0x7f07d0023a8f)
__libc_start_main at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
_start at /home/me/.local/share/gap-4.12.2/gap (unknown line)

chriselrod mentioned this issue Mar 30, 2020

QuickSort hangs tkf/ThreadsX.jl#104

Open

dfdx mentioned this issue May 19, 2021

Fix delivery & error callbacks and polling dfdx/RDKafka.jl#13

Merged

xgdgsc mentioned this issue Nov 12, 2022

handle signals gracefully? JuliaLang/Downloads.jl#192

Open

brenhinkeller added error handling Handling of exceptions by Julia or the user multithreading Base.Threads and related functionality labels Nov 20, 2022

vtjnash linked a pull request Feb 1, 2024 that will close this issue

signal handling: User-defined interrupt handlers #49541

Open

7 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fatal: error thrown and no exception handler available. #34184

fatal: error thrown and no exception handler available. #34184

ghost commented Dec 23, 2019 •

edited by ghost

Loading

JeffBezanson commented Dec 23, 2019

ghost commented Dec 23, 2019 •

edited by ghost

Loading

ssfrr commented Jan 2, 2020 •

edited

Loading

JeffBezanson commented Jan 2, 2020

ssfrr commented Jan 2, 2020

JeffBezanson commented Jan 2, 2020

robsmith11 commented May 20, 2020

arnaudh commented Dec 19, 2022

jibnew commented Jun 16, 2023

fatal: error thrown and no exception handler available. #34184

fatal: error thrown and no exception handler available. #34184

Comments

ghost commented Dec 23, 2019 • edited by ghost Loading

JeffBezanson commented Dec 23, 2019

ghost commented Dec 23, 2019 • edited by ghost Loading

ssfrr commented Jan 2, 2020 • edited Loading

JeffBezanson commented Jan 2, 2020

ssfrr commented Jan 2, 2020

JeffBezanson commented Jan 2, 2020

robsmith11 commented May 20, 2020

arnaudh commented Dec 19, 2022

jibnew commented Jun 16, 2023

ghost commented Dec 23, 2019 •

edited by ghost

Loading

ghost commented Dec 23, 2019 •

edited by ghost

Loading

ssfrr commented Jan 2, 2020 •

edited

Loading