segfault when trying to use wrds python module

[tbeason]

Affects: PythonCall

Describe the bug

Segfault when trying to use python module wrds via PythonCall.jl on Julia 1.11 RC3. The lines below were hand-typed. It crashed as soon as I hit the period after wrds.

Installed wrds.py (from pip) and pyarrow, both with CondaPkg.jl. PythonCall.jl and CondaPkg.jl are up to date. This is on Windows 11.

julia> using PythonCall

julia> wrds = pyimport("wrds")
Python: <module 'wrds' from 'C:\\Users\\beasont\\Documents\\GitHub\\RealizedVarianceCode\\code\\spy\\.CondaPkg\\env\\Lib\\site-packages\\wrds\\__init__.py'>

julia> db = wrds.
Please submit a bug report with steps to reproduce this fault, and any error messages that follow (in their entirety). Thanks.
Exception: EXCEPTION_ACCESS_VIOLATION at 0x7ffa693b6add -- PyType_Lookup at C:\Users\beasont\Documents\GitHub\RealizedVarianceCode\code\spy\.CondaPkg\env\python312.dll (unknown line)
in expression starting at none:0
PyType_Lookup at C:\Users\beasont\Documents\GitHub\RealizedVarianceCode\code\spy\.CondaPkg\env\python312.dll (unknown line)
PyObject_Dir at C:\Users\beasont\Documents\GitHub\RealizedVarianceCode\code\spy\.CondaPkg\env\python312.dll (unknown line)
PyObject_Dir at C:\Users\beasont\.julia\packages\PythonCall\Nr75f\src\C\pointers.jl:303 [inlined]
macro expansion at C:\Users\beasont\.julia\packages\PythonCall\Nr75f\src\Core\Py.jl:132 [inlined]
pydir at C:\Users\beasont\.julia\packages\PythonCall\Nr75f\src\Core\builtins.jl:216
propertynames at C:\Users\beasont\.julia\packages\PythonCall\Nr75f\src\Core\Py.jl:293
unknown function (ip: 000002be992a7282)
complete_symbol at C:\workdir\usr\share\julia\stdlib\v1.11\REPL\src\REPLCompletions.jl:195
#complete_identifiers!#55 at C:\workdir\usr\share\julia\stdlib\v1.11\REPL\src\REPLCompletions.jl:1167
complete_identifiers! at C:\workdir\usr\share\julia\stdlib\v1.11\REPL\src\REPLCompletions.jl:1069 [inlined]
completions at C:\workdir\usr\share\julia\stdlib\v1.11\REPL\src\REPLCompletions.jl:1420
#complete_line#85 at C:\workdir\usr\share\julia\stdlib\v1.11\REPL\src\REPL.jl:622
complete_line at C:\workdir\usr\share\julia\stdlib\v1.11\REPL\src\REPL.jl:619
jfptr_complete_line_12455 at C:\Users\beasont\.julia\juliaup\julia-1.11.0-rc3+0.x64.w64.mingw32\share\julia\compiled\v1.11\REPL\u0gqU_e6ieL.dll (unknown line)
check_for_hint at C:\workdir\usr\share\julia\stdlib\v1.11\REPL\src\LineEdit.jl:385
#143 at C:\workdir\usr\share\julia\stdlib\v1.11\REPL\src\LineEdit.jl:2521
jl_apply at C:/workdir/src\julia.h:2157 [inlined]
jl_f__call_latest at C:/workdir/src\builtins.c:875
#invokelatest#2 at .\essentials.jl:1054 [inlined]
invokelatest at .\essentials.jl:1051 [inlined]
#30 at C:\workdir\usr\share\julia\stdlib\v1.11\REPL\src\LineEdit.jl:1705
jfptr_YY.30_9971 at C:\Users\beasont\.julia\juliaup\julia-1.11.0-rc3+0.x64.w64.mingw32\share\julia\compiled\v1.11\REPL\u0gqU_e6ieL.dll (unknown line)
macro expansion at C:\workdir\usr\share\julia\stdlib\v1.11\REPL\src\LineEdit.jl:2855 [inlined]
macro expansion at .\lock.jl:273 [inlined]
#282 at C:\workdir\usr\share\julia\stdlib\v1.11\REPL\src\LineEdit.jl:2845
jfptr_YY.282_10545 at C:\Users\beasont\.julia\juliaup\julia-1.11.0-rc3+0.x64.w64.mingw32\share\julia\compiled\v1.11\REPL\u0gqU_e6ieL.dll (unknown line)
jl_apply at C:/workdir/src\julia.h:2157 [inlined]
start_task at C:/workdir/src\task.c:1202
Allocations: 3844822 (Pool: 3844523; Big: 299); GC: 6

Please submit a bug report with steps to reproduce this fault, and any error messages that follow (in their entirety). Thanks.
Exception: EXCEPTION_ACCESS_VIOLATION at 0x7ffa694d2164 -- Py_FinalizeEx at C:\Users\beasont\Documents\GitHub\RealizedVarianceCode\code\spy\.CondaPkg\env\python312.dll (unknown line)
in expression starting at none:0
Py_FinalizeEx at C:\Users\beasont\Documents\GitHub\RealizedVarianceCode\code\spy\.CondaPkg\env\python312.dll (unknown line)
Py_FinalizeEx at C:\Users\beasont\.julia\packages\PythonCall\Nr75f\src\C\pointers.jl:303 [inlined]
#37 at C:\Users\beasont\.julia\packages\PythonCall\Nr75f\src\C\context.jl:194
unknown function (ip: 000002be992a7824)
_atexit at .\initdefs.jl:456
jfptr__atexit_69699.1 at C:\Users\beasont\.julia\juliaup\julia-1.11.0-rc3+0.x64.w64.mingw32\lib\julia\sys.dll (unknown line)
jl_apply at C:/workdir/src\julia.h:2157 [inlined]
ijl_atexit_hook at C:/workdir/src\init.c:271
ijl_exit at C:/workdir/src\init.c:207
jl_exception_handler at C:/workdir/src\signals-win.c:337 [inlined]
jl_exception_handler at C:/workdir/src\signals-win.c:229
__julia_personality at C:/workdir/src\win32_ucontext.c:28
_chkstk at C:\WINDOWS\SYSTEM32\ntdll.dll (unknown line)
RtlFindCharInUnicodeString at C:\WINDOWS\SYSTEM32\ntdll.dll (unknown line)
KiUserExceptionDispatcher at C:\WINDOWS\SYSTEM32\ntdll.dll (unknown line)
PyType_Lookup at C:\Users\beasont\Documents\GitHub\RealizedVarianceCode\code\spy\.CondaPkg\env\python312.dll (unknown line)
PyObject_Dir at C:\Users\beasont\Documents\GitHub\RealizedVarianceCode\code\spy\.CondaPkg\env\python312.dll (unknown line)
PyObject_Dir at C:\Users\beasont\.julia\packages\PythonCall\Nr75f\src\C\pointers.jl:303 [inlined]
macro expansion at C:\Users\beasont\.julia\packages\PythonCall\Nr75f\src\Core\Py.jl:132 [inlined]
pydir at C:\Users\beasont\.julia\packages\PythonCall\Nr75f\src\Core\builtins.jl:216
propertynames at C:\Users\beasont\.julia\packages\PythonCall\Nr75f\src\Core\Py.jl:293
unknown function (ip: 000002be992a7282)
complete_symbol at C:\workdir\usr\share\julia\stdlib\v1.11\REPL\src\REPLCompletions.jl:195
#complete_identifiers!#55 at C:\workdir\usr\share\julia\stdlib\v1.11\REPL\src\REPLCompletions.jl:1167
complete_identifiers! at C:\workdir\usr\share\julia\stdlib\v1.11\REPL\src\REPLCompletions.jl:1069 [inlined]
completions at C:\workdir\usr\share\julia\stdlib\v1.11\REPL\src\REPLCompletions.jl:1420
#complete_line#85 at C:\workdir\usr\share\julia\stdlib\v1.11\REPL\src\REPL.jl:622
complete_line at C:\workdir\usr\share\julia\stdlib\v1.11\REPL\src\REPL.jl:619
jfptr_complete_line_12455 at C:\Users\beasont\.julia\juliaup\julia-1.11.0-rc3+0.x64.w64.mingw32\share\julia\compiled\v1.11\REPL\u0gqU_e6ieL.dll (unknown line)
check_for_hint at C:\workdir\usr\share\julia\stdlib\v1.11\REPL\src\LineEdit.jl:385
#143 at C:\workdir\usr\share\julia\stdlib\v1.11\REPL\src\LineEdit.jl:2521
jl_apply at C:/workdir/src\julia.h:2157 [inlined]
jl_f__call_latest at C:/workdir/src\builtins.c:875
#invokelatest#2 at .\essentials.jl:1054 [inlined]
invokelatest at .\essentials.jl:1051 [inlined]
#30 at C:\workdir\usr\share\julia\stdlib\v1.11\REPL\src\LineEdit.jl:1705
jfptr_YY.30_9971 at C:\Users\beasont\.julia\juliaup\julia-1.11.0-rc3+0.x64.w64.mingw32\share\julia\compiled\v1.11\REPL\u0gqU_e6ieL.dll (unknown line)
macro expansion at C:\workdir\usr\share\julia\stdlib\v1.11\REPL\src\LineEdit.jl:2855 [inlined]
macro expansion at .\lock.jl:273 [inlined]
#282 at C:\workdir\usr\share\julia\stdlib\v1.11\REPL\src\LineEdit.jl:2845
jfptr_YY.282_10545 at C:\Users\beasont\.julia\juliaup\julia-1.11.0-rc3+0.x64.w64.mingw32\share\julia\compiled\v1.11\REPL\u0gqU_e6ieL.dll (unknown line)
jl_apply at C:/workdir/src\julia.h:2157 [inlined]
start_task at C:/workdir/src\task.c:1202
Allocations: 3844822 (Pool: 3844523; Big: 299); GC: 6

Your system
Please provide detailed information about your system:

• The operating system
• The version of Julia, Python, PythonCall, JuliaCall and any other affected packages
• If an issue with PythonCall, the output of Base.versioninfo(), Pkg.status() and CondaPkg.status().
• If an issue with JuliaCall, the output of pip list or conda list and juliapkg.status().

Additional context
Add any other context about the problem here.

[tbeason]

FYI I was starting julia with julia -t auto and getting this error. I tried without spinning up a thread pool and it seems to be working now. So this is probably related to multithreaded execution of something.

[dzonimn]

I can confirm that this still exists on the final Julia 1.11 release version as well. However, I did downgrade to Python versions 3.9 and 3.11 and it works fine for those, so it could be a Python 3.12 issue (in combination with Julia 1.10 to 1.11). The issue also exists for 3.13.

Edit: this also still exists on Julia 1.11.1