Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Kernel does not start with Julia 1.9 #1062

Closed
zickgraf opened this issue Dec 16, 2022 · 14 comments
Closed

Kernel does not start with Julia 1.9 #1062

zickgraf opened this issue Dec 16, 2022 · 14 comments

Comments

@zickgraf
Copy link

From JuliaLang/julia#47196:

Summary: JuliaLang/julia@b7201d6 introduces a regression in IJulia. When trying to start a Julia kernel in a Jupyter notebook, Julia segfaults.

  1. The output of versioninfo()
Julia Version 1.9.0-DEV.1602
Commit 69f8a7b6481 (2022-10-17 08:33 UTC) (note: bisected to the commit mentioned above)
Platform Info:
  OS: Linux (x86_64-linux-gnu)
  CPU: 4 × Intel(R) Core(TM) i5-7500T CPU @ 2.70GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-14.0.6 (ORCJIT, skylake)
  Threads: 1 on 4 virtual cores
  1. How you installed Julia
    Both via compiling from source and via downloading a nightly build.
  2. A minimal working example (MWE), also known as a minimum reproducible example
    Install IJulia, run notebook() and create a new notebook with kernel "Julia 1.9.0-DEV". After a timeout of about 1 minute, an error message "Connection failed" appears. Running everything with strace shows that the child Julia process launched by IJulia has segfaulted.

According to JuliaLang/julia#47196 (comment) and JuliaLang/julia#47196 (comment) this is due to a bug in IJulia.

@fortunewalla
Copy link

I too have been facing the same problem with both 1.10 & 1.9. At first I thought it was my setup but after looking up online & trying different combinations of solutions, I felt IJulia might not be set to work beyond the 1.8.x versions.

1.8.3 works and has a stable connection

WIn 10
python 3.9

tried through Julia built-in conda/jupyter
tried through external conda/jupyter
tried through external mamba/jupyter

1.9 Jupyter messages and the nudge loop keeps repeating

    To access the server, open this file in a browser:
        file:///C:/Users/julius/AppData/Roaming/jupyter/runtime/jpserver-15220-open.html
    Or copy and paste one of these URLs:
        http://localhost:8888/?token=58fe86c9c9a3dbbbef9bc2bb13c6072a9093cdfc403839d9
     or http://127.0.0.1:8888/?token=58fe86c9c9a3dbbbef9bc2bb13c6072a9093cdfc403839d9
[I 2022-12-20 11:01:55.684 ServerApp] Creating new notebook in
[W 2022-12-20 11:01:57.418 ServerApp] 404 GET /nbextensions/widgets/notebook/js/extension.js?v=20221220110129 ([email protected]) 0.00ms referer=http://127.0.0.1:8888/notebooks/Untitled6.ipynb?kernel_name=julia-1.9
[I 2022-12-20 11:01:57.418 ServerApp] Kernel started: ecb7abb3-a7f7-42b7-a022-0989323afc82
Starting kernel event loops.
[W 2022-12-20 11:02:57.450 ServerApp] Timeout waiting for kernel_info reply from ecb7abb3-a7f7-42b7-a022-0989323afc82
[I 2022-12-20 11:02:57.452 ServerApp] Connecting to kernel ecb7abb3-a7f7-42b7-a022-0989323afc82.
[W 2022-12-20 11:03:02.014 ServerApp] Nudge: attempt 10 on kernel ecb7abb3-a7f7-42b7-a022-0989323afc82
[W 2022-12-20 11:03:07.057 ServerApp] Nudge: attempt 20 on kernel ecb7abb3-a7f7-42b7-a022-0989323afc82
[W 2022-12-20 11:03:12.119 ServerApp] Nudge: attempt 30 on kernel ecb7abb3-a7f7-42b7-a022-0989323afc82
[W 2022-12-20 11:03:17.180 ServerApp] Nudge: attempt 40 on kernel ecb7abb3-a7f7-42b7-a022-0989323afc82
[W 2022-12-20 11:03:22.261 ServerApp] Nudge: attempt 50 on kernel ecb7abb3-a7f7-42b7-a022-0989323afc82
[W 2022-12-20 11:03:27.323 ServerApp] Nudge: attempt 60 on kernel ecb7abb3-a7f7-42b7-a022-0989323afc82
[W 2022-12-20 11:03:32.384 ServerApp] Nudge: attempt 70 on kernel ecb7abb3-a7f7-42b7-a022-0989323afc82
[W 2022-12-20 11:03:37.466 ServerApp] Nudge: attempt 80 on kernel ecb7abb3-a7f7-42b7-a022-0989323afc82
[W 2022-12-20 11:03:42.527 ServerApp] Nudge: attempt 90 on kernel ecb7abb3-a7f7-42b7-a022-0989323afc82
[W 2022-12-20 11:03:47.570 ServerApp] Nudge: attempt 100 on kernel ecb7abb3-a7f7-42b7-a022-0989323afc82
[W 2022-12-20 11:03:52.630 ServerApp] Nudge: attempt 110 on kernel ecb7abb3-a7f7-42b7-a022-0989323afc82
[E 2022-12-20 11:03:57.469 ServerApp] Uncaught exception GET /api/kernels/ecb7abb3-a7f7-42b7-a022-0989323afc82/channels?session_id=2cd542471d9c439aa70009a5e7d6ada4 (127.0.0.1)
    HTTPServerRequest(protocol='http', host='127.0.0.1:8888', method='GET', uri='/api/kernels/ecb7abb3-a7f7-42b7-a022-0989323afc82/channels?session_id=2cd542471d9c439aa70009a5e7d6ada4', version='HTTP/1.1', remote_ip='127.0.0.1')
    Traceback (most recent call last):
      File "C:\mamba\envs\j\lib\site-packages\tornado\websocket.py", line 944, in _accept_connection
        await open_result
      File "C:\mamba\envs\j\lib\site-packages\jupyter_server\services\kernels\websocket.py", line 68, in open
        await self.connection.connect()
    asyncio.exceptions.TimeoutError: Timeout
[I 2022-12-20 11:03:57.613 ServerApp] Saving file at /Untitled6.ipynb
C:\mamba\envs\j\lib\site-packages\nbformat\__init__.py:128: MissingIDFieldWarning: Code cell is missing an id field, this will become a hard error in future nbformat versions. You may want to use `normalize()` on your notebooks before validations (available since nbformat 5.1.4). Previous versions of nbformat are fixing this issue transparently, and will stop doing so in the future.
  validate(nb)
[W 2022-12-20 11:03:58.612 ServerApp] Replacing stale connection: ecb7abb3-a7f7-42b7-a022-0989323afc82:2cd542471d9c439aa70009a5e7d6ada4
[W 2022-12-20 11:04:58.626 ServerApp] Timeout waiting for kernel_info reply from ecb7abb3-a7f7-42b7-a022-0989323afc82
[I 2022-12-20 11:04:58.626 ServerApp] Connecting to kernel ecb7abb3-a7f7-42b7-a022-0989323afc82.
[W 2022-12-20 11:05:03.185 ServerApp] Nudge: attempt 10 on kernel ecb7abb3-a7f7-42b7-a022-0989323afc82
[W 2022-12-20 11:05:08.244 ServerApp] Nudge: attempt 20 on kernel ecb7abb3-a7f7-42b7-a022-0989323afc82
[W 2022-12-20 11:05:13.325 ServerApp] Nudge: attempt 30 on kernel ecb7abb3-a7f7-42b7-a022-0989323afc82
[W 2022-12-20 11:05:18.391 ServerApp] Nudge: attempt 40 on kernel ecb7abb3-a7f7-42b7-a022-0989323afc82
[W 2022-12-20 11:05:23.476 ServerApp] Nudge: attempt 50 on kernel ecb7abb3-a7f7-42b7-a022-0989323afc82
[W 2022-12-20 11:05:28.518 ServerApp] Nudge: attempt 60 on kernel ecb7abb3-a7f7-42b7-a022-0989323afc82
[W 2022-12-20 11:05:33.613 ServerApp] Nudge: attempt 70 on kernel ecb7abb3-a7f7-42b7-a022-0989323afc82
[W 2022-12-20 11:05:38.711 ServerApp] Nudge: attempt 80 on kernel ecb7abb3-a7f7-42b7-a022-0989323afc82
[W 2022-12-20 11:05:43.775 ServerApp] Nudge: attempt 90 on kernel ecb7abb3-a7f7-42b7-a022-0989323afc82
[W 2022-12-20 11:05:48.835 ServerApp] Nudge: attempt 100 on kernel ecb7abb3-a7f7-42b7-a022-0989323afc82
[W 2022-12-20 11:05:53.896 ServerApp] Nudge: attempt 110 on kernel ecb7abb3-a7f7-42b7-a022-0989323afc82
[I 2022-12-20 11:05:57.629 ServerApp] Saving file at /Untitled6.ipynb
[E 2022-12-20 11:05:58.666 ServerApp] Uncaught exception GET /api/kernels/ecb7abb3-a7f7-42b7-a022-0989323afc82/channels?session_id=2cd542471d9c439aa70009a5e7d6ada4 (127.0.0.1)
    HTTPServerRequest(protocol='http', host='127.0.0.1:8888', method='GET', uri='/api/kernels/ecb7abb3-a7f7-42b7-a022-0989323afc82/channels?session_id=2cd542471d9c439aa70009a5e7d6ada4', version='HTTP/1.1', remote_ip='127.0.0.1')
    Traceback (most recent call last):
      File "C:\mamba\envs\j\lib\site-packages\tornado\websocket.py", line 944, in _accept_connection
        await open_result
      File "C:\mamba\envs\j\lib\site-packages\jupyter_server\services\kernels\websocket.py", line 68, in open
        await self.connection.connect()
    asyncio.exceptions.TimeoutError: Timeout
[W 2022-12-20 11:06:00.623 ServerApp] Replacing stale connection: ecb7abb3-a7f7-42b7-a022-0989323afc82:2cd542471d9c439aa70009a5e7d6ada4
[W 2022-12-20 11:07:00.640 ServerApp] Timeout waiting for kernel_info reply from ecb7abb3-a7f7-42b7-a022-0989323afc82
[I 2022-12-20 11:07:00.640 ServerApp] Connecting to kernel ecb7abb3-a7f7-42b7-a022-0989323afc82.
[W 2022-12-20 11:07:05.228 ServerApp] Nudge: attempt 10 on kernel ecb7abb3-a7f7-42b7-a022-0989323afc82

1.10 Jupyter messages and the nudge loop keeps repeating

    To access the server, open this file in a browser:
        file:///C:/Users/julius/AppData/Roaming/jupyter/runtime/jpserver-17644-open.html
    Or copy and paste one of these URLs:
        http://localhost:8888/?token=67a8bd13457993c7d131c49d022f847e7bed8c55ca4f4be0
     or http://127.0.0.1:8888/?token=67a8bd13457993c7d131c49d022f847e7bed8c55ca4f4be0
[I 2022-12-20 10:47:34.930 ServerApp] Generating new user for token-authenticated request: a035a9161c8d4494b52156abe5464c9d
[I 2022-12-20 10:47:44.076 ServerApp] Creating new notebook in
[W 2022-12-20 10:47:45.780 ServerApp] 404 GET /nbextensions/widgets/notebook/js/extension.js?v=20221220104659 ([email protected]) 15.65ms referer=http://127.0.0.1:8888/notebooks/Untitled5.ipynb?kernel_name=julia-1.10
[I 2022-12-20 10:47:45.780 ServerApp] Kernel started: 35bdc00d-90dc-4802-b110-d7cf8a700f83
[W 2022-12-20 10:48:45.810 ServerApp] Timeout waiting for kernel_info reply from 35bdc00d-90dc-4802-b110-d7cf8a700f83
[I 2022-12-20 10:48:45.812 ServerApp] Connecting to kernel 35bdc00d-90dc-4802-b110-d7cf8a700f83.
[W 2022-12-20 10:48:50.379 ServerApp] Nudge: attempt 10 on kernel 35bdc00d-90dc-4802-b110-d7cf8a700f83
[W 2022-12-20 10:48:55.432 ServerApp] Nudge: attempt 20 on kernel 35bdc00d-90dc-4802-b110-d7cf8a700f83
[W 2022-12-20 10:49:00.506 ServerApp] Nudge: attempt 30 on kernel 35bdc00d-90dc-4802-b110-d7cf8a700f83
[W 2022-12-20 10:49:05.552 ServerApp] Nudge: attempt 40 on kernel 35bdc00d-90dc-4802-b110-d7cf8a700f83
[W 2022-12-20 10:49:10.609 ServerApp] Nudge: attempt 50 on kernel 35bdc00d-90dc-4802-b110-d7cf8a700f83
[W 2022-12-20 10:49:15.663 ServerApp] Nudge: attempt 60 on kernel 35bdc00d-90dc-4802-b110-d7cf8a700f83
[W 2022-12-20 10:49:20.700 ServerApp] Nudge: attempt 70 on kernel 35bdc00d-90dc-4802-b110-d7cf8a700f83
[W 2022-12-20 10:49:25.756 ServerApp] Nudge: attempt 80 on kernel 35bdc00d-90dc-4802-b110-d7cf8a700f83
[W 2022-12-20 10:49:30.834 ServerApp] Nudge: attempt 90 on kernel 35bdc00d-90dc-4802-b110-d7cf8a700f83
[W 2022-12-20 10:49:35.867 ServerApp] Nudge: attempt 100 on kernel 35bdc00d-90dc-4802-b110-d7cf8a700f83
[W 2022-12-20 10:49:40.936 ServerApp] Nudge: attempt 110 on kernel 35bdc00d-90dc-4802-b110-d7cf8a700f83
[E 2022-12-20 10:49:45.831 ServerApp] Uncaught exception GET /api/kernels/35bdc00d-90dc-4802-b110-d7cf8a700f83/channels?session_id=6c7ed51c5c3341978caf1e5965519c47 (127.0.0.1)
    HTTPServerRequest(protocol='http', host='127.0.0.1:8888', method='GET', uri='/api/kernels/35bdc00d-90dc-4802-b110-d7cf8a700f83/channels?session_id=6c7ed51c5c3341978caf1e5965519c47', version='HTTP/1.1', remote_ip='127.0.0.1')
    Traceback (most recent call last):
      File "C:\mamba\envs\j\lib\site-packages\tornado\websocket.py", line 944, in _accept_connection
        await open_result
      File "C:\mamba\envs\j\lib\site-packages\jupyter_server\services\kernels\websocket.py", line 68, in open
        await self.connection.connect()
    asyncio.exceptions.TimeoutError: Timeout
[I 2022-12-20 10:49:46.633 ServerApp] Saving file at /Untitled5.ipynb
C:\mamba\envs\j\lib\site-packages\nbformat\__init__.py:128: MissingIDFieldWarning: Code cell is missing an id field, this will become a hard error in future nbformat versions. You may want to use `normalize()` on your notebooks before validations (available since nbformat 5.1.4). Previous versions of nbformat are fixing this issue transparently, and will stop doing so in the future.
  validate(nb)
[W 2022-12-20 10:49:47.635 ServerApp] Replacing stale connection: 35bdc00d-90dc-4802-b110-d7cf8a700f83:6c7ed51c5c3341978caf1e5965519c47
[W 2022-12-20 10:50:47.668 ServerApp] Timeout waiting for kernel_info reply from 35bdc00d-90dc-4802-b110-d7cf8a700f83
[I 2022-12-20 10:50:47.668 ServerApp] Connecting to kernel 35bdc00d-90dc-4802-b110-d7cf8a700f83.
[W 2022-12-20 10:50:52.240 ServerApp] Nudge: attempt 10 on kernel 35bdc00d-90dc-4802-b110-d7cf8a700f83
[W 2022-12-20 10:50:57.296 ServerApp] Nudge: attempt 20 on kernel 35bdc00d-90dc-4802-b110-d7cf8a700f83
[W 2022-12-20 10:51:02.364 ServerApp] Nudge: attempt 30 on kernel 35bdc00d-90dc-4802-b110-d7cf8a700f83
[W 2022-12-20 10:51:07.418 ServerApp] Nudge: attempt 40 on kernel 35bdc00d-90dc-4802-b110-d7cf8a700f83
[W 2022-12-20 10:51:12.478 ServerApp] Nudge: attempt 50 on kernel 35bdc00d-90dc-4802-b110-d7cf8a700f83
[W 2022-12-20 10:51:17.577 ServerApp] Nudge: attempt 60 on kernel 35bdc00d-90dc-4802-b110-d7cf8a700f83
[W 2022-12-20 10:51:22.638 ServerApp] Nudge: attempt 70 on kernel 35bdc00d-90dc-4802-b110-d7cf8a700f83
[W 2022-12-20 10:51:27.700 ServerApp] Nudge: attempt 80 on kernel 35bdc00d-90dc-4802-b110-d7cf8a700f83
[W 2022-12-20 10:51:32.782 ServerApp] Nudge: attempt 90 on kernel 35bdc00d-90dc-4802-b110-d7cf8a700f83
[W 2022-12-20 10:51:37.844 ServerApp] Nudge: attempt 100 on kernel 35bdc00d-90dc-4802-b110-d7cf8a700f83
[W 2022-12-20 10:51:42.908 ServerApp] Nudge: attempt 110 on kernel 35bdc00d-90dc-4802-b110-d7cf8a700f83
[E 2022-12-20 10:51:47.694 ServerApp] Uncaught exception GET /api/kernels/35bdc00d-90dc-4802-b110-d7cf8a700f83/channels?session_id=6c7ed51c5c3341978caf1e5965519c47 (127.0.0.1)
    HTTPServerRequest(protocol='http', host='127.0.0.1:8888', method='GET', uri='/api/kernels/35bdc00d-90dc-4802-b110-d7cf8a700f83/channels?session_id=6c7ed51c5c3341978caf1e5965519c47', version='HTTP/1.1', remote_ip='127.0.0.1')
    Traceback (most recent call last):
      File "C:\mamba\envs\j\lib\site-packages\tornado\websocket.py", line 944, in _accept_connection
        await open_result
      File "C:\mamba\envs\j\lib\site-packages\jupyter_server\services\kernels\websocket.py", line 68, in open
        await self.connection.connect()
    asyncio.exceptions.TimeoutError: Timeout
[W 2022-12-20 10:51:49.611 ServerApp] Replacing stale connection: 35bdc00d-90dc-4802-b110-d7cf8a700f83:6c7ed51c5c3341978caf1e5965519c47
[W 2022-12-20 10:52:49.622 ServerApp] Timeout waiting for kernel_info reply from 35bdc00d-90dc-4802-b110-d7cf8a700f83
[I 2022-12-20 10:52:49.624 ServerApp] Connecting to kernel 35bdc00d-90dc-4802-b110-d7cf8a700f83.
[W 2022-12-20 10:52:54.200 ServerApp] Nudge: attempt 10 on kernel 35bdc00d-90dc-4802-b110-d7cf8a700f83
[W 2022-12-20 10:52:59.290 ServerApp] Nudge: attempt 20 on kernel 35bdc00d-90dc-4802-b110-d7cf8a700f83

@stevengj
Copy link
Member

Fixed by 30ff84b

@stevengj
Copy link
Member

New version registered: JuliaRegistries/General#74432

@ctkelley
Copy link

I am having the same problem with 1.9-rc1. Is the new version what pkg gives me when I do

pkg> update

@ctkelley
Copy link

Spoke too soon. I can't get a kernel on 1.8.5 either. I'm on an Mac mini with an M2 Pro running the latest OS.

Starting over with an empty .julia directory did not help.

@fortunewalla
Copy link

Unless it is a Mac specific issue, in my Windows 10, both 1.8.5 & 1.10-829 nightly work.

The thing I found out is that every Julia update somehow resets the project path inside the jupyter kernel.json file. So I had to manually alter the kernel.json file.

[7073ff75] IJulia v1.24.0

1.8.5 works

image

1.10 works

image

@ctkelley
Copy link

ctkelley commented Mar 24, 2023

Thanks.

Where is that .json file? I assume it is somewhere in .julia/packages/IJulia, but can't find it.

Have you been able to open notebooks made with older versions of Julia, like 1.6 or 1.7?

@fortunewalla
Copy link

Have you been able to open notebooks made with older versions of Julia, like 1.6 or 1.7?

Haven't tried it. 😇

Where is that .json file? I assume it is somewhere in .julia/packages/IJulia, but can't find it.

On windows it is the standard location,

(jupenv) C:\python3>jupyter kernelspec list
Available kernels:
  python3                C:\python3\envs\jupenv\share\jupyter\kernels\python3
  ir                     C:\Users\julius\AppData\Roaming\jupyter\kernels\ir
  julia-1.10             C:\Users\julius\AppData\Roaming\jupyter\kernels\julia-1.10
  julia-1.8              C:\Users\julius\AppData\Roaming\jupyter\kernels\julia-1.8

But it depends on whether you installed Conda or Juptyer through Julia or outside Julia.

If you installed it through Julia, then the kernel.json file will be somewhere inside the Julia folder or in the Julia project path.

What happened with me inside the kernel.json is described here: https://discourse.julialang.org/t/quarto-julia-cli-kernel-died-before-replying-to-kernel-info/91416/4

@ctkelley
Copy link

ctkelley commented Mar 24, 2023

I am at a loss. Nothing works consistently and I have no idea how to proceed. I cannot find the .json file. I have tried taking out

ENV["PYTHON"]=""
ENV["CONDA_JL_USE_MINIFORGE"] = "1"

out of my startup.jl, but that did not seem to change anything. I can only hope that this gets fixed eventually. As it stands now, I cannot use IJulia. Even if I create a new notebook, I can't reopen it later.

I tried using the Intel builds with Rosetta, and got similar results.

@fortunewalla
Copy link

I am at a loss. Nothing works consistently and I have no idea how to proceed. I cannot find the .json file. I have tried taking out

ENV["PYTHON"]=""
ENV["CONDA_JL_USE_MINIFORGE"] = "1"

out of my startup.jl, but that did not seem to change anything. I can only hope that this gets fixed eventually. As it stands now, I cannot use IJulia. Even if I create a new notebook, I can't reopen it later.

I tried using the Intel builds with Rosetta, and got similar results.

what is the path of your python?

also what does the command jupyter kernelspec list print out?

@ctkelley
Copy link

I am not sure where python is.

.julia/conda/deps.jl is

const ROOTENV = "/Users/ctk/.julia/conda/3/aarch64"
const MINICONDA_VERSION = "3"
const USE_MINIFORGE = true
const CONDA_EXE = "/Users/ctk/.julia/conda/3/aarch64/bin/conda"

and
jupyter kernelspec list
returns

jupyter: Command not found.

I get a kernel from new notebooks in my home directory. In subdirectories and/or with old notebooks I get the problem. I am working on an MWE.

@fortunewalla
Copy link

fortunewalla commented Mar 26, 2023

I am not sure where python is.

So you installed Conda from Julia right?

const ROOTENV = "/Users/ctk/.julia/conda/3/aarch64"
const MINICONDA_VERSION = "3"
const USE_MINIFORGE = true
const CONDA_EXE = "/Users/ctk/.julia/conda/3/aarch64/bin/conda"

what Julia command did you use to install all this?

and jupyter kernelspec list returns

jupyter: Command not found.

jupyter doesn't work because it cannot find the python path

I get a kernel from new notebooks in my home directory. In subdirectories and/or with old notebooks I get the problem. I am working on an MWE.

What is MWE?

@ctkelley
Copy link

ctkelley commented Mar 26, 2023

Turns out my problem had nothing to do with Python. I had some stale Manifest.toml files that were breaking IJulia. See #1068 where I closed the issue.

MWE = minimal working example

@pulsedynamic
Copy link

Hi !

I noticed this issue pops up when you search for why doesn't Jupyter find installed IJulia kernel when installed in a project environment instead of default one.

It seems that either IJulia kernel is not installed OR the path where Jupyter looks for the kernel is not right. What I've done as a workaround is:

Install IJulia normally as a dependency for a project:
JULIA_PROJECT=<your_project_name> julia -e 'using Pkg; Pkg.instantiate();'

Then install an IJulia kernel through this command:
JULIA_PROJECT=<your_project_name> julia -e 'using IJulia; IJulia.installkernel("Julia");'

When installed like this, Jupyter seems to acknowledge it and you can keep it as a dependency alongside your other packages to maintain its versioning which is very useful when deploying this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants