execute-cell doesn't work on password-protected notebook server

[krvkir]

Check list

• Read the "Avoiding Common Emacs Traps" section
  • attempt to start zeroein as described in manual gave nothing: emacs seems to ignore lisp code in zeroein.el, the only strange message I noticed was "eval-buffer: Buffer is read-only: #<buffer Messages>"
• Does IPython works from the web browser?
• Confirm the problem was not due to badly compiled
  files by removing all the *.elc files from source directory of EIN and
  its dependencies.
• Confirm that the libraries are loaded as expected and are at the required version.
  (You can check the location in the "System info" section below)

Description of the problem

When running the ipython notebook server protected with password, attempt to execute any cell in any notebook fails with "Kernel is not ready yet!" message.

Steps to reproduce the problem

1. configure ipython notebook to use password (as described in http://ipython.org/ipython-doc/1/interactive/public_server.html)
   • create ~/.ipython/profile_nbserver/ipython_notebook_config.py
   • add into it the line c.NotebookApp.password = "your-password-sha1-hash"
2. run ipython notebook --profile=nbserver
3. open some notebook with ein and run any cell
   • ein:notebooklist-login RET <port> RET <password>
   • ein:notebooklist-open
   • open any notebook
   • run any cell with ein:worksheet-execute-cell
4. now comment the line with the password in ipython_notebook_config.py
5. open some notebook and execute cell -- it works

Expected output

ein: [warn] Kernel is not ready yet! (or closed already.)

Your EIN configuration (in .emacs.d/init.el or somewhere else)

no special configuration (all configs have default values)

Your IPython configuration

1. What is your IPython version? (run ipython --version):
   3.1.0
2. How do you start IPython? (e.g., ipython notebook --port 9999):
   (see "Steps to reproduce the problem")
3. What is your IPython notebook port number or URL?:
   localhost:8888 or localhost:9999 (seems like port does not make difference)

Additional information (if any)

• tried running the server on two archlinux machines
• tried connecting to server from archlinux and windows 7 computers

System info:

`ein:dev-sys-info' produce: (void-variable python-version)```

<-- Have no idea what is it, few days ago I tried generating report and system info was here...

While creating this buffer the following message apprears:
"Eager macro-expansion failure: (error "(ein:$kernel-stdin-activep kernel) is not a valid place expression") "

[millejoh]

Actually a duplicate of #17, but thanks for the info. I think I have a clue of what some of the problem is and will try to look at this over the next few days as time permits.

[millejoh]

This is what I have found out so far:

1. You can authenticate with the IPython/Jupyter notebook server using ein:notebooklist-login. After calling this a cookie is generated (very easy to see if you are using curl as the backend for emacs-request) and you can then use the REST API to list and get notebook data.
2. The problem starts when ein tries to open a websocket connection to the kernel. The notebook server generates a 403 forbidden response. I think because emacs-websocket doesn't know anything about the security cookie generated during the curl request.

Not sure if that makes sense, but for the moment that is my theory on what's happening. Somehow we need to provide the security cookie with the websocket connect request.

[millejoh]

I understand now what the problem is. Unfortunately it requires some changes to emacs-websocket. I have submitted an issue (ahyatt/emacs-websocket#38) so we will see what happens.

[millejoh]

Sorry for the wait, but this should be working now, as of 127039e. emacs-websocket just added this functionality, as of 542c6fec7447119c681d2a87da90a40223a81a36, and I have merged in support into ein. If you are downloading from MELPA you should see the updates sometime today. Let me know if you run into any problems.

[krvkir]

Sorry but it still doesn't work for me. Updated everything updatable from MELPA to no avail. When I try to execute cell I still get this message:
ein: [warn] Kernel is not ready yet! (or closed already.)

Maybe there's something I'm doing wrong?

PS Here is my system info:

("EIN system info"
 :emacs-version "GNU Emacs 24.5.1 (x86_64-w64-mingw32)
 of 2015-03-28 on KAEL"
 :emacs-bzr-version nil
 :window-system w32
 :emacs-variant nil
 :os
 (:uname nil
         :lsb-release nil)
 :image-types
 (svg png gif tiff jpeg xpm xbm pbm)
 :image-types-available
 (svg png gif tiff jpeg xpm xbm pbm)
 :request
 (:backend url-retrieve)
 :ein
 (:version "0.7"
           :source-dir "~/AppData/Roaming/.emacs.d/elpa/ein-20150705.2027/")
 :lib
 ((:name "websocket"
         :path "~/.emacs.d/elpa/websocket-20150626.2049/websocket.el"
         :featurep t
         :version-var websocket-version
         :version "1.4")
  (:name "request"
         :path "~/.emacs.d/elpa/request-20140316.417/request.elc"
         :featurep t
         :version-var request-version
         :version "0.2.0")
  (:name "auto-complete"
         :path "~/.emacs.d/elpa/auto-complete-20150618.1949/auto-complete.elc"
         :featurep t
         :version-var nil
         :version nil)
  (:name "auto-complete"
         :path "~/.emacs.d/elpa/auto-complete-20150618.1949/auto-complete.elc"
         :featurep t
         :version-var nil
         :version nil)
  (:name "popup"
         :path "~/.emacs.d/elpa/popup-20150626.711/popup.elc"
         :featurep t
         :version-var popup-version
         :version "0.5.3")
  (:name "python"
         :path
         "~/_soft/emacs-bin-w64-24.5-rc1/emacs/share/emacs/24.5/lisp/progmodes/python.elc"
         :featurep t
         :version-var nil
         :version nil)
  (:name "python-mode"
         :path "~/.emacs.d/elpa/python-mode-20150703.143/python-mode.el"
         :featurep nil
         :version-var nil
         :version nil)))

[millejoh]

Is there a cookie file in the ~/.emacs.d/request/? (I was going to say that using curl as a backend for request is causing the problems, but apparently you are using url-retrieve per the above info).

[millejoh]

Another thing to check - make sure you are downloading websocket from melpa and not gnu, I noticed when I download the 1.4 version from gnu I see the same behavior that you do.

In my (partial) defense - I didn't even know until I tried downloading from melpa that there was also a package in elpa. I guess in general life is much better now in Emacs with package management, but maybe not so much in this case.

[krvkir]

Thanks for your replies.

There's no cookie or even request folder in my ~/.emacs.d.

I found these lines appearing in *Messages* buffer after ein:notebooklist-login:

REQUEST [error] Callback is not called when stopping process! Explicitly calling from timer.
ein: [info] Login to http://10.0.10.62:9999 complete. Now you can open notebook list by `ein:notebooklist-open'.

After that ein successfully connects to notebooks catalog but fails to execute cells.

PS I tried this with both elpa and melpa versions of websocket, nothing changed.

[millejoh]

I get those same messages, but I can get the websocket connection to open. Jupyter doesn't return a 200 status code when authenticating, which confuses request.

The websocket connection is failing because it can't find the authentication cookie. We can double check this if you call ein:dev-start-debug before trying to open a notebook. After the opening the notebook go to the websocket debug buffer, name should be something like `websocket ws://..... What are its contents?

Two other things to check:

1. After succesfull call to ein:notebooklist-login. Call M-x ielm, then type (request-cookie-alist "127.0.0.1" "/"). Tell me what result it returns, if any.
2. Call M-x customize-group RET request. What is the setting for `Request Storage Directory'?

[millejoh]

Finally got this one, I hope, with #94.

[3rd3]

I am still getting this error with ein 20160422.629.

[3rd3]

Setting the Request Backend to url-retrieve solved it for now.