Nudge: resolving shell future -- header missing date

[apowers313]

Describe the bug
I'm developing a Jupyter kernel using Rust, which is strongly typed and will panic if JSON messages don't conform to the Jupyter protocol spec.

It appears that notebook's Nudge sends a message without the date field in the header. After turning on debugging in the Jupyter config, this is what I see NotebookApp sending:

[D 12:18:27.630 NotebookApp] Nudge: shell info reply received: 63ed6dfd-55ab-4ebf-a4f3-c0a2008266d1
[D 12:18:27.630 NotebookApp] Nudge: resolving shell future: 63ed6dfd-55ab-4ebf-a4f3-c0a2008266d1
{'buffers': [],
'content': {},
'header': {'msg_id': 'e0faed03b3f14c1e8583a1fb2932838a',
'msg_type': 'kernel_info_request',
'session': 'e143836731e24cc294f3bca537db0c8d',
'username': 'username',
'version': '5.2'},
'metadata': {},
'parent_header': {}}
[b'<IDS|MSG>',
b'03792d724070ed3eb024166cbcdfbb25e9999f525800b8537bfad65ff46d9922',
b'{"msg_id": "e0faed03b3f14c1e8583a1fb2932838a", "username": "username", "sess'
b'ion": "e143836731e24cc294f3bca537db0c8d", "msg_type": "kernel_info_request",'
b' "version": "5.2"}',
b'{}',
b'{}',
b'{}']
[]

To Reproduce
I'm not sure how to get the kernel into the "Nudge: resolving shell future" state...

Expected behavior
header should include date field.

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

• OS: MacOS 10.15.7
• Browser Chrome 96.0.4664.110

jupyter --version
Selected Jupyter core packages...
IPython : 7.29.0
ipykernel : 6.5.1
ipywidgets : not installed
jupyter_client : 7.0.6
jupyter_core : 4.9.1
jupyter_server : 1.11.2
jupyterlab : 3.2.4
nbclient : 0.5.9
nbconvert : 6.3.0
nbformat : 5.1.3
notebook : 6.4.6
qtconsole : not installed
traitlets : 5.1.1

[kevin-bates]

Hi @apowers313. In looking into this, I believe this is an issue in the Notebook front-end that has been in existence prior to the "nudge" functionality (introduced in the 6.1.6 release - so you should see this with notebook<=6.1.5 as well).

I suspect your Rust kernel is relatively new and, because you're seeing the debug messages relative to the nudge functionality, and because the nudge functionality is focused on kernel_info_request sequences, it appears to be associated. However, the nudge functionality relies on the jupyter_client library to construct the message header - which unconditionally sets a date field along with a 5.3 protocol version indicator.

Since the front-end will always follow up the kernel connection establishment with a kernel_info_request of its own, what you see is a request from the front-end that is serviced by the on_message() method of the handler - which essentially passes the message payload directly to the jupyter_client send() method, that, in this case, assumes the payload to be valid, resulting in the kernel's panic.

I have verified that JupyterLab constructs an appropriate message header, so would think your Rust kernel will behave correctly using jupyter lab.

It also looks like we should follow up with a fix to the message construction in the notebook front-end to include an appropriate date field and value. Is that something you'd like to tackle?