Server: Buffer full

[ovove]

Staring with Zellij 0.34.4 I get occasional crashes. I am unfortunately not able to come up with a reproducible way recreate it.
Whenever I get the crash, the only thing I see is this message

Error occurred in server:
Buffer full

Basic information

zellij --version: 0.34.4
stty size: 46 178
uname -av: Darwin oMac 22.2.0 Darwin Kernel Version 22.2.0: Fri Nov 11 02:08:47 PST 2022; root:xnu-8792.61.2~4/RELEASE_X86_64 x86_64

List of programs you interact with as, PROGRAM --version: output cropped meaningful, for example:
zsh --version: zsh 5.8.1 (x86_64-apple-darwin22.0)
emacs --version: GNU Emacs 28.2
clang --version: clang version 15.0.6

Further information
n/a

[raphCode]

This usually happens when the client stalls and can't consume all the messages the server sent in a timely matter. Can this be related to high system load in your case?
So to speak this message is a "feature" that was introduce to stop zellij from hanging once a client doesn't respond.

Luckily you can reattach to your session afterwards.

[ovove]

Not sure about the load at the time of the crash. I have an 8 core macbook air, that I mostly use for some simple development during my free time. Nothing serious though. I'd be surprised if the load average goes much higher than 8-ish. But for sure, I can try to keep a close I to the load average and see if I can correlate it to any zellij crashes,

[imsnif]

Also, which terminal emulator are you using?

[ovove]

I'm using iTerm2

[danillos]

It is happening to me too. Iterm2 + Helix.

The editor was slow, but I'm pretty sure is not a resource issue.

I moved to Kitty and no issues so far.

Just an update: It happened with Alacritty too.

[aniketd]

This potentially causes loss of done work. I have seen this happen twice within the last 2 days.

uname -av: Linux nixos 5.15.85 #1-NixOS SMP Wed Dec 21 16:36:38 UTC 2022 x86_64 GNU/Linux
 terminal: Gnome Console
    panes:
      - 1 helix-editor
      - 4 other zsh sessions running programs that can generate a lot of output very quickly.

[imsnif]

Until I issue a fix for this (need to investigate a little further) I can recommend as a workaround to use a GPU accelerated terminal. This likely happens because the terminal emulator doesn't read STDOUT fast enough, Using something like Alacritty should help a lot.

[fbernier]

I've also been experiencing this on my ryzen desktop since the last release.

using:

• fully updated arch linux
• alacritty
• mosh
• fish shell

[sleepytraveler]

I run into this a lot where I'm running zellij on a remote machine I'm connected over ssh. Local terminal emulator can be kitty or wezterm - both hit the issue. Local shell and remote shell are fish. When a build command starts spewing a number of lines I get the "Buffer full" error.

Machine is running: Ubuntu SMP Tue Nov 22 19:54:14 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Haswell CPU with 16 cores running at 2.1GHz.

[imsnif]

Just FYI, there's a fix for this in the works: #2068

[alex35mil]

Alacritty. Also seeing it relatively often. Mostly when something produces a lot of output.

[matu3ba]

I experienced the very same problem with a pile of queued > events over ssh connection to resize a neovim window. (using hydra.nvim)

[raphCode]

Using ssh might increase the change to hit the error. As an workaround, try mosh inplace of ssh.

[sleepytraveler]

@raphCode - thank you for the suggestion to use mosh.

I am happy to report that while using mosh, the number of times I hit this issue is greatly reduced. Hopefully any one else running into this issue while using zellij on a remote system over ssh finds this discussion.

[alex35mil]

FWIW I do everything on local machine and still hit this relatively often. And I think quite often it happens when I scroll very fast using my mouse.

[raphCode]

Just connect with mosh to localhost 🤡

The PR was merged, so it might be better in the next release...

[bluss]

Using zellij 0.34.4 I can reproduce the "Buffer full" error using the following command:

/bin/ls -l --hyperlink=always -R

It reaches around line 35000 before it crashes (which happens quickly) it emits the "Buffer full" message

If I drop the hyperlink option it lists all 450 000 lines correctly.

ls version: ls (GNU coreutils) 8.30
Terminal: GNOME Terminal 3.36.2 using VTE 0.60.3 +BIDI +GNUTLS +ICU +SYSTEMD

[matu3ba]

It reaches around line 35000 before it crashes (which happens quickly)

If I get buffer full, then the server becomes completely unresponsive, so I need to manually kill it with kill -SIGKILL $(pgrep zellij).
Do you have the same problem?

[imsnif]

@matu3ba - can you help us consistently reproduce the case where the server becomes unresponsive? @har7an worked on fixing this issue (the fix is now in main) but could not reproduce this behavior. For us every time this happened we were able to reattach.

[hudon]

I was able to reproduce this error consistently, with this setup:

• macOS locally, SSH'd into Ubuntu 22.04.01
• locally: macOS, Running Alacritty with fish shell and zellij locally
• SSH'd into a server with Ubuntu 22.04.01, also fish shell, then running zellij again
• zellij 0.34.4 both local and remote

I run zellij nested like this bc I use it locally for tabs/panes, and remotely for sessions.

Once it's setup, I run ls -l /usr/bin (which has about 910 items) and scroll the mouse at the same time. This crashes the nested (ie. remote) zellij server every time with "buffer full".

I noticed that when I don't run it nested (ie. I SSH into my server without running zellij locally first), the crash does not happen.

[matu3ba]

can you help us consistently reproduce the case where the server becomes unresponsive?

No, I tried to reproduce it, but I was not able to do so yet.

I would emphasize to use Windows Terminal for WSL, which can handle incoming data much faster as that also was leading to buffer full for me. It is available in the Microsoft store.
The other shells appear to have an incomplete osc52 implementation, which works on zellij, but does not work for nvim-osc52.

Mosh is allowed to silently drop data (not even creating an offline queue on server or client etc with fallback slow but reliable mode), which I think is unacceptable. Aside, it does not support osc52 mobile-shell/mosh#637.
Dropping data or slowness is a fundamental shell problem having no separate control and data channel. Mosh could have improved that due to controlling server and client, but unfortunately they did not.

[hudon]

Installed 0.35.2 on both local and remote, and I can no longer reproduce this bug #2041 (comment). Thank you so much!!

[ovove]

I'm now at Zellij 0.36.0 and I haven't seen the reported issue for quite some time now. So perhaps this ticket can be closed now?

[matu3ba]

I'm now at Zellij 0.36.0 and I haven't seen the reported issue for quite some time now. So perhaps this ticket can be closed now?

Me too. Could be nice, if it is confirmed, that this specific buffer full error is logged properly to prevent silent/hidden regressions.

[har7an]

@matu3ba If this specific error does reoccur, it will now properly terminate the offending clients connection and print a pretty verbose error message (See #2068 (comment)). The server will keep running in the background and can usually be reattached to.

I'll close the issue now, feel free to reopen it in case it does occur again. Happy to hear it's fixed!