Help with using Notcurses through Neovim

[dccsillag]

I've finally been able to take some time to tackle an issue in one of my projects, dccsillag/magma-nvim#15. The idea over there is to use Notcurses in order to show images in the terminal in a more portable manner.

I've successfully come up with the following (sketched!) C code to show an image on some given position and dimensions:

#include <notcurses/notcurses.h>
#include <stdio.h>
#include <unistd.h>

int main(void) {
    struct notcurses_options options = {};
    struct notcurses* nc = notcurses_init(&options, NULL);
    if (!nc) {
        puts("Couldn't initialize notcurses");
        return -1;
    }

    // Create ncplane
    struct ncplane_options ncp_opts = {
        .y = 0,
        .x = 0,
        .rows = 100,
        .cols = 120,
    };
    struct ncplane* n = ncplane_create(notcurses_stdplane(nc), &ncp_opts);

    // Show image
    struct ncvisual* ncv = ncvisual_from_file("out.png");
    if (!ncv) {
        notcurses_stop(nc);
        puts("couldn't load ncvisual from file");
        return -1;
    }
    struct ncvisual_options ncv_opts = {
        .scaling = NCSCALE_SCALE,
        .blitter = NCBLIT_PIXEL,
    };
    ncv_opts.n = n;
    if (!ncvisual_blit(nc, ncv, &ncv_opts)) {
        notcurses_stop(nc);
        puts("couldn't blit");
        return -1;
    }

    // Render
    if (notcurses_render(nc)) {
        notcurses_stop(nc);
        puts("Couldn't render");
        return -1;
    }

    sleep(1);

    ncplane_erase(n);

    if (notcurses_render(nc)) {
        notcurses_stop(nc);
        puts("Couldn't render");
        return -1;
    }

    sleep(1);

    ncplane_destroy(n);

    return notcurses_stop(nc);
}

In my case, I need to write my code in Python. Based on the code above, I came up with the following script which does the same:

from notcurses.notcurses import lib, ffi
import sys
import time


# Initialize Notcurses
nc_opts = ffi.new("struct notcurses_options *")
nc_opts.flags = (lib.NCOPTION_NO_ALTERNATE_SCREEN
                 | lib.NCOPTION_NO_CLEAR_BITMAPS
                 | lib.NCOPTION_NO_WINCH_SIGHANDLER
                 | lib.NCOPTION_PRESERVE_CURSOR
                 | lib.NCOPTION_SUPPRESS_BANNERS)
nc = lib.notcurses_init(nc_opts, sys.__stdout__)
assert nc

# Setup plane
ncplane_options = ffi.new("struct ncplane_options *")
ncplane_options.y = 0
ncplane_options.x = 0
ncplane_options.rows = 50
ncplane_options.cols = 50
n = lib.ncplane_create(lib.notcurses_stdplane(nc), ncplane_options)

# Show image
ncv = lib.ncvisual_from_file(b"out.png")
assert ncv != 0

ncv_opts = ffi.new("struct ncvisual_options *")
ncv_opts.scaling = lib.NCSCALE_SCALE
ncv_opts.blitter = lib.NCBLIT_PIXEL
ncv_opts.n = n
assert lib.ncvisual_blit(nc, ncv, ncv_opts)

lib.ncvisual_destroy(ncv)

lib.notcurses_render(nc)

time.sleep(1)

# Erase the image
lib.ncplane_erase(n)

lib.notcurses_render(nc)

time.sleep(1)

# Clean up
lib.ncplane_destroy(n)
lib.notcurses_stop(nc)

This works as expected when run as python test.py (with that code in test.py). However, I need to run something along the lines of this Python code above from Neovim. When attempting to do so, via:

:py3 exec(open("/tmp/notcurses-tests/test.py").read())

I get no errors, but I see no output. The same thing happens if I do

:py3 os.system("./a.out")

where a.out is the executable compiled from the C code above.

I expect this was probably because Neovim did something like redirect stdout to somewhere weird.

I tried a bunch of hacky and non-portable solutions (e.g., go up the ppid tree to get the containing pty's file descriptor) but none of them worked at all.

So now I'm drawing a blank. What could I do to use basic notcurses functionality (creating a plane, drawing an ncvisual to it, rendering that ncvisual to a part of the terminal grid, and later erasing it) from a program that likes to do weird things with stdout? Or even, is there something I could do to help debug this?

In any case, thank you for your time!

[dankamongmen]

i'll look into this ASAP, but python is definitely not my area of expertise, so it might take a bit of exploration and experimentation. thanks for the report! i'll see what i can do to help you =].

[dccsillag]

Thank you very much! I understand Python is not quite your thing, take your time -- if you need/want help, hit me up.

[dccsillag]

BTW, this isn't a Python-specific issue -- it's more like a Neovim-specific issue, as evidenced by the fact that doing a system call of the compiled C code has the same result. That means that if we can solve this via C this should be solved on the Python end as well.

[dankamongmen]

yeah, realized that when i looked at it later. it's friday night here, and i've got some people coming over, but i've got this slated for my first thing to look into tomorrow =]. sorry for the delay

[dccsillag]

No worries, take your time! Again, thank you very much!

[dankamongmen]

alright, taking a look at this now =]

[dankamongmen]

ahhh ok so your entire issue is getting something displayed up from within NeoVim. gotcha. if it captures stdout, it will indeed not display. what if you tried something like ./a.out > /dev/tty? is redirection possible in this context?

if not, a small wrapper that opened /dev/tty and dup()ed it onto stdout ought work.

alternatively you could write the output to /dev/tty from within neovim?

[dankamongmen]

perhaps i ought add a NOTCURSES_OPTION_FORCETTY that attempts to write to the controlling tty rather than stdout...hrmmm.

[dankamongmen]

perhaps i ought add a NOTCURSES_OPTION_FORCETTY that attempts to write to the controlling tty rather than stdout...hrmmm.

this feels kinda unnecessary, since the calling code ought always be able to set up stdout the way they want. this is at least true in C. i'd think python could do so as well; let me look around a bit.

[dankamongmen]

so here you can see what's going down. first i run ncplayer -k, which dumps to stdout. i then rediect to atma.txt. nothing is displayed, but i can then cat that output, and it is displayed.

so yeah, neovim is taking that output up, presumably by creating a pair of pipes before execing and dup()ing that pipes to the child process's standard I/O. that way, it gets the output as a buffer. so you either need it to display that output without any processing (i doubt it will let you do this), or redup() /dev/tty in your program.

[dankamongmen]

btw note that the second output is oddly placed, obscuring the line from which it was launched. that's because there's cursor-positioning info inside that file. this is another reason why it would be better to redup() /dev/tty, as opposed to printing the output from within neovim.

[dankamongmen]

if this whole concept of dup()ing /dev/tty onto stdout isn't something well known to most developers, maybe i ought go ahead and do the option...

[dccsillag]

Just woke up to this! 😄

Okay, I can try duping into /dev/tty. But I have two questions:

1. How portable is dup()ing to /dev/tty? In particular, is it a POSIX thing or is it a Linux thing?
2. dup()ing is easy. However, I'm not sure how I can get the controlling tty; I actually already have some code which will give me the controlling pty, and I can try to use that, but first I'd like to know if they're the same thing; and, at the same time, know if there's some nice way of getting it (I think the code we have right now travels up the ppid tree until reaching a process with a pty)

[dccsillag]

Also, just to check, I'd be using dup2, not dup, right?

[dankamongmen]

dup2(), which is what you'll probably actually end up using, is UNIX Magic From Beyond the Dawn of Time, i.e. i'm pretty sure it precedes POSIX. it's certainly discussed at length in the first edition of Advanced Programming in the Unix Environment (Stevens 1992). i use it on BSDs and macOS. it's not on windows.

/dev/tty is an alias for the controlling tty =]

[dccsillag]

dup2(), which is what you'll probably actually end up using, is UNIX Magic From Beyond the Dawn of Time, i.e. i'm pretty sure it precedes POSIX. it's certainly discussed at length in the first edition of Advanced Programming in the Unix Environment (Stevens 1992). i use it on BSDs and macOS. it's not on windows.

Gotcha! I'll implement it and then ask for MacOS people to test it then. :)

/dev/tty is an alias for the controlling tty =]

Ah! I didn't know that! Yeah, I'm glad I asked that, haha

[dankamongmen]

some other things to note:

1. you'll need to position the cursor appropriately. if neovim leaves it where you want it, use NOTCURSES_OPTION_PRESERVE_CURSOR and the standard plane will be initialized with the cursor at that location.

2. you'll definitely want NOTCURSES_OPTION_INHIBIT_ALTERNATE_SCREEN (or whatever) and NOTCURSES_OPTION_NOBANNERS

[dankamongmen]

yep! sorry for being so slow in getting back to you on this; i saw your two code samples and filed this away as a "will have to go debug user's code" problem

[dankamongmen]

honestly ncplayer -k filename might do exactly what you want, once you have the dup(2).

[dccsillag]

yep! sorry for being so slow in getting back to you on this; i saw your two code samples and filed this away as a "will have to go debug user's code" problem

No worries!

[dccsillag]

Quick question -- with what options should I open /dev/tty? write-only?

[dankamongmen]

like i'm not promising antyhing but the following might work:

#!/bin/sh

ncplayer -k "$@" > /dev/tty

since you can hit /dev/tty from any scope, this uses the shell to do the rebinding, and then shows whatever with ncplayer. it might not work perfectly, but it ought suffice to test the basic idea.

[dccsillag]

Ah, fair enough

[dccsillag]

I'll try that

[dankamongmen]

Quick question -- with what options should I open /dev/tty? write-only?

all you ought need is write, yeah, unless you don't lol

[dankamongmen]

this is all contingent on neovim not redrawing the screen after it calls out, which it might do because of exactly this kind of thing. if that's the case, nothing's going to work; you'll need some different kind of neovim integration.

[dccsillag]

all you ought need is write, yeah, unless you don't lol

LOL

like i'm not promising antyhing but the following might work:

#!/bin/sh

ncplayer -k "$@" > /dev/tty

since you can hit /dev/tty from any scope, this uses the shell to do the rebinding, and then shows whatever with ncplayer. it might not work perfectly, but it ought suffice to test the basic idea.

I tried that, it sort of didn't work; I'll try to record a video, but essentially, instead of the escapes being interpreted by the terminal emulator, they were displayed in escaped form. Maybe neovim is the actual /dev/tty?

[dccsillag]

this is all contingent on neovim not redrawing the screen after it calls out, which it might do because of exactly this kind of thing. if that's the case, nothing's going to work; you'll need some different kind of neovim integration.

I'm pretty sure we don't need to worry about that, but we'll see.

[dccsillag]

I tried that, it sort of didn't work; I'll try to record a video, but essentially, instead of the escapes being interpreted by the terminal emulator, they were displayed in escaped form. Maybe neovim is the actual /dev/tty?

For some reason I can't reproduce that anymore... what happens now is the following: when run from Neovim, test.sh (which has that shell script you sent) exits with exit code 256, having shown nothing.

[dankamongmen]

ok, i'll have to look at what neovim is doing, exactly. i assume the script does work outside of neovim?

[dccsillag]

Oh yeah, it does.

[dccsillag]

(I'm takiing a quick break from this now, brb)

[dankamongmen]

yeah i've got a few other things to cycle to as well. here's the upshot: if you can get something displayed from outside neovim within your script (i.e. it's not just neovim printing it after you return), you ought be able to use the same method to get this working.

what terminal btw? if you throw an export | grep TERM in your script, what TERM is being exported? any?

also try notcurses-info -v and paste any output it generates (it'll be writing to stderr). let's see.

[dccsillag]

For some reason I can't reproduce that anymore... what happens now is the following: when run from Neovim, test.sh (which has that shell script you sent) exits with exit code 256, having shown nothing.

Looks like it actually exits with exit code 1, having complained about the following:

./test.sh: line 4: /dev/tty: No such device or address

what terminal btw? if you throw an export | grep TERM in your script, what TERM is being exported? any?

Same as my terminal, st:

export COLORTERM="truecolor"
export TERM="st-256color"

Similar results if we run it instead on Kitty:

export COLORTERM="truecolor"                                                                                          
export TERM="xterm-kitty"                                                                                             
export TERMINFO="/usr/lib/kitty/terminfo"                                                                             

also try notcurses-info -v and paste any output it generates (it'll be writing to stderr). let's see.

The output is quite large. I've attached a file with that, generated via notcurses-info -v &> ncinfo.txt:

ncinfo.txt

[dankamongmen]

./test.sh: line 4: /dev/tty: No such device or address

very strange!

is there no /dev/tty device available to you outside of neovim?

what happens if you launch ls -l /dev/tty in place of the script from within neovim?

do you have a link to the documentation for running stuff from neovim? does it describe what it's doing to that environment?

[dccsillag]

./test.sh: line 4: /dev/tty: No such device or address

very strange!

is there no /dev/tty device available to you outside of neovim?

what happens if you launch ls -l /dev/tty in place of the script from within neovim?

Outside Neovim:

~/tmp/notcurses-tests $ ls -l /dev/tty
crw-rw-rw- 5,0 root 31 out 07:25 /dev/tty

Inside Neovim:

:!ls -l /dev/tty
crw-rw-rw- 1 root tty 5, 0 out 31 07:25 /dev/tty

Huh.......... it's even the same if I do it from the script...

:!./test.sh
crw-rw-rw- 1 root tty 5, 0 out 31 07:25 /dev/tty
export COLORTERM="truecolor"
export TERM="st-256color"
./test.sh: line 5: /dev/tty: No such device or address
 
shell returned 1

The script looks like this now, btw:

#!/bin/sh
 
ls -l /dev/tty
export | grep TERM
ncplayer -k cat.png > /dev/tty

do you have a link to the documentation for running stuff from neovim? does it describe what it's doing to that environment?

I'll try to look for that. Worst case scenario we can open an issue there asking for this information.

[dccsillag]

With :help :!

							*:!cmd* *:!* *E34*
:!{cmd}			Execute {cmd} with 'shell'. See also |:terminal|.

			The command runs in a non-interactive shell connected
			to a pipe (not a terminal). Use |:terminal| to run an
			interactive shell connected to a terminal.

			Backgrounded ("&") commands must not write to stdout
			or stderr, the streams are closed immediately. |E5677|
			Use |jobstart()| instead. >
				:call jobstart('foo', {'detach':1})
<
			Any "!" in {cmd} is replaced with the previous
			external command (see also 'cpoptions'), unless
			escaped by a backslash.  Example: ":!ls" followed by
			":!echo ! \! \\!" executes "echo ls ! \!".

			Any "|" in {cmd} is passed to the shell, you cannot
			use it to append a Vim command.  See |:bar|.

			Any "%" in {cmd} is expanded to the current file name.
			Any "#" in {cmd} is expanded to the alternate file name.
			Special characters are not escaped, use quotes or
			|shellescape()|: >
				:!ls "%"
				:exe "!ls " . shellescape(expand("%"))
<
			Newline character ends {cmd} unless a backslash
			precedes the newline.  What follows is interpreted as
			another |:| command.

			After the command has been executed, the timestamp and
			size of the current file is checked |timestamp|.

			If the command produces too much output some lines may
			be skipped so the command can execute quickly.  No
			data is lost, this only affects the display.  The last
			few lines are always displayed (never skipped).

			To avoid the hit-enter prompt use: >
				:silent !{cmd}

Also, :help shellredir:

						*'shellredir'* *'srr'*
'shellredir' 'srr'	string	(default ">", ">&" or ">%s 2>&1")
			global
	String to be used to put the output of a filter command in a temporary
	file.  See also |:!|.  See |option-backslash| about including spaces
	and backslashes.
	The name of the temporary file can be represented by "%s" if necessary
	(the file name is appended automatically if no %s appears in the value
	of this option).
	The default is ">".  For Unix, if the 'shell' option is "csh" or
	"tcsh" during initializations, the default becomes ">&".  If the
	'shell' option is "sh", "ksh", "mksh", "pdksh", "zsh", "zsh-beta",
	"bash" or "fish", the default becomes ">%s 2>&1".  This means that
	stderr is also included.  For Win32, the Unix checks are done and
	additionally "cmd" is checked for, which makes the default ">%s 2>&1".
	Also, the same names with ".exe" appended are checked for.
	The initialization of this option is done after reading the vimrc
	and the other initializations, so that when the 'shell' option is set
	there, the 'shellredir' option changes automatically unless it was
	explicitly set before.
	In the future pipes may be used for filtering and this option will
	become obsolete (at least for Unix).
	This option cannot be set from a |modeline| or in the |sandbox|, for
	security reasons.

In my case, I have shellredir=>%s 2>&1 (the default).

I'll take a look if anything different happens with that jobstart, but I think the /dev/tty situation might be explained by the following bit from the start of :help :!: "The command runs in a non-interactive shell connected to a pipe (not a terminal)".

(The :terminal thing it mentions right after would run the command inside Neovim's own terminal emulator, which I've already verified doesn't handle any proprietary image formats.)

[dccsillag]

You know, if you don't mind, I think I'm actually going to open an issue on Neovim itself, asking about how we should do this. Someone there ought to know.

[dankamongmen]

sorry to have been distracted away from this, but i intend to look at it today!

[dankamongmen]

did you hear anything back from the neovim people? they were probably busy getting 6.0 out the door, but maybe they can provide some insight now. if you've got a bug link, do please provide it!

[dccsillag]

sorry to have been distracted away from this, but i intend to look at it today!

No worries at all. Same here.

did you hear anything back from the neovim people? they were probably busy getting 6.0 out the door, but maybe they can provide some insight now. if you've got a bug link, do please provide it!

Just opened a topic on their discourse: https://neovim.discourse.group/t/accessing-the-tty-from-neovim-to-show-images-with-notcurses/1597. Apparently that's where they like their questions.