Make non-fullscreen apps possible

[alexwnovak]

What I'm Looking For

I'm interested in drawing controls over the current terminal, instead of switching away to a completely different screen.

What I've Tried

I've tried just about everything I can think of, but I always end up with a terminal full of blue. I'm on Windows 10, .NET 4.7.1.

• Creating a new Toplevel with a frame of 100x1 and using that with Application.Run
• I thought maybe the Init() call was goofing me up, so I tried subclassing Toplevel and setting those values myself, then using that subclass with Application.Run<T> to use the Func<Toplevel> overload of Application.Init
• A variety of Window or Toplevel instances that set the bounds smaller than the terminal extents

I didn't see anything in the docs or any Google results or any Stack Overflow tags. Am I just missing it? Or is this not supported? (If not, I can fill out an enhancement issue 😁)

Thanks!

[migueldeicaza]

I had not considered that scenario, so some work will be required.

This would only work on Windows where it is possible to control part of the screen without touching the rest, on terminals, we do not really know what the screen contains when we draw, nor can we "save" the contents that we draw on.

So possible, but you will have to figure it out :-)

[alexwnovak]

Thx for the reply. I had worried we'd need Win32 APIs and couldn't rely on VT sequences. Oh well. XD

[migueldeicaza]

Update: I have discovered that it is possible to read the contents of the terminal on some terminals. What terminal are you interested in?

If this were to work, we could create a "screenshot" view that would be initialized with data before the application starts to run. Perhaps an Application.Init (screenshot: true), and then we could surface an Application.StartupView that contains the original view that happens to be the top-level where you add elements. So it is technically going to repaint, but it will look like the original text.

This should work with some xterm-derivatives, not sure if this will work with other terminals like the ones used in VSCode, the Windows terminal and the Linux console.

[alexwnovak]

On Windows, I used to run ConEmu exclusively until Windows Terminal came out. On Mac I run zsh under iTerm2. I'm not much of a Linux user.

This is a neat idea, and I'd love it if it were a cross-plat solution that everyone could get in on, but I'm not holding my breath. 😎 Thanks for looking into it.

[tig]

FWIW, I really want this to be an option for out-consolegridview. See the use case of using OCGV for displaying history. In that use-case it would be ideal if OCGV didn't take over the whole screen.

PowerShell/ConsoleGuiTools#96

[JustinGrote]

Related to OCGV but would probably need to be implemented downstream, is if it took up half the view, and which half depended on where the current cursor position was. As long as gui.cs allowed a setting to choose "top half" or "lower half" then its probably possible.

[tig]

Related to OCGV but would probably need to be implemented downstream, is if it took up half the view, and which half depended on where the current cursor position was. As long as gui.cs allowed a setting to choose "top half" or "lower half" then its probably possible.

Neat idea. So ocgv -PopupRight would cause only the right part of the screen, starting at the prompt cursor location to be occluded by ocgv? ocgv -PopupBelow would cause only space below the prompt cursor to be occluded? Etc.?

[JustinGrote]

@tig I would expect it to be more intelligent than that. ocgv would either only take over the top half or bottom half of the screen depending on if your prompt was at the top of the screen (e.g. new window) vs bottom (youv'e been doing stuff a while). Basically the goal would be to not overlay over your existing command so you can maintain visual context of what you're working on.

[BDisp]

This is a neat idea, and I'd love it if it were a cross-plat solution that everyone could get in on, but I'm not holding my breath. 😎

I think the changes I did in the NetDriver, starting using virtual sequences may be the best solution for cross-platform, but it need to be worked better yet.

[BDisp]

Neat idea. So ocgv -PopupRight would cause only the right part of the screen, starting at the prompt cursor location to be occluded by ocgv? ocgv -PopupBelow would cause only space below the prompt cursor to be occluded? Etc.?

The recent changes I did in the ListView and TextView already manipulate the scrolling top/bottom and left/right. The ScrollBarView I had included is only for position indicator and notify fthe host for changes positions, but the scrolling redraw is up to host.

[migueldeicaza]

What we could do is ensure that the Toplevel only takes over a part of the screen, and that it does not go beyond it - at least this should be possible on Windows. To complete the story, we could retrieve the contents of the screen and store them while we run and restore them on exit.

This gave me the idea: instead of making the code work with a subset of the screen, we could have a "PreviousScreenshotView" that is a view that gets loaded with the original contents - this would allow for example to have views that can change size, windows popup and other things, without having to worry about not touching something in particular.

On shut down, this view could repaint the original contents.

This on Windows should be pretty easy.

For Unix, I never thought this could be possible, but then I remembered that in xterm and newer terminals there is a way of retrieving the contents of the terminal. This is what various test-suites use to validate that terminals use.

While this would be limited to xterm-like terminals, I think Windows + xterm should cover pretty much anything out there. The only Unix challenge would be to precisely map the previous screen attributes to ncurses attributes, but that is a minor problem.

[tig]

// I've changed the title of this Issue to make it a feature request.

[BDisp]

My PR #1331, with Mdi Container could be a solution for this. Setting Toplevel.IsMdiContainer as true we can create many Windows and Toplevels and navigate throught them.

[tthiery]

I would love using a terminal roughly like that ...

$ cat log.log
A
B
C
D
E
$ dotnet run app-with-gui.cs
==================================================================
| What is your favourite color: _Purple__________                |
| What is your favourite pet:   _Dog_____________                |
| [Save] [Cancel]                                                |
==================================================================
$ wc Program.cs
234134

I tried Specte.Console as an alternative but that library is not supporting multi-widget re-rendering and is more focused on outputting information. I want to use one of the two libraries to built a console UI for a work item library.

[jdhitsolutions]

I'll add my vote to this feature request. My primary use case would be a PowerShell session running in Windows Terminal.

[migueldeicaza]

After thinking about this for a few years, I think that the solution is simple.

We do need a "View" that is "OriginalView", and this can be used as the background view, upon which you would run everything else. The OriginalView would be initialized on Windows using the Console API to retrieve the contents of the screen, and on Unix/Xterms/network connections you can use the escape sequences to read the screen contents (DECRQCRA sequence).

It might not be fast, so I recommend that this is an opt-in feature.

The reason that it might not be fast is because DECRQCRA is used to checksum the screen, it is not really an API to read the contents, so you need to scan the columns/rows one at a time to capture the contents.

https://vt100.net/docs/vt510-rm/DECRQCRA.html
DECRQCRA

[tznind]

I'd like to understand more about this approach. Why store the screen history? Is the goal to have the 'background' view render that content?

We should identify use cases to better understand the requirements and complexity:

1. A static 'blocking' rectangular windowed app.
2. A draggable windowed app that renders underlying screen history (but is still blocking, meaning the user cannot continue using the terminal normally, i.e., task switching).
3. The ability to continue normal terminal use while a rectangular window renders a Terminal.Gui app.

These seem to be in order of increasing difficulty. I think the screen history bit is required for the second use case, and the third sounds quite challenging, if not impossible.

Are there other libraries that have this kind of feature we can investigate for ideas?

UPDATE:
Sorry I see you went into more detail above about why we need history. But still think it's worth clarifying what a 'first pass' at this feature would look like.

Are there any security considerations to reading old terminal buffer? It could have sensitive stuff in it.

[migueldeicaza]

Some terminals disable that escape sequence for security reasons.

But if you are running code, there is very little you can do to stop the app from doing whatever they want, like they could just an external screenshot app for example.

[tig]

Relevant. Mintty is leading the charge in many ways. See status line.

https://github.com/mintty/mintty/blob/master/wiki/CtrlSeqs.md

[tig]

Specifically here: https://github.com/mintty/mintty/blob/master/wiki/CtrlSeqs.md#status-line--area

It should be trivial to add support for this. Add a Application.RunAsTerminalStatusLine and Application.TerminalStatusLineHeight. When set, upon Init we would emit #"^[[2;{Application.TerminalStatusLineHeight}$~". On Shutdown, we would emit ^[[0$~.

I believe this would automatically constrain the TUI app to the bottom TerminalStatusLineHeight of the terminal.

[BDisp]

A view that displays the contents of the buffer is a good idea. However, this content does not differentiate between newlines whether it is a word wrap or whether it is actually a line break, because it does not store '\n' or '\r\n'. Therefore, when resizing the console, the formatting will be corrupted and it will never be possible to reformat correctly.

[tig]

A view that displays the contents of the buffer is a good idea. However, this content does not differentiate between newlines whether it is a word wrap or whether it is actually a line break, because it does not store '\n' or '\r\n'. Therefore, when resizing the console, the formatting will be corrupted and it will never be possible to reformat correctly.

Theoretically, it should be possible for a terminal to retain the source data in the back buffer so that it can resize the back buffer properly. I recall reading some terminal has actually but I don't remember which one.

Regardless, I'm now convinced that we can make great progress on addressing this customer need I simply taking advantage of statusline.

While it doesn't give the full flexibility of a "non-fullscreen " app it does meet the Spirit of it.

[dodexahedron]

Some terminals disable that escape sequence for security reasons.

But if you are running code, there is very little you can do to stop the app from doing whatever they want, like they could just an external screenshot app for example.

Very very rarely, as even things like less, man, etc depend on it for their default operation (though they do support just dumping to the active terminal as well).

[migueldeicaza]

I believe less/man rely on switching screens, not on reading the contents of the screen.

That is why it is preserved.

[dodexahedron]

Correct. I thought that's what you were referring to. The alternate screen buffer switching is how pretty much every app that preserved the terminal state works. Standard part of the C stdlib.

[migueldeicaza]

It is not, because the feature request here is to draw something that appears to leave the original buffer untouched.

When you switch to the other screen, you do not have any of the old contents, so we would not get the desired effect.

To get the desired effect, we need to somehow get the contents of the screen, so we can render at will and draw at will there.

You could draw on top of the existing UI, but in a destructive way (so dragging, or showing/hiding) would permanently destroy the contents.

[BDisp]

I tried something that get the contents but I'm struggling with non-bmp glyphs, color and resizing issues. But I'll try find any some workarounds to overcome them.

[dodexahedron]

Oh oh oh.

Meaning overlay of the current environment, not switching and restoring. 🤦‍♂️

Ok, I'm on the same page now.

[dodexahedron]

Yeah grabbing the screen contents without already being the owner of the TTY is a no-no for clearly obvious reasons.

We'd need to basically host a terminal, effectively becoming a whole shell to do it cleanly I think. Feels out of scope, especially for v2.

That said, if a user wants that kind of behavior, they can host their own terminal in their terminal.gui app.

[migueldeicaza]

While there is such a tool (XtermSharp - a full terminal emulator, and one that had support for an earlier version of Terminal.Gui) that does not let us "read" the existing content - it would only work if we started a new session there.

That said, the feature could be added to Windows without any of the downsides.

[tig]

None of you commented on my statusline idea above. Did you see it?

[migueldeicaza]

I did not quite understand how that would work.

That merely sets aside a region to not be changed (in rows), but my reading of the intent is to be able to layer an arbitrary set of views, and move those, while preserving the illusion of the background.

[tznind]

Perhaps we can tackle these possibilities seperately?

An isolated screen region that is fixed and runs a Terminal.Gui app is probably the easier of the two and still of value in many of these 'non fullscreen' use cases?

[jdhitsolutions]

I think my comment resurrected this thread so I feel obligated to chime in. First off, I am not a developer. I have been using this to build things in a PowerShell session. You can think of me as a power user. The technical details you've been throwing around are way above my pay grade.

What I would love to have, is the ability to create a smaller TUI that doesn't take over the entire console. I'd like to be able to create a small entry form that is displayed over the contents of the current screen. Like a popup. In a perfect world, it would be nice to specify a position in the terminal for the TUI. I wouldn't expect the user to have to reposition the TUI which sounds complicated to have to redraw the underlying terminal.

Where this gets (more) complicated is that I would need to support this in Windows, Linux and MacOS which means I have no idea what terminal might be running PowerShell.

To give you an idea, I can create this:

But it takes over the entire console.

If the consensus is that this is not practical, I'm OK. I figure it never hurts to ask the question.

[tig]

This is precisely why I suggested StatusLine. It does not allow arbitrary popups but DOES address the intent of the OP.

[BDisp]

An option that allow arbitrary popups where moving it will always showing the original contents is a better solution. As @migueldeicaza said using a View that holds the original content and refreshed when the popup moves is the better approach to achieve this. I tried with the TextView which allow wrap and custom rune cell color and it almost work. The problem was colors that isn't respecting the original content, some glyphs not rendered well and on console resizing it isn't possible distinguish between new lines and word wrap. I only started this on WindowsDriver but at least is something.

[tznind]

An option that allow arbitrary popups where moving it will always showing the original contents is a better solution.

I think it is worth trying both because as you say, its a tough task to 'deal with any arbitrary content that might be in console on load' in a cross platform way and also handling environmental factors such as SSH, PuTTy etc.

Running in fixed isolated area should be way easier and more robust.

It can even have the same API interface so application end user can switch between the two depending on runtime environment? e.g. with UserSettings

[BDisp]

It can even have the same API interface so application end user can switch between the two depending on runtime environment? e.g. with UserSettings

On Unix systems it's possible to switch between apps by pressing Ctrl-Z and typing fg to return to the previous app.