What's the goal of the TUI mode?

[favonia]

Hi, I was curious about dune build --watch --display tui when I saw 3.11 is out, but then got confused about its purpose after trying it. What do I gain besides dune build --watch? It seems I cannot do anything more than that. I checked through GitHub issues tagged with tui and became more puzzled. It seems the leading developer of this feature is targeting the integrated terminal inside VS Code (see #8770), and maybe there are plans to tighten the integration with the editor. However, if we are already inside VS Code, wouldn't it be easier to use LSP instead? What's the goal of this new TUI mode within VS Code? I failed to find more information from the changelog and the codebase, and felt I must have missed something here.

---

PS: There are also some Unicode issues: It assumes ᚛, ―, and ᚜ will always align, but that's not the case in my terminal font (and nothing in the Unicode standard implies they should). It also seems to assume the box drawing characters with "Ambiguous" East Asia width always have the visual width of one cell, which is unfortunately not true, either, and that's why they are "Ambiguous". However, before trying to fix the character choice, I feel it's more important to understand the goal of the TUI mode.

[favonia]

Hi, Thanks for your interest. I suggest opening a discussion since this is not an issue with dune and there's no action required to fix it.

@emillon I think it is an issue with dune: maybe the new mode should be redesigned, and at very least the problematic Unicode characters must be replaced and more documentation should be added. My concern implies that changes need to be made. However, I agree that we don't have a consensus on what exactly should be done yet, so maybe a discussion is indeed more appropriate. I believe (or hope) there will be actionable items generated from this discussion.

[Alizter]

I have known about the potential difficulties East Asian character sets present for TUI, but I haven't yet got a solution to dealing with them. I will need to do some more research and ask other developers how they handle this problem. AFAIK there isn't a silver bullet but there should be many good-enough solutions. Please do create an issue specifically about it however.

Regarding the rendering of the ogham relative to a horizontal rule. I also know about this issue and I have not yet found a better solution. Whether or not this symbol aligns depends on the font the terminal emulator is using. In my testing this has been the case for around half. Other alternative characters are too clunky in my opinion, hence why I haven't changed it. I'm open to suggestions but it's not a priority right now. Please again open a specific issue for this.

[favonia]

@Alizter Hmm. I think you might have confused "East Asian character sets" with "East Asian widths." They are very different. Please consult UAX 11 for detailed definitions and discussions. To clarify, my comment about East Asian widths has nothing to do with "East Asian character sets."

[Alizter]

@favonia It seems I don't fully understand the issue then. I understood what you wrote to mean: "if you use box drawing characters and east asian characters then they don't align due to the ambiguous east asian width of the box drawing characters". How do the east asian character widths apply when not using those character sets?

[favonia]

East Asian widths are a property of Unicode characters that is not defined only for East Asian characters. Please consult UAX 11 of the Unicode standard.

[favonia]

Forgot to say this: East Asian (legacy) character sets contain non-"East Asian" characters. Maybe that's why you were confused by the width property? Again please consult the Unicode standard---I could explain it but prefer not to spend too much time repeating the relevant parts of the standard while trying to focus on my main question (why we need the TUI mode in dune).

[Alizter]

I agree that there is not much documentation available so hopefully I can clear some things up. I've been meaning to write some documentation on TUI eventually so this might be a good place to start.

What is the TUI mode?

The Dune TUI (terminal user interface) is intended to be an easy-to-use, portable and interactive alternative to the usual command-line usage of Dune.

What does it do?

Today in 3.11 you can click on messages to minimise them, open up a help screen, resize your window as you like, scroll through messages, and quit using q.

What are the plans for the future?

The current features seem limited, and that's OK. Hence why we marked TUI as "experimental" in the 3.11.0 change log.

The current feature I am working on that will build on TUI is the jobs tab:

• feature(monitor): add Jobs tab in tui mode #8601

This will allow the users to view the status of processes that Dune is running and interact with them appropriately.

In order to not have such a wide feature disparity with regular Dune commands, we also wish to implement a subset of the functionality for regular the dune monitor command. Unfortunately this means taming a long standing issue with multiline status lines being cleared improperly, which is quite a complicated thing to do. This is what I am currently working on and the latest progress can be found in #8619.

We have all sorts of other ideas like:

• Clicking a button to promote a diff.
• Modifying build goals
• Displaying jobs (which I mentioned above)
• Navigating to errors

Each of these will have their own specific features soon, but for the time are working on them one-at-a-time.

How does it work?

The UI is based on the Notty library and allows us to draw images easily without worrying too much about rendering considerations within Dune. It also uses nottui from lwd which provides support for UI elements based on Notty.

In the first release of TUI back in 3.8.0, we drew the exact same thing as --display quiet using only Notty and with version 3.11.0 we added support for some limited interactivity to further demonstrate the utility.

GUI frameworks have been around for years; Why do all this in the terminal?

The main answer is portability. We would like Dune to:

1. Run on just about every platform supported by OCaml.
2. Do so with no system dependencies (or an acceptable few in the case of watch mode).

It's very difficult to find a GUI framework that meets these two requirements. Any sophisticated GUI framework will have a library that needs to be depended on, and simpler frameworks tend to be lacking in cross-platform support.

There are likely two things that anybody running Dune has: A terminal and a web browser. Both of these options satisfy our constraints and we have chosen to go with the first since that's how everybody is running Dune already. It is also probably more familiar to programmers and light weight compared to spinning up a browser.

Another advantage of having a terminal based application is that it becomes easy to use Dune via other modes of interaction such as ssh. This coupled with the dune monitor command #8152 will allow users to inspect remote Dune processes.

TUI is also very suitable for usage in a terminal multiplexer such as tmux.

How does this interact with VSCode/OCaml Platform/LSP?

The TUI mode is not meant to replace any LSP functionality but only to compliment it. As more features get added it can pave the way for similar features to be added elsewhere.

In the future, Dune will probably become based more on RPC which means that regular users will interact with it in the command line less frequently, instead interacting via LSP to build, run tests, promote files etc. In that world, it makes sense to have a way to connect to a running Dune process which is what dune monitor does (and will likely be what other Dune commands do too). When this is done, it will be helpful to have a nice easy-to-use interface that is not tied to a particular LSP client.

Where TUI is really useful is for Dune/OCaml specific features that LSP does not support. This allows the feature to be implemented once for Dune rather than leaving it to the other LSP clients to additionally support.

What are some other examples of other TUI applications?

My go to example would be lazygit https://github.com/jesseduffield/lazygit which also has some similar design constraints to us. It is essentially an interactive git client that runs in your terminal and is meant to be easy to use.

For more examples on GitHub see here: https://github.com/rothgar/awesome-tuis

[favonia]

@rgrinberg Thank you for the comment that a PR (if I find time) will not be wasted. However, I hope I can get some clarification on your comment about the 3.7.0 versus 3.8.0. The vendering of notty and uutf did not seem to happen in 3.7.0 because you have personally reverted the vendoring in 8a4a113 (#6780) before the final release of 3.7.0), and the vendoring of 0install did not seem to happen in 3.8.0, either. (Yes, that seems to have caused the 5-second increase, and I understand that might be needed, so I did not complain about it!) Did I get this part of the history wrong? The detail is very relevant here because it can be a good estimation about how much saving we could have by moving the fancy TUI into a separate package. I will be much less motivated if the difference is only one second.

[rgrinberg]

No, you're right. You should be able to get accurate information by removing TUI temporarily.

[favonia]

@ejgallego Thanks for your comments. I'm curious about the CI: we are using https://github.com/ocaml/setup-ocaml and its designer has made a deliberate choice not to cache the entire opam env. I already observed that the uploading of the archived directory takes quite some time so I tend to agree with the designer (until I have time doing more experiments). Do you have evidence showing it's not the case? If so, maybe we should change how setup-ocaml works?

[favonia]

@ejgallego The long story is that we have tried the Docker approach and the cache-it-all strategy before we transitioned to setup-ocaml, and both can be annoying but I don't want to elaborate here. The existence of https://github.com/ocaml/setup-ocaml really saves us lots of maintenance efforts. It sounds that you are suggesting a more efficient script with lower maintenance, and I am very interested.

[favonia]

@rgrinberg Thinking about it, even if I manage to make a PR to make it a package, which will probably make me easier (shorter build time), I probably would not have time to maintain it, at least not as actively as @Alizter. I'm still holding the view that it's better to make the tool a separate package, but I'm also following the same principle of "people who contribute most, decide". I have said enough in this discussion and perhaps I should leave it as it is now. @emillon @ejgallego Thank you all for the conversation.