Add support for ANSI colors

[kimikage]

This starts from a Discourse topic: https://discourse.julialang.org/t/how-do-i-enable-ansi-colored-text-output-within-documenter-jl/48398

This changes:

• add a new field / argument ansicolor to User / makedocs()
  • for global setting
• add support for a keyword argument ansicolor in the @example block
  • for per-block setting to overide the global setting
• use the ansicolor option for the :color I/O context property
• translate "text/plain" output with ANSI escape codes in HTMLWriter
• add styles for ANSI colors to the default themes

These changes allow you to colorize the lazy print objects in the @example output blocks.

However, there are some issues to be resolved.

I've put an ANSI escape code translator for HTML in a new package AnsiColoredPrinters.jl to make this feature work. The package is experimental and under development, and there are things left to be discussed before it is registered, including the name of the package. I won't refuse to contribute it as part of Documenter.jl, but I prefer to modularize the functionality into minimal packages.

The other is that currently there is no way to specify the :color I/O context property for output to stdout / stderr. (Edit: solved by JuliaDocs/IOCapture.jl#1)
This means that even if the @example or @repl code potentially supports ANSI colors, we cannot get the colored result.

Also, although ansicolor is false by default and is the responsibility of the user, we need to strip the ANSI escape codes if the output format does not support ANSI colors. (This will be implemented soon.)

When the solutions for the above are found, we will need to discuss the compatibility with highlight.js, the theme for system 16 colors, etc.

[fredrikekre]

Nice! Is there ever a reason not to have this by default? Perhaps we can just look at the command line flag instead.

[kimikage]

One problem with IOCapture is that redirect_{stdout|stderr}(::IOContext) is only available for Julia v1.6 (or later). (cf. JuliaLang/julia#36688)
Possible measures include:

• giving up the color display of stdout/stderr on v1.5 or earlier
• emulating PipeEndpoint as a subclass of LibuvStream
• pirating get(::PipeEndpoint, key, default)

Do you have any other good ideas?

[fredrikekre]

giving up the color display of stdout/stderr on v1.5 or earlier

I think this is fine.

[kimikage]

giving up the color display of stdout/stderr on v1.5 or earlier

I think this is fine.

I also think it's the safer of the three. Even if we don't offer the colored stdout/stderr as a feature of these packages, perhaps defining get(::PipeEndpoint, key, default) in the user code (i.e. "docs/make.jl"), would not be too much of a problem.

[kimikage]

There is still a lot of work to be done, but the stdout/stderr and @repl blocks are now colored in HTML.
Also, although I haven't fully tested it yet, the interference with highlight.js (v10.3.1) doesn't seem to be a problem. I'm looking forward to the improved highlighting. 😍

As suggested in #1441 (comment), instead of adding the ansicolor keyword argument to makedocs, the :color property of stdout, which is determined by the --color command line flag, is now used as the default value.

[kimikage]

Currently, I added the ansicolor keyword argument to the @example/@repl block, but it might be better to generalize it to IOContext. (cf. issue #1450)
However, we will face the problem of PipeEndpoint again. 😕

[kimikage]

The system 16 colors are now generated from variables such as $red by default.

So, I changed the default color scheme while maintaining as much of the current look as possible. (I'm going to separate the color scheme changes into another PR.)
Edit: The color change was purged for now.

[mortenpi]

I've put an ANSI escape code translator for HTML in a new package AnsiColoredPrinters.jl to make this feature work. The package is experimental and under development, and there are things left to be discussed before it is registered, including the name of the package. I won't refuse to contribute it as part of Documenter.jl, but I prefer to modularize the functionality into minimal packages.

I think the functionality is quite well-defined and general, so it make sense to have it as a separate package. Would you mind having that under the JuliaDocs umbrella though?

I would love to have you as its primary maintainer, but it would be good if there is a wider set of people who would can, if need be, release quick fixes, since the Julia master branch will be depending on this code.

[kimikage]

Would you mind having that under the JuliaDocs umbrella though?

After making some miscellaneous changes (I think they will take about a week), I'd like to transfer the package to JuliaDocs. 😃

However, before we do that, we need to decide on the package name. Even if it is based on the current name, I'd like to change "Ansi" to "ANSI".

[mortenpi]

However, before we do that, we need to decide on the package name. Even if it is based on the current name, I'd like to change "Ansi" to "ANSI".

I also think that the capitalized name would look better.

[kimikage]

ANSIColoredPrinters' documentation is still unfriendly, but its code coverage is now over 90%. So I will apply for the transfer.
https://github.com/kimikage/ANSIColoredPrinters.jl/issues/10

[kimikage]

Sorry I haven't been able to update this PR. I will try to rebase it in a few days.

[fredrikekre]

Thanks for the rebase. Something seems wrong with the dark theme:

[fredrikekre]

It also looks like hljs overrules things like

<code class="language-julia-repl ansi">julia&gt; printstyled("hello, world\n"; color=:blue)
<span class="sgr34">hello, world</span></code>

turning it into

<code class="language-julia-repl ansi hljs"><span class="hljs-meta">julia&gt;</span><span class="language-julia"> printstyled(<span class="hljs-string">"hello, world\n"</span>; color=:blue)
</span>hello, world</code>

[fredrikekre]

For

```@repl
println("hello")
```

perhaps we need to emit something like

<pre>
<code "language-julia-repl">
println("hello")
</code>
<code "nohighlight">
hello
</code>
</pre>

i.e. split into two blocks where only the first is used with hljs

[fredrikekre]

I have some fixes for my last two comments which I can push after. I think that this PR can be merged once the dark theme css is fixed and I will make a follow up PR:

[fredrikekre]

OMG in the test package I tried this on I had a custom css file overriding the documenter one.....

[kimikage]

I was considering the approach of writing a highlight.js plugin, but the plugin API does not work well with DOM manipulation.
For this reason, I think the approach proposed above of separating the REPL input and output is more elegant.

In this PR, I will postpone the ansi color support for @repl (comment out some code) and write documentation and tests.

[fredrikekre]

Sounds good

[fredrikekre]

.

[mortenpi]

Thank you guys for working on this! Fredrik and I briefly discussed this on Slack how to proceed with this in terms of versions:

• We'll make this opt-in 0.27.4, to get this out to the users faster (to be toggled by a new ansicolor keyword to HTML).
• We'll enable this by default in 0.28.0. So we'll have a follow-up PR that changes the default value (either now, or closer to the release). This is to avoid a large behavior change in a patch release.

Otherwise, if Fredrik is happy with this, then from my side this just needs a CHANGELOG entry.

[fredrikekre]

I pushed those changes and added docs + changelog.