Haddock: Fix very confusing formatting errors #2622

DigitalBrains1 · 2023-12-12T15:55:34Z

Because of mistakes escaping special characters, some documentation showed the operators $ and * where the Functor and Applicative operators were intended. Since they are all very common operators, this can be very confusing to the reader.

Additionally, it was discovered that later versions of Haddock start interpreting Markdown-style links in code blocks, breaking the rendering of the primitive blackboxes in Clash.Tutorial. This problem does not occur with ~~GHC 9.0.2 (Haddock 2.25.1)~~ GHC 8.10.7 (Haddock 2.24.2), which we use to build the documentation for Hackage. But it does occur with at least GHC 9.6.

To make such things less problematic, we decided to use bird tracks (lines starting with > ) for code blocks that are not Haskell code, and only use the @...@ style of code block for Haskell code. Bird tracks don't use any formatting at all. All code blocks with something other than Haskell code have been converted to bird tracks. Additionally, some escaping issues in Haskell code blocks were fixed when they were noticed.

My intention is to upload new 1.8.1 documentation with this fix to Hackage. That's going to require a small amount of creativity, as I don't want to upload the changes of 4ac8481. But that should work out just fine.

[edit]
After this PR was merged, I discovered that I had somehow misinterpreted which version of GHC we use to build the documentation for Hackage. I thought it was 9.0.2, and that is what the commit message for this PR says. But we use an even older one, we build the documentation for Hackage with 8.10.7. This does not really change anything, though.
[/edit]

Still TODO:

~~Write a changelog entry (see changelog/README.md)~~
~~Check copyright notices are up to date in edited files~~ I didn't bother with this.

leonschoorl · 2023-12-12T17:11:58Z

Here is another one:

clash-compiler/clash-prelude/src/Clash/Explicit/Testbench.hs

Line 452 in 930641c

    
           --     done           = exposeClockResetEnable (expectedOutput (topEntity <$> testInput)) clk rst

leonschoorl · 2023-12-13T12:11:16Z

There are some more weird one:

[dist-newstyle/build/x86_64-linux/ghc-9.6.3/clash-prelude-1.9.0]$ rg 'href="[^"#]{0,9}"' .
./clash-prelude/Clash-Explicit-Testbench.html
62:    done           = exposeClockResetEnable (expectedOutput (topEntity <a href="$">$</a> testInput)) clk rst
                                                                             ^^^^^^^^
./clash-prelude/Clash-Tutorial.html
920:            ~SYM<a href="~SYM[5]">2</a> &lt;= ~TOBV[~ARG[9]][~TYP[9]];
                           ^^^^^^^^
922:          ~RESULT &lt;= fromSLV(~SYM<a href="~SYM[4]">2</a>)
                                                 ^^^^^^^^
933:            ~SYM<a href="~SYM[5]">2</a> &lt;= ~ARG[9];
                             ^^^^^^^^
935:          ~RESULT &lt;= ~SYM<a href="~SYM[4]">2</a>
                                         ^^^^^^^^

./clash-prelude/Clash-Prelude-Testbench.html
93:    done           = exposeClockResetEnable (expectedOutput (topEntity <a href="$">$</a> testInput)) clk rst
                                                                             ^^^^^^^^

./clash-prelude/Clash-Signal-Bundle.html
20:  bundle   (MkPair as bs) = MkPair <a href="$">$</a> as <a href="*">*</a> bs
                                                              ^^^^^^^^
23:  unbundle pairs = MkPair (getA <a href="$">$</a> pairs) (getB <a href="$">$</a> pairs)
                                                                     ^^^^^^^^

./clash-prelude/Clash-Sized-RTree.html
82:<a href="BLANKLINE">BLANKLINE</a>
            ^^^^^^^^

./clash-prelude/Clash-Explicit-Signal.html
383:  bundle   (MkPair as bs) = MkPair <a href="$">$</a> as <a href="*">*</a> bs
                                         ^^^^^^^^              ^^^^^^^
386:  unbundle pairs = MkPair (getA <a href="$">$</a> pairs) (getB <a href="$">$</a> pairs)
                                      ^^^^^^^^                        ^^^^^^^^

./clash-prelude/Clash-Signal.html
742:  bundle   (MkPair as bs) = MkPair <a href="$">$</a> as <a href="*">*</a> bs
                                         ^^^^^^^^              ^^^^^^^^

745:  unbundle pairs = MkPair (getA <a href="$">$</a> pairs) (getB <a href="$">$</a> pairs)
                                       ^^^^^^^^                       ^^^^^^^^

./clash-prelude/Clash-Prelude.html
402:  bundle   (MkPair as bs) = MkPair <a href="$">$</a> as <a href="*">*</a> bs
                                          ^^^^^^^^             ^^^^^^^^
405:  unbundle pairs = MkPair (getA <a href="$">$</a> pairs) (getB <a href="$">$</a> pairs)
                                       ^^^^^^^^                       ^^^^^^^^

./clash-prelude/Clash-HaskellPrelude.html
143:   <code>x = fromEnum n' - fromEnum n</code>, <code>c x = bool (&gt;=) (<a href="=)">(x</a> 0)</code>
                                                                               ^^^^^^^^

DigitalBrains1 · 2023-12-13T16:48:13Z

./clash-prelude/Clash-HaskellPrelude.html
143:   <code>x = fromEnum n' - fromEnum n</code>, <code>c x = bool (&gt;=) (<a href="=)">(x</a> 0)</code>
                                                                               ^^^^^^^^

That one is upstream. It renders the part <=) (x > in that line as a link.

DigitalBrains1 · 2023-12-13T17:13:48Z

./clash-prelude/Clash-Tutorial.html
920:            ~SYM<a href="~SYM[5]">2</a> &lt;= ~TOBV[~ARG[9]][~TYP[9]];
                           ^^^^^^^^
922:          ~RESULT &lt;= fromSLV(~SYM<a href="~SYM[4]">2</a>)
                                                 ^^^^^^^^
933:            ~SYM<a href="~SYM[5]">2</a> &lt;= ~ARG[9];
                             ^^^^^^^^
935:          ~RESULT &lt;= ~SYM<a href="~SYM[4]">2</a>
                                         ^^^^^^^^

[edit 2]
Corresponding source code:

clash-compiler/clash-prelude/src/Clash/Tutorial.hs

Lines 1190 to 1192 in 930641c

    
                       ~SYM[2](~SYM[5]) <= ~TOBV[~ARG[9]][~TYP[9]]; 
        
                     end if; 
        
                     ~RESULT <= fromSLV(~SYM[2](~SYM[4]))

[/edit 2]

Oh no. That one does not occur when building with GHC 9.0.2, which we use for Hackage doc uploads. It does occur on GHC 9.6.3. I can fix it, that's not the problem. The problem is this: I've often said "the only way to know that your Haddock comes out right is to look at the result carefully and click all links". Apparently that's not enough. Now, when we upgrade to a new Haddock version, we're apparently supposed to carefully reread our entire generated documentation to see if Haddock hasn't changed its interpretation. Undoable.

~~I haven't checked Haddock's changelog to see if this is mentioned somewhere, but I'm not holding much hope.~~ The changelog does not mention this change in interpretation.

[edit]
Note that Haddock's documentation on special characters puts you on the wrong track here. The characters used in this problematic bit aren't listed. You're supposed to imply from the documentation on [Markdown style](links) that those characters can also be special.

We should probably not over-quote those Markdown-style links to prevent it looking like LaTeX. Currently, it's rendered correctly, but they might (by accident?) start rendering it differently in a future version.
[/edit]

leonschoorl

I haven't run it through haddock, but all the edits look good

leonschoorl · 2023-12-18T14:40:40Z

clash-prelude/src/Clash/Tutorial.hs

@@ -1187,9 +1187,9 @@ BlackBox:
      begin
        if ~IF~ACTIVEEDGE[Rising][0]~THENrising_edge~ELSEfalling_edge~FI(~ARG[3]) then
          if ~ARG[7] ~IF ~ISACTIVEENABLE[4] ~THEN and ~ARG[4] ~ELSE ~FI then
-            ~SYM[2](~SYM[5]) <= ~TOBV[~ARG[9]][~TYP[9]];
+            ~SYM\[2](~SYM[5]) <= ~TOBV[~ARG[9]][~TYP[9]];


Instead of these very precise escapes in problem areas, have considered changing to the bird track style code blocks?

Ie change the whole block from:

@ line 1 ... line n @

to:

> line 1 > ... > line n

As I understand it those don't have any parsing for formatting/special characters.

You can't do that for some of the code blocks where you want to have embedded links for functions used. But when you don't need that, like in these blackboxes, it'll be more robust against future edits.

I 'unno. It's a large diff, it's busy work, I find the @...@ style more pleasant to look at, and it doesn't solve a fundamental issue. But if you feel strongly, I can change it.

BTW, I did check the whole source for more instances of this issue. And there aren't any more instances. Your grep correctly identified all of them.

it'll be more robust against future edits.

I feel somewhat defeated in this area. It feels like putting out a single twig in a forest fire. Haddock is just really hard to write and maintain.

I might write a vim syntax file to colour every special character (according to Haddock) in Haskell comments. That way I'll at least see directly that I need to watch out what I'm writing.

I feel somewhat defeated in this area. It feels like putting out a single twig in a forest fire. Haddock is just really hard to write and maintain.

:(

DigitalBrains1 · 2023-12-24T14:55:52Z

clash-prelude/src/Clash/Explicit/Signal.hs

-instance KnownDomain "System" where
-  type KnownConf "System" = 'DomainConfiguration "System" 10000 'Rising 'Asynchronous 'Defined 'ActiveHigh
+instance KnownDomain 'System' where
+  type KnownConf 'System' = 'DomainConfiguration 'System' 10000 'Rising 'Asynchronous 'Defined 'ActiveHigh


Is this what we want? It could either refer to the type synonym System or to the type-level string "System", I believe. They are the same thing. So it is either this or

instance KnownDomain \"System\" where type KnownConf \"System\" = 'DomainConfiguration \"System\" 10000 'Rising 'Asynchronous 'Defined 'ActiveHigh

Because of mistakes escaping special characters, some documentation showed the operators `$` and `*` where the Functor and Applicative operators were intended. Since they are all very common operators, this can be very confusing to the reader. Additionally, it was discovered that later versions of Haddock start interpreting Markdown-style links in code blocks, breaking the rendering of the primitive blackboxes in `Clash.Tutorial`. This problem does not occur with GHC 9.0.2 (Haddock 2.25.1), which we use to build the documentation for Hackage. But it does occur with at least GHC 9.6. To make such things less problematic, we decided to use bird tracks (lines starting with '> ') for code blocks that are not Haskell code, and only use the '@...@' style of code block for Haskell code. Bird tracks don't use any formatting at all. All code blocks with something other than Haskell code have been converted to bird tracks. Additionally, some escaping issues in Haskell code blocks were fixed when they were noticed.

martijnbastiaan

We should probably ban @ completely. But that's probably hard to do without doctest like GHC API use..

DigitalBrains1 · 2024-01-14T17:06:01Z

If you ban @ you can't link identifiers anymore in Haskell code blocks. That's baby and bathwater territory. My suggestion initially was "use bird tracks unless you wish to use formatting", but Christiaan quite decisively said "let's use @ ... @ blocks for Haskell code and bird tracks for other things".

Because of mistakes escaping special characters, some documentation showed the operators `$` and `*` where the Functor and Applicative operators were intended. Since they are all very common operators, this can be very confusing to the reader. Additionally, it was discovered that later versions of Haddock start interpreting Markdown-style links in code blocks, breaking the rendering of the primitive blackboxes in `Clash.Tutorial`. This problem does not occur with GHC 9.0.2 (Haddock 2.25.1), which we use to build the documentation for Hackage. But it does occur with at least GHC 9.6. To make such things less problematic, we decided to use bird tracks (lines starting with '> ') for code blocks that are not Haskell code, and only use the '@...@' style of code block for Haskell code. Bird tracks don't use any formatting at all. All code blocks with something other than Haskell code have been converted to bird tracks. Additionally, some escaping issues in Haskell code blocks were fixed when they were noticed. (cherry picked from commit cb401b8)

Because of mistakes escaping special characters, some documentation showed the operators `$` and `*` where the Functor and Applicative operators were intended. Since they are all very common operators, this can be very confusing to the reader. Additionally, it was discovered that later versions of Haddock start interpreting Markdown-style links in code blocks, breaking the rendering of the primitive blackboxes in `Clash.Tutorial`. This problem does not occur with GHC 9.0.2 (Haddock 2.25.1), which we use to build the documentation for Hackage. But it does occur with at least GHC 9.6. To make such things less problematic, we decided to use bird tracks (lines starting with '> ') for code blocks that are not Haskell code, and only use the '@...@' style of code block for Haskell code. Bird tracks don't use any formatting at all. All code blocks with something other than Haskell code have been converted to bird tracks. Additionally, some escaping issues in Haskell code blocks were fixed when they were noticed. (cherry picked from commit cb401b8) # Conflicts: # clash-ffi/src/Clash/FFI/VPI/Object/Value/Parse.hs # clash-lib/src/Clash/Netlist/Util.hs # clash-lib/src/Clash/Primitives/DSL.hs # clash-prelude/src/Clash/Annotations/TopEntity.hs # clash-prelude/src/Clash/Explicit/Testbench.hs # clash-prelude/src/Clash/Intel/ClockGen.hs # clash-prelude/src/Clash/Tutorial.hs # clash-prelude/src/Clash/Xilinx/ClockGen.hs

Because of mistakes escaping special characters, some documentation showed the operators `$` and `*` where the Functor and Applicative operators were intended. Since they are all very common operators, this can be very confusing to the reader. Additionally, it was discovered that later versions of Haddock start interpreting Markdown-style links in code blocks, breaking the rendering of the primitive blackboxes in `Clash.Tutorial`. This problem does not occur with GHC 9.0.2 (Haddock 2.25.1), which we use to build the documentation for Hackage. But it does occur with at least GHC 9.6. To make such things less problematic, we decided to use bird tracks (lines starting with '> ') for code blocks that are not Haskell code, and only use the '@...@' style of code block for Haskell code. Bird tracks don't use any formatting at all. All code blocks with something other than Haskell code have been converted to bird tracks. Additionally, some escaping issues in Haskell code blocks were fixed when they were noticed. (cherry picked from commit cb401b8) Co-authored-by: Peter Lebbing <[email protected]>

Because of mistakes escaping special characters, some documentation showed the operators `$` and `*` where the Functor and Applicative operators were intended. Since they are all very common operators, this can be very confusing to the reader. Additionally, it was discovered that later versions of Haddock start interpreting Markdown-style links in code blocks, breaking the rendering of the primitive blackboxes in `Clash.Tutorial`. This problem does not occur with GHC 9.0.2 (Haddock 2.25.1), which we use to build the documentation for Hackage. But it does occur with at least GHC 9.6. To make such things less problematic, we decided to use bird tracks (lines starting with '> ') for code blocks that are not Haskell code, and only use the '@...@' style of code block for Haskell code. Bird tracks don't use any formatting at all. All code blocks with something other than Haskell code have been converted to bird tracks. Additionally, some escaping issues in Haskell code blocks were fixed when they were noticed.

DigitalBrains1 added documentation needs-review backport 1.6 backport 1.8 labels Dec 12, 2023

DigitalBrains1 marked this pull request as draft December 13, 2023 16:56

DigitalBrains1 force-pushed the bundle-doc-fix branch from 0baf704 to 3fe1c2f Compare December 13, 2023 16:57

DigitalBrains1 force-pushed the bundle-doc-fix branch from 3fe1c2f to 386c639 Compare December 14, 2023 14:57

DigitalBrains1 changed the title ~~Bundle doc: Fix very confusing formatting error~~ Haddock: Fix very confusing formatting errors Dec 14, 2023

DigitalBrains1 marked this pull request as ready for review December 14, 2023 14:58

leonschoorl approved these changes Dec 18, 2023

View reviewed changes

DigitalBrains1 force-pushed the bundle-doc-fix branch from 386c639 to 0b5994b Compare December 22, 2023 15:34

DigitalBrains1 marked this pull request as draft December 22, 2023 15:35

DigitalBrains1 removed the needs-review label Dec 22, 2023

DigitalBrains1 commented Dec 24, 2023

View reviewed changes

DigitalBrains1 force-pushed the bundle-doc-fix branch from 0b5994b to 8c3b343 Compare January 9, 2024 16:40

DigitalBrains1 marked this pull request as ready for review January 9, 2024 16:40

DigitalBrains1 requested a review from leonschoorl January 9, 2024 16:40

martijnbastiaan approved these changes Jan 14, 2024

View reviewed changes

DigitalBrains1 merged commit cb401b8 into master Jan 14, 2024
13 checks passed

DigitalBrains1 deleted the bundle-doc-fix branch January 14, 2024 17:09

This was referenced Jan 14, 2024

Haddock: Fix very confusing formatting errors (copy #2622) #2640

Closed

Haddock: Fix very confusing formatting errors (copy #2622) #2641

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Haddock: Fix very confusing formatting errors #2622

Haddock: Fix very confusing formatting errors #2622

DigitalBrains1 commented Dec 12, 2023 •

edited

Loading

leonschoorl commented Dec 12, 2023

leonschoorl commented Dec 13, 2023 •

edited

Loading

DigitalBrains1 commented Dec 13, 2023

DigitalBrains1 commented Dec 13, 2023 •

edited

Loading

leonschoorl left a comment

leonschoorl Dec 18, 2023 •

edited

Loading

DigitalBrains1 Dec 18, 2023

DigitalBrains1 Dec 18, 2023 •

edited

Loading

martijnbastiaan Jan 14, 2024

DigitalBrains1 Dec 24, 2023

martijnbastiaan left a comment

DigitalBrains1 commented Jan 14, 2024

Haddock: Fix very confusing formatting errors #2622

Haddock: Fix very confusing formatting errors #2622

Conversation

DigitalBrains1 commented Dec 12, 2023 • edited Loading

Still TODO:

leonschoorl commented Dec 12, 2023

leonschoorl commented Dec 13, 2023 • edited Loading

DigitalBrains1 commented Dec 13, 2023

DigitalBrains1 commented Dec 13, 2023 • edited Loading

leonschoorl left a comment

Choose a reason for hiding this comment

leonschoorl Dec 18, 2023 • edited Loading

Choose a reason for hiding this comment

DigitalBrains1 Dec 18, 2023

Choose a reason for hiding this comment

DigitalBrains1 Dec 18, 2023 • edited Loading

Choose a reason for hiding this comment

martijnbastiaan Jan 14, 2024

Choose a reason for hiding this comment

DigitalBrains1 Dec 24, 2023

Choose a reason for hiding this comment

martijnbastiaan left a comment

Choose a reason for hiding this comment

DigitalBrains1 commented Jan 14, 2024

DigitalBrains1 commented Dec 12, 2023 •

edited

Loading

leonschoorl commented Dec 13, 2023 •

edited

Loading

DigitalBrains1 commented Dec 13, 2023 •

edited

Loading

leonschoorl Dec 18, 2023 •

edited

Loading

DigitalBrains1 Dec 18, 2023 •

edited

Loading