WIP: Hal 2017

[fisx]

See jgm/pandoc#1852. This is work in the context of http://nfa.imn.htwk-leipzig.de/HAL2017/.

Make most types polymoprhic in the string type, and use the convertible-strings package to use lazy text internally. This is a non-breaking change; the new type is Inline' string, and there is an alias Inline = Inline' String.

The polymorphism can be rolled back later if everybody uses the new pandoc string type, but it's possible that there is no performance penalty as ghc can compile all the id conversions away.

[fisx]

=================== before

benchmarking simple walk
time                 149.0 μs   (148.7 μs .. 149.2 μs)
                     1.000 R²   (1.000 R² .. 1.000 R²)
mean                 149.1 μs   (148.9 μs .. 149.4 μs)
std dev              864.2 ns   (636.7 ns .. 1.146 μs)
             
benchmarking walk concatMap
time                 170.6 μs   (170.3 μs .. 170.9 μs)
                     1.000 R²   (1.000 R² .. 1.000 R²)
mean                 170.5 μs   (170.2 μs .. 170.9 μs)
std dev              1.055 μs   (744.3 ns .. 1.604 μs)
             
benchmarking walk lists
time                 151.4 μs   (151.2 μs .. 151.7 μs)
                     1.000 R²   (1.000 R² .. 1.000 R²)
mean                 151.3 μs   (151.1 μs .. 151.5 μs)
std dev              693.4 ns   (569.6 ns .. 894.7 ns)


=================== after with String

benchmarking simple walk
time                 370.0 μs   (368.9 μs .. 371.1 μs)
                     1.000 R²   (1.000 R² .. 1.000 R²)
mean                 370.6 μs   (369.5 μs .. 372.2 μs)
std dev              4.620 μs   (3.108 μs .. 6.644 μs)

benchmarking walk concatMap
time                 348.5 μs   (347.5 μs .. 349.6 μs)
                     1.000 R²   (1.000 R² .. 1.000 R²)
mean                 348.6 μs   (347.7 μs .. 349.8 μs)
std dev              3.538 μs   (2.150 μs .. 5.863 μs)

benchmarking walk lists
time                 326.5 μs   (325.2 μs .. 327.7 μs)
                     1.000 R²   (1.000 R² .. 1.000 R²)
mean                 325.8 μs   (324.8 μs .. 327.0 μs)
std dev              3.781 μs   (2.761 μs .. 5.282 μs)


=================== after with ST

benchmarking simple walk
time                 375.1 μs   (374.0 μs .. 376.3 μs)
                     1.000 R²   (1.000 R² .. 1.000 R²)
mean                 375.2 μs   (374.5 μs .. 376.3 μs)
std dev              3.028 μs   (2.059 μs .. 4.557 μs)

benchmarking walk concatMap
time                 354.2 μs   (352.8 μs .. 355.5 μs)
                     1.000 R²   (1.000 R² .. 1.000 R²)
mean                 353.4 μs   (352.5 μs .. 354.6 μs)
std dev              3.321 μs   (2.405 μs .. 4.598 μs)

benchmarking walk lists
time                 325.3 μs   (324.1 μs .. 326.5 μs)
                     1.000 R²   (1.000 R² .. 1.000 R²)
mean                 324.6 μs   (323.9 μs .. 325.6 μs)
std dev              2.802 μs   (1.975 μs .. 4.197 μs)


=================== after with LT

benchmarking simple walk
time                 382.7 μs   (381.4 μs .. 384.2 μs)
                     1.000 R²   (1.000 R² .. 1.000 R²)
mean                 382.3 μs   (381.2 μs .. 383.9 μs)
std dev              4.561 μs   (3.489 μs .. 6.354 μs)

benchmarking walk concatMap
time                 352.4 μs   (351.5 μs .. 353.5 μs)
                     1.000 R²   (1.000 R² .. 1.000 R²)
mean                 352.5 μs   (351.8 μs .. 353.5 μs)
std dev              2.644 μs   (1.762 μs .. 4.275 μs)

benchmarking walk lists
time                 345.5 μs   (344.1 μs .. 346.9 μs)
                     1.000 R²   (1.000 R² .. 1.000 R²)
mean                 345.7 μs   (344.7 μs .. 347.4 μs)
std dev              4.301 μs   (2.852 μs .. 6.397 μs)

[tarleb]

The general idea here is to find a not-so-painful path to make pandoc use Text instead of String everywhere. There should be a conversion function

convertPandoc :: Pandoc String -> Pandoc Text

which could be used in Text.Pandoc.App, enabling conversion of modules one-by-one. Switching to Text would no longer be one huge, herculean task, but consist of many managable and beginner-friendly steps.

This will also allow us to experiment with and benchmark lazy vs strict text.

[jgm]

Obsoleted by switch to Text.