optex as pdf-engine in pandoc

[paessens]

Hi,
how about including OpTeX as a PDF-engine in Pandoc (https://pandoc.org/)?

my workflow is usually from RStudio using rmarkdown/knitr to compile into pdf via pandoc and using lualatex as a PDF-engine (https://bookdown.org/yihui/rmarkdown/pdf-document.html#latex_engine or see https://pandoc.org/MANUAL.html#option--pdf-engine).

This chain is a lot slower than just using pdftex as engine, but missing some of the later innovations in luatex. But sometimes kind of a clean-slate could bring speed into the process since less old APIs are deprecated and can be shut out.

cheers
Heinrich

[olsak]

That sounds very interesting. But have no deep knowledges about pandoc. If somebody else (than me) is able to include OpTeX as PDF engine in Pandoc, then I am ready to co-oporate.

[robertbachmann]

Pandoc has support for adding custom output formats via Lua.

I built a mostly working prototype last year. It still has
some bugs and the documentation is unfinished.

I've been meaning to prepare are more polished version for public
release, ... however my day job became more stressful and I currently
don't have the time and energy to do so.

Hopefully I can get back to it in Q2 or Q3 of 2022.

[olsak]

It would be great. We'll wait until Q2 or Q3.

[JHonaker]

Any update on the Pandoc converter @robertbachmann ?

[robertbachmann]

Pandoc added a new Lua API.
I took me more time than anticipated to migrate my code from the old to the new API.

I still need to fix a couple of things and write a shot user manual. I'm hoping to release end of march.

Note: table support is a bit more complicated and will be added in April/May.

[robertbachmann]

I've made some significant progress, however I still need to finish a couple of things.

I managed to integrate pandoc's syntax highlighting functionality. So the user can choose whether he wants to highlight a listing with hisyntax or with pandoc's highlighter. (The latter supports more programming languages).

I also managed to integrate with pandoc's citeproc. So the user can choose between using bibtex via OpTeX or let pandoc handle citations. (The latter offers support for more styles).

Process the citations in the file, replacing them with rendered citations and adding a bibliography. Citation processing will not take place unless bibliographic data is supplied, either through an external file specified using the --bibliography option or the bibliography field in metadata, or via a references section in metadata containing a list of citations in CSL YAML format with Markdown formatting. The style is controlled by a CSL stylesheet specified using the --csl option or the csl field in metadata. (If no stylesheet is specified, the chicago-author-date style will be used by default.)

I'm still finishing up the last feature and I'm about 40% done with the documentation. So I'm hoping to release within the next weeks.

[robertbachmann]

Unfortunatly the project was delayed.
In May I was in the middle of rewritting/polishing the code BUT then we had an unexpected large-scale re-enginierung effort at work that keept use busy for a couple of months.

I've picked up working on the OpTeX writer again in autumn, but progress is mostly slow.

It deepends on how many "context switches" I need to do during the work day.
On some days I just work on my assigned task, then it's very easy (and fun) to do 1-2 hours of work on the OpTeX writer.
On other days I need to adivse and/or assist collegeas, then I'm less focused in the evening.

Currently I'm unable to predict a release date.

Things I've done in the last couple of months: See my comment at robertbachmann/pandoc-OpTeX#1 (comment)

[robertbachmann]

panoptexum

Petr posted:

Note that my student Jan Martínek created pandoc module which converts from pandoc documents to OpTeX. See
https://github.com/spasa47/panoptexum. The OpTeX reader isn't implemented yet.

This program is an alternative that covers most of the core markdown features.

[robertbachmann]

I had a quick look at panoptexum today and I think it should work fine for users that are somewhat familiar with OpTeX/TeX.

• core markdown features are covered.
• for other things you can directly put OpTeX markdown as a "raw block"

# Some heading

````{=optex}
{\Red Danger:} Do not use \code{fdisk} without a backup.
````

• in some cases you might need to manually correct the generated OpTeX markup.

[robertbachmann]

I did a quick code review and noticed two things:
1.

Writer.Inline.Str = function(str)
  return str.text -- str.text should be escaped here (e.g: % -> \%)
end

Writer.Block.RawBlock = function(rawBlock)
  -- should check if rawBlock.format == 'optex' or rawBlock.format == 'tex'
  return { rawBlock.text }
end

[robertbachmann]

Basic math support could be added like this:

-- Note: pseudo code
writer.Math = function(elem)
	local s = elem.text
	if elem.mathtype == 'InlineMath' then
		return '$' .. s .. '$' 
	else
		return '$$\n'  .. s .. '$$\n' 
	end
end

[robertbachmann]

final note: Some form of automated regression testing would be beneficial; I've put some notes here: spasa47/panoptexum#4

All other things that come to mind would be rather time consuming...