use mdbook for docs

[krmanik]

Update documentation for AnkiDroid with mdbook

Fixes #97

Implemented by taking help from anki-manual and mdbook-binaries

Images

[abdnh]

Great work!

But one issue I see not tackled here is translations. Will the old manuals continue to be built and served?

I maintain the Arabic translations (of both ankitects/anki-manual and AnkiDroid docs), and I'm willing to port the translation to mdBook too. I've already done that for the computer manual, using basically the same process. The Ankitects doc translations are not committed to the official repo, so they don't have to deal with translations. One option is to use do the same here and require translators to maintain forks or separate repos. Though this makes the process less accessible for less technical translators.

Anyway, if we continue to maintain the translations on the same repo, I can help with some build steps specific to Arabic. I actually had to maintain a fork of mdBook itself to support Arabic search, because mdBook doesn't support any language other than English by default. If this complicates things a lot, maybe I'll just move the translation to a fork.

[krmanik]

Hi, The multiple language seems to be not implemented in the mdbook. So, I have created dir for each language as individual mdbook src dir. I have split the docs into respective languages and topic/category. Now implementing for build process so that will be easier to deploy.
View the changes here https://github.com/krmanik/ankidroiddocs

This PR will be closed

[abdnh]

Thank you! I'll test that version out.

[mikehardy]

Hey @abdnh ! Thanks for helping with translations, it is no small or easy task. Just as a statement of preference, I would like - if it is possible - for translations to be in one spot, it seems like maintaining forks and such would be difficult, because then you are maintaining build processes and infrastructure as well as language translation.

I don't have anything specific to add, just that "all in one spot" is my preference. If we need to maintain a local fork of mdbook that is something we could look in to

Related question: have you tried posting your changes to mdbook as a PR so that they support non-english in their main line?

[bnknndy]

Glad I checked this pr before I continued trying to figure out how to set up the ASCII editor. Looking forward to the swap-out

[abdnh]

More context on multilingual search support in mdBook:

mdBook uses elasticlunr-rs for search support, but doesn't take advantage of its support for non-English languages. I had to port an Arabic stemmer implementation from lunr-languages to elasticlunr-rs. I contributed my changes back to elasticlunr-rs, and they were merged recently (See mattico/elasticlunr-rs#40). mdBook was just recently updated to use the new version of elasticlunr-rs that includes my changes. What remains is changing mdBook to make use of the other stemmers provided by elasticlunr-rs. My fork of mdBook is unsuitable in its current state for merging into main. I have to think about the best way to add optional support for multilingual search to mdBook then suggest the changes. We can then take advantage of search support in all or most translations here.

Until I or someone else does that, we'll have to live with broken search in the translations (maybe not a big deal, I doubt most users touch the docs).

[abdnh]

@krmanik it seems Markdown links with titles in the translations are broken (in your main branch). Check out this for example: https://github.com/krmanik/ankidroiddocs/blob/main/mdbook/ja/src/importing-anki-files.md

[krmanik]

Hi, I have just updated for Chinese language. (English and Chinese is complete with formatting). The other two Arabic and Japanese need to modify. If you can fork it push the changes for Arabic then it will be helpful. View the change for Chinese here krmanik@00ecbbc, similar has to be done for other two language. I will start for Japanese later.

[abdnh]

Will update Arabic and send a PR.

[abdnh]

@krmanik how did you convert the AsciiDoc files to Markdown, by the way?

[krmanik]

Automated process

sudo apt install pandoc asciidoc

asciidoc -b docbook manual.asc

pandoc -f docbook -t markdown_strict manual.xml -o manual.md

Then copy paste to respective category.

I have used manual process.

Split the .asc file into respective category

[[ header 1 ]]
...
...
[[ header 2 ]]
...
...

The content between [[ header 1 ]] and [[ header 2 ]] cut/paste into header-1.md file

In header-1.md file

1. Removed [[ header 1 ]]

2. Changed header text to md

Ascii	md
==	#
===	##
====	###

3. Change :: with new line and ####

• Ascii

Some text :: Some other text

• md

#### Some text
Some other text

4. Changed link

• Ascii
  link:https://example.com[wiki]
• md
  [wiki](https://example.com)
  Also updated for anki docs with latest link, English md files can be used for references.

Also if getting error The web page exists, but not the anchor then change it to

<a href="https://example.com">example</a>

5. Changes link to internal docs pages

• Ascii
  <deckPicker, the deck list>
• md
  [the deck list](deck-picker.md)
  in md the # can be used to link subsection

6. Change list with numbers

• Ascii

.
.
.

• md

1.
2.
3.

More info here https://docs.asciidoctor.org/asciidoc/latest/asciidoc-vs-markdown
Also view this commit for similar changes krmanik@00ecbbc

[krmanik]

@abdnh Just finished for Japanese. If you have not started yet then I will update it for Arabic also.

[abdnh]

If you have not started yet then I will update it for Arabic also.

I've started with Arabic. I think I will just proofread the pages and fix errors manually and use some Find and Replace. It seems internal links and some headings in the Arabic translation were not converted. Some pages also have their original English text, even though they are actually translated.

[krmanik]

If you have not started yet then I will update it for Arabic also.

I've started with Arabic. I think I will just proofread the pages and fix errors manually and use some Find and Replace. It seems internal links and some headings in the Arabic translation were not converted. Some pages also have their original English text, even though they are actually translated.

Thanks

[abdnh]

@krmanik I've sent a PR: krmanik#4

[krmanik]

Thanks, I will merge it now.

[abdnh]

@krmanik Sent a PR to fix errors in the Arabic manual: krmanik#6

Do you think it's time to open a new PR for the other branch?

[krmanik]

Hi, sorry, I didn't get the notification for this. I have merged PR and will create PR here for the new docs we have created.
Thanks

[abdnh]

Hi, sorry, I didn't get the notification for this.

Yes, apparently GitHub doesn't send notifications for PRs in forks by default. One has to manually select "All Activity" for that.

[NameLessGO]

Unfortunately, there wasn't any changes to rust-lang/mdBook#5

However, mdbooks permits plugins, and found a translation plugin which would be great in Anki and Ankimobile docs as well

from https://google.github.io/comprehensive-rust/

https://github.com/google/mdbook-i18n-helpers/

[abdnh]

However, mdbooks permits plugins, and found a translation plugin which would be great in Anki and Ankimobile docs as well

I was planning to look into mdbook-i18n-helpers and suggest using it in the desktop manual. It looks promising.

[dae]

I found an example of how it splits text up:

https://github.com/google/comprehensive-rust/blob/main/po/fr.po

The current approach is translators fork the desktop repo, and then modify the files one by one.

Pros:

• It's easy to see each sentence in context.
• Translators have more freedom to e.g. combine two sentences into one, or vice versa.
• Translators can work with any plain text editor, and don't need special software
• As the translated manuals are hosted by the primary translator, they have full control, and no work/interaction is required by Anki devs
• No extra CI/automation is required
• The manual is a large body of text, and putting it into the translation system will bias the % complete figures towards zero, which may be slightly demotivating.

Cons:

• The Git workflow is difficult for less technically-inclined users
• Harder to collaborate / contribute small amounts
• User needs to follow git history to keep track of changes to the base manual

I don't have strong feelings either way, but am a bit short on time at the moment, so if you've explored using the helpers and would like to go down the helper path, it may be a little while until I have a chance to make the required CI/workflow changes.

[krmanik]

I am updating ankidroiddocs using mdbook-i18n-helpers along with integrating workflow of crowdin (krmanik/ankidroiddocs/mdbook-i18n)

Anki uses https://mozilla-pontoon.readthedocs.io/en/latest/ which supports po files. In case of Ankidroid, we can use crowdin which also support po, https://store.crowdin.com/gnu-gettext, so user have to just edit po files.

The i18n files from AnkiDroid uploaded to crowdin, then translated files are downloaded to respective directory. Similar can be done for ankidroiddocs also.

Also, manual bundled with AnkiDroid, so the manual can be exported as epub which is archive file of html and css or other format, then rendered in AnkiDroid.

[abdnh]

• It's easy to see each sentence in context.

• Translators have more freedom to e.g. combine two sentences into one, or vice versa.

These for me are the biggest advantages of the current approach.

As the translated manuals are hosted by the primary translator, they have full control, and no work/interaction is required by Anki devs

I used this power to make search work in the Arabic manual. It was a lot of work (I had to fork mdbook) but I think was worth it. Search is a big part of what makes the manual useful for me. mdbook-i18n-helpers doesn't provide a solution for this.

I also don't like the fact that existing translations need to be manually converted to the new system. It's basically doing the whole work of translation and proofreading. But maybe that's a good price to pay for advantages of the new approach, which are quite attractive. The new approach is far more accessible to casual contributors, especially with integration into Crowdin/Pontoon as @krmanik has done in #119.