Support for multilingual content authoring

[gibujin]

The good people at Netlify has confirmed to me that multilingual content is not really supported. But that doesn't necessarily mean that its not possible. For example having a post with a title field * languages supported doesn't really seem like a good idea. Another approach could be to create multiple admin interfaces (/admin/en, /admin/es etc.)for each language (if thats possible, haven't tested it). How would you approach this, any other suggestions?

[erquhart]

If you're looking for multiple languages in the admin interface, there's a PR here that's close: #403

[kations]

@erquhart I think @gibujin means multilingual content, that would be a nice feature. Maybe there is a way to define languages in config.yml like ['us', 'fr', 'de'] and than you have a global language select in the admin ui to create content for a specific lang.

[erquhart]

In that case I'd expect it to be a bit of both, as selecting a global language setting should cause the UI to be labelled in that language. It sounds like you're looking to have the same content in multiple languages, so I imagine you'd want some way to see what entries exist that have no translation in the current language? Beyond that each language should publish to a separate directory, and we'd need certain string properties in the config to start accepting an object of language-keyed strings.

Thoughts?

[tech4him1]

@erquhart Is this a duplicate of #342?

[erquhart]

I think @fool was more pointing at handling automation of translation. This is about supporting the manual creation of multilingual content.

[fool]

Not even "automation" so much as "easy support for". But yes not in the admin gui, in the content. E.g. Being able to take a template from an English post with graphics and code blocks and turning it Spanish. One click "clone" with ability to identify articles that are missing translations would be most of it. Configuring your site to serve the different content would be out of scope for the cms by my reckoning On Dec 8, 2017 23:56, "Shawn Erquhart" <[email protected]> wrote: I think @fool <https://github.com/fool> was more pointing at handling automation of translation. This is about supporting the manual creation of multilingual content. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#716 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAN-oBGRQfIQC0kvYnAkfF8_tI9xkU3Dks5s-j05gaJpZM4P_IL-> .

[blgo]

@fool
I agree, the clone functionality would be ideal ! It will allow to match the original filename with the translated version.

In addition, It should allow to add a language suffix to the filename: i.e. "mypost.es.md".

However, I realised that I cannot create this suffix ".es" using slug for naming the file because the code remove dots from slug.

Any ideas?

EDIT: I am not requesting a major change, but some configuration that would allow to easily create miltilingual content

[erquhart]

@gibujin you can already create multiple cms instances as you described, since the CMS is just a single page app.

@blgo @fool can you point to any examples out there of a CMS with the described clone functionality?

[tech4him1]

@blgo The issue with extension suffixes is here: #877

[fool]

haven't used one but this link makes it seem like maybe drupal and joomla both did it in 2015: https://www.acclaro.com/blog/how-to-choose-cms-for-multilingual-website/

…

On Dec 20, 2017 10:15, "Shawn Erquhart" ***@***.***> wrote: @gibujin <https://github.com/gibujin> you can already create multiple cms instances as you described, since the CMS is just a single page app. @blgo <https://github.com/blgo> @fool <https://github.com/fool> can you point to any examples out there of a CMS with the described clone functionality? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#716 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAN-oB2n46-evcyNEO5HSw35Rzkt_YQIks5tCU7AgaJpZM4P_IL-> .

[erquhart]

So it sounds like we need product level contributions at this point. So far we have:

• clone functionality for existing entries that need to be copied and translated
• some way of seeing which articles have/have not been translated into a given language
• ability to configure file names based on language

Anything missing? UX considerations? How do we make this unobtrusive?

[m4rrc0]

just dropping my 2 cents. Contentful has a very neat UX for content translation / 'locales'. You may find it inspiring to have a look.

[erquhart]

Great tip @MarcCoet, will do! Feel free to drop specifcs, screenshots, etc if you think any aspect is particularly impactful for the project.

[andresgutgon]

@erquhart do you have a plan in terms of timings to develop multilingual?

[erquhart]

There's no active development at the moment, still need to determine the right approach. You're welcome to help move things along if you're interested in digging in.

[whmountains]

I'm a freelancer in South America 🇨🇱 and almost all my clients need a Spanish site for local clients and an English translation to reach foreign tourists. So having official support for multiple languages would be really useful!

I agree with @MarcCoet that Contentful has a really convenient UI for translations. Basically there are some checkboxes in the sidebar that let you select which languages to work on, and then it will show a copy of each field for each language. Here's a screenshot to show what I mean:

IMO an anti-pattern for localization is to simply offer a language switcher with no ability to view multiple languages at once. You have to open multiple windows to see the original text and new text and it's annoying, although still much better than nothing.

[erquhart]

Awesome, thanks for the specifics @whmountains.

@hcavalieri this ^^ just pointed out an issue with the tabbed i18n approach from #1166 - folks are going to want to look at multiple translations at once, especially while translating.

[whmountains]

folks are going to want to look at multiple translations at once, especially while translating.

Exactly. I'm glad you inferred my use-case even though I forgot to state it explicitly. 😄

[barthc]

So we have a PR #2988 that attempts to solve this issue. The approach is based on comments #716 (comment) and #716 (comment) . Any feedback is highly appreciated, thanks.

[bencao]

To provide a different perspective, my current approach for the issue is to setup separate netlify sites (and cms admin) for different locales.

the repo directory structure looks like this:

.
├── app
│   ├── gatsby-browser.js
│   ├── gatsby-config.js
│   ├── gatsby-node.js
│   ├── gatsby-ssr.js
│   ├── package.json
│   ├── src
│   ├── static
│   └── yarn.lock
├── cn
│   ├── assets
│   ├── cms-config.yml
│   ├── data
│   └── netlify.toml
└── en
    ├── assets
    ├── cms-config.yml
    ├── data
    └── netlify.toml

for the different locales, I created multiple netlify sites, each pointing its base directory to the locale it targets, such as en.

netlify.toml file in the locale folder contains instructions for deployment, which copies data and assets into app folder on deployment, so app can assume those are available:

[build]
  ignore = "git diff --quiet HEAD^ HEAD ../app/ ./"
  publish = "app/public/"
  command = "cp -r ../app . && cp -r data ./app/ && cp -r assets ./app/static/ && cp cms-config.yml ./app/static/admin/config.yml && cd ./app && npm install -g yarn && yarn install && yarn build"

cms-config.yml will be copied as app/static/admin/config.yml:

backend:
  name: git-gateway
  branch: master

public_folder: assets
media_folder: en/assets            # media folder should point to the assets directory in given locale folder

collections:
  - label: "Pages"
    name: "pages"
    files:
      - file: "en/data/header.json"        # should also point to data directory in given locale
        label: "Navbar"
        name: "navbar"
        delete: false
        fields:
          - {
              label: "Logo",
              name: "logo",
              widget: "object",
              fields:
                [
                  { label: "Desktop Logo", name: "desktop", widget: "image" },
                  { label: "Mobile Logo", name: "mobile", widget: "image" },
                ],
            }

and lastly, the development support is implemented as a script in package.json:

"scripts": {
    "bootstrap": "cp ../en/cms-config.yml static/admin/config.yml && rm -rf static/assets && cp -r ../en/assets static/ && rm -rf data && cp -r ../en/data .",
    "bootstrap-cn": "cp ../cn/cms-config.yml static/admin/config.yml && rm -rf static/assets && cp -r ../cn/assets static/ && rm -rf data && cp -r ../cn/data .",
    ...
  },

hope the above solution offer help to someone facing the similar situation.

[joallard]

So I can see from #2988 that we're still looking for UI/UX solutions on this. Thanks @barthc for the initiative by the way!

To explain my background, I'm a francophone who's been scouring the English internets for a long time; translating Wordpress posts has been my job a long time ago; I've worked on Middleman's I18n features; and am a total UI-UX geek. So I've been thinking about localization interfaces for a long time.

Here's what I'd like to see in an L10n interface. Explanations after the image.

---

• Side by side view: This is a life saver for any kind of localization. I have on my left the content already populated in one language, and on the other, the one to fill in. I can quickly look side to side what to translate.

• Quick language switcher: I can select what is the language I see on each side. Selecting a language will not make me lose my changes, all data is in local memory. This allows me to translate between more than 2 languages, with the one I want for reference. If I'm not sure of what to write in a third language, I can consult both the first and second.

  (DatoCMS does this kinda well, but no side-by-side last I checked)

• Save button: The Save button saves all languages. (Linkedin's interface is really clunky about this) It should be part of the contract that switching a language does not throw your work away!

As a matter of implementation, those are enough for me and a huge leap forward. This could be PR One.

Now, the nice to haves:

• Dirty indicator: An indicator tells me about what I don't see. It could either be: (1) I've made changes to fields in language A but not in language B, so indicator tells me maybe I should; or (2) simply that this language contains changes. (A tooltip would probably be helpful here)(See Linkedin Profile About Us localization, which is an interesting though imperfect example.)

• Synced scroll on rich text: If you're translating content that spans more than the textarea height, you'll run into an issue quickly: what you're seeing on the left isn't scrolled the same as what you're seeing on the right. As a poor man solution, you could scroll manually. Ideally, an algorithm would guess what is the semantic position of paragraphs on both sides and synchronize that.

  Sidenote:
  This doesn't seem trivial, and the use cases are probably different on text creation vs text edition. I would neither assume that both versions contain the same amount of paragraphs. - I do wonder whether translation software has a solution/algorithms for this. A poor man's solution I've used in the past with Vimdiff was to label places with Markdown/HTML comments on both sides to force it to sync. (eg. <!-- 5 -->)

• I could also conceivably just collapse language B in the case I don't need it. I wonder about the usefulness of this feature.

[joallard]

Now on to the index view and filtering:

(Keep in mind, I don't really know what the index view looks like, sorry about that)

---

---

Index/Collection view

On the index view, I want to get a gist of which version is present or needs work ("done-ness"). A faded bubble could indicate absence, a warning color could indicate need to update.

This suggests two things:

• The indicator talked about previously would probably be useful to indicate "need to update"
• Changes in one language don't necessarily mean Need to update in all others. The UI could conceivably ask that question when a change is saved. (You just corrected something, do other languages need to be corrected too? Or did you correct a typo? - this, but unobstrusively, maybe a checkbox next to the Save button)

Also, this could probably support max 6-8 languages/bubbles in sight, optionally on two rows.

You can also see one Nice to have feature is to display content in some language.

Filtering

If I'm a translator, I probably need to know which records need work on.

• Show me records where es, fr are empty
• Show me records where es, fr are either empty or need update

If I'm an editor, I probably need to know what's there and that I can publish

• Show me records where all languages are up to date
• Show me records where en, fr are up to date

Those are the filtering use cases I can come up with for now. I'm mainly thinking about Airtable's filtering interface as a matter of inspiration.

To divide it into PRs:

1. Side-by-side form I18n and data model
2. Dumb-ish sync'ed scroll
3. Filtering
4. Dirty indicator
5. Collection doneness bubbles
6. Good-ish sync'ed scroll
7. Collection: Display localized content

[erquhart]

@joallard I could have sworn I responded to your comments long ago, sorry about that.

I'm really loving the approach you've laid out. It brings the best of what we've discussed so far, rounds out the editor experience with sensible functionality, and matches really well to our current editor model (two big panes side by side).

This issue is very close to hitting active development, I'm in strong agreement with the order of implementation and prioritization you proposed as well. Thank you!

[joallard]

@erquhart I'm really glad it's appreciated! I've been wanting to see a good I18n interface in a CMS for a long time. I'm open to be reached if there's any question or if I can help.

Since then I've taken a look at what Ghost does for blog articles (turns out their "multi-language" feature is not really one after all) and more precisely that fact that textareas don't scroll, they just expand/fill the page. That could be a simple way to work around the scroll problem: no scroll! The box just resizes according to its content. That would more or less do it for a variety of cases. We just have to ensure the next cells in the columns are at the same height, and off we go.

[erezrokah]

Update:
@barthc and @jmdrawneek both offered to help with this issue, which is highly appreciated.

Copying my comment from #2988 (comment):

I can see two main efforts here:

1. Having the backends support persisting multiple content files (e.g. saving both post.en.md and post.fr.md under the same entry). This should actually be configured - multiple files vs multiple fields in a single file, and defaults to the most common approach of static site generators.
2. Adding a side by side translation editor UI per #716 (comment).

Unless @barthc or @jmdrawneek have already made significant progress in implementing this feature, we might be able to split up the work.

As @barthc is more familiar with the backends code I'm thinking he can do 1, and as doing the UI part doesn't require that much pre-knowledge of the code base @jmdrawneek can do 2.

I might be completely off here as well, thoughts?

[inwardmovement]

As a side note, it would be great to have flexibility as to how translations are handled: by file name (post.en.md / post.fr.md) or by folder (en/post.md / fr/post.md), as Hugo for example allows both.

[joallard]

As you folks are implementing this, feel free to reach out to me on Keybase or by email if you need a sounding board on this! I'm really glad to see the enthusiasm!

[barthc]

Another PR #3366 based on @joallard solution.

[chriskirknielsen]

Another PR #3366 based on @joallard solution.

@barthc Will the diff_folder allow for a fully customisable path? Like {locale}/posts/post.md? It's how I have my current setup, and while I could certainly rework it, it would be amazing to be able to keep it as is while using this new feature. :)

[barthc]

@barthc Will the diff_folder allow for a fully customizable path? Like {locale}/posts/post.md? It's how I have my current setup, and while I could certainly rework it, it would be amazing to be able to keep it as is while using this new feature. :)

The collection path config will allow you to do that. But the path config option for multi-content is currently disabled in the PR.

[tbille]

hey after showing my example on discourse, I got suggested to come and link my project here.

I just want to share the way I implemented to add multilingual support on my site.
As mentioned in the topic there is still some manual process that can be automated (Github Actions or similar?) and some parts might need polishing but the idea is there.

Project: https://github.com/tbille/multilingual-jekyll-netlify/

Just to quote. This is my source of inspiration to make jekyll multilingual. I tweaked a bit the solution to make it work with netlify-cms

[joallard]

Excellent link @tbille, Durand is spot on about the goals.

I can't help but highlight one of them, that was discussed in the PR:

Flexibility

Our website can be translated in as many languages as wanted. Each page may be translated or not in the various languagesregardless of the version, they may be pages which are not translated in each language. All pages and posts can be placed as desired.

---

The only lowlight? I can never endorse plain wrong date formats such as 01/09/2014, or even 2014/09/01 ;). (Friendly reminder that YYYY-MM-DD is the only acceptable date format – except when the month has letters)