Translate taxonomy terms

[herbdool]

Describe your issue or idea

Like @quicksketch's pared down approach for menu items #225 perhaps we can approach taxonomy terms the same way.

This would mean terms have a language but we leave translation sets for contrib.

PR by @herbdool: backdrop/backdrop#2360

[bjoern-st]

@herbdool I think, this is a must have for multilingual sites. Any work in progress here?

[herbdool]

@bjoern-st I do not have any work in progress. Do you want to take a stab at this? I imagine it can be similar to the approach in #225 -- easier to implement and also easier for end users to understand. Anything more complex could go into contrib.

[bjoern-st]

@herbdool I'm not sure if I like the contrib idea. Taxonomy belongs to core so it should be translatable by default in core.
I'm just finishing my port of my profile_regpath module but after that I'll have a deeper look in the taxonomy module.

[herbdool]

@bjoern-st I think you misunderstand me. I'm not saying all taxonomy translation should be in contrib, just the more complicated ways of translating taxonomy. If you take a look at the issue linked above for menu translation @quicksketch took the approach of allowing for locale translation for menu items -- all menu items will be translatable. But he didn't implement "translation sets" which is more complicated and confusing to most people. My opinion is that the same approach is good for taxonomy.

[bjoern-st]

@herbdool Okay, I really didn't get that point.
By now I don't know if it is possible to us an approach similar to #225 from @quicksketch. As far as I understand this PR, he is using the node title from the underlying / translated node to create the translated menu item.
I don't see how this could work with taxonomies .. Do you have any ideas to that point ?

[bjoern-st]

@herbdool @olafgrabienski
A first shot in the dark ;-)
Since vocabularies and their terms are fieldable entities the cleanest way for translation might be following the same steps as it is done for node translation.
This would mean, that we have minor changes at the taxonomy module. We would need respecting to nodes an additional database column like ttid with the id of the source vocabulary respectively source term.
Then hopefully everything that needs to be done is to follow the same steps in the translation module as it is done for nodes.

[herbdool]

@bjoern-st That sounds like it might work. Though a simpler version wouldn't have translation sets, which nodes use.

I took a look through the PR and didn't see where you mention "using the node title from the underlying / translated node to create the translated menu item."

[olafgrabienski]

@herbdool To understand the simple approach which would be similar to the current approach regarding menu items (#225), I'm trying to reformulate @quicksketch's approach described in #225 (comment), here for taxonomy terms:

1. If Language module is enabled each taxonomy term allows the assigning of a language for that individual term.
2. The vocabularies would not have any language settings.
3. Administering a vocabulary would have a select list at the top, allowing you to filter the vocabulary by a particular language.
4. Creating a new taxonomy term would default to the language that the vocabulary is currently filtered by in the admin UI.
5. Undefined language taxonomy terms continue to show up in all languages.
6. Taxonomy terms that have a language defined only show up when the interface language matches the taxonomy term language.

So, it wouldn't be possible to translate a taxonomy term directly but to assign a language to a term and to create respective terms in another language, and these terms are not really 'connected' right? I guess that would work good with predefined vocabularies: first you create terms in different languages. Then, adding and translating node content, you just choose the terms created before in the languages you need.

How would that work with free tagging? Let's say you add an English node and tag it with the term "example". How do we tell the term to be in English language as well? Should that happen automatically? Doesn't sound bad on first sight, but what about undefined language terms in this case?

[olafgrabienski]

@bjoern-st @herbdool Regarding the different approaches (simple, without translation sets vs. like node translation) I'm double-minded at the moment:

In my opinion, long-term it should be possible to just translate everything in Backdrop by simply using a "Translate" link, no matter how the translation is implemented technically. The node-like approach would allow such a behaviour in this case. If this approach is difficult to implement, or as long as it is controversial, I'd be very happy with a simple approach as well.

[bjoern-st]

@herbdool @olafgrabienski
I think I found some kind of roadmap for this issue.
It might be a good idea to do this in a submodule of the taxonomy module that can be rolled out in a release and than activated if needed.

I see several pros for that approach:

• concerning on the problems rolling out translatable menu items in 1.10 when the site breaks until you run update.php we would have a safe upgrade path. The necessary database updates can be done when this (sub)module is activated.
• we could take advantage of the great work that is already done in the i18n module
• it is an option, so the taxonomy module stays untouched. To make the taxonomy module make translatable by default would result in a lot of unnecessary stuff going on if you do not need the translation option.

What do you think?

[olafgrabienski]

@bjoern-st Thanks for sharing your submodule idea! Are you suggesting it for any of the above mentioned approaches --simple, without translation sets vs. like node translation -- or to a particular one?

So far I'm not sure about the idea... Depends also on the translation approach:

• With the simple approach, in my opinion it's more straightforward to provide taxonomy translation if Language module is enabled.
• Going the node-like approach, should we consider to imitate the content-type multilingual settings?

I'd suggest to decide about the translation approach first (and to discuss the submodule idea later).

There are still open questions even for the simple translation approach, e.g. how would it work with free tagging, see #2934 (comment). (Going to tag the issue with "needs feedback".)

[herbdool]

I've taken an initial stab at this backdrop/backdrop#2360. I believe this was the simplest approach to implement.

Since we don't want to put too many different ways of translating into core, either we stick with this approach and leave taxonomy locale for contrib, or we leave this for contrib and add ability to translate the terms as strings in locale to core.

Like menu item translating this just provides the basic ability to set a language for a term. It doesn't provide ability to put terms into translation sets. This to could be left for contrib if we want to take that approach.

The PR still needs some work: existing tests need to pass and maybe write some new. But it can be manually tested before that.

[klonos]

I cannot say much from a technical approach view, since this is way over my head. My main concern is multilingual D7 sites being ported to Backdrop. If we can accomodate that, then I am assuming it'd be fine.

[klonos]

...@herbdool I have done a very rough pass and left some comments in the PR. Do you plan to add an option to enable multilingual support per vocabulary?

[herbdool]

The intent of this PR, similar to the intent of the multilingual menus, is not to replicate all the functionality of i18n but of providing a basic option in core. So that means that some sites can be upgraded to Backdrop but others not. It all depends on the multiple, complex options they used in i18n.

@klonos, what do you mean by "option to enable multilingual support per vocabulary"? Currently the PR just focuses on the terms. A term can be "undefined" by default or it can be a language.

[klonos]

@herbdool in the content type configuration form, there is an option to enable multilingual support:

...if this option is not enabled, then the language selection dropdown menu is not shown in the node add/edit form for that content type. I would expect the same functionality to be available per vocabulary.

[olafgrabienski]

if this option is not enabled, then the language selection dropdown menu is not shown in the node add/edit form for that content type. I would expect the same functionality to be available per vocabulary

As I saw in the sandbox site, the option is now available in the PR.

[olafgrabienski]

Impressions from my look at the sandbox

I've created a couple of test posts, first in a vanilla site, then with enabled multilingual support in a vocabulary and later also with multilingual in the post content type.

With multilingual support, the language filter on the term page of a vocabulary works fine!

The filter options are: "All languages", "English", "Italian" and so on, depending of the enabled languages on the site. As the filter help text explains, Terms with the "All" language will be shown in every language. In my opinion, to manage vocabularies with many terms, it would be useful to also have "only" options for the single languages: "Only English" and so on.

In an early comment above I've asked how we would set the language of taxonomy terms if editors create terms via the tag style widget. Here's an idea: When a person writes a post and if she sets the language of the post, then the tags of the post should be set to the same language. Testing the sandbox, actually (and first unconsciously), I expected such a behaviour. @herbdool What do you think about it?

That's all for the moment. Let me know when I should have a look at specific other parts of the PR!

[herbdool]

Thanks Olaf for reviewing.

1. I kept the language filter the same as the menu links filter. I think we should be consistent across both. Perhaps a separate issue can be created to add "only" options for both menus and taxonomy.
2. I didn't do anything with autocomplete but I think you've got a point. Should it follow the selected language even if the interface is in a different language? What seems more intuitive?
3. Could you test the tokens as well? I don't know if they're working.
4. I wonder if we need a new test for this functionality.
5. I've given some thought to how to make it friendly with an updated i18n contrib module if and when that gets worked on. And also ensure someone upgrading from D7 will at least have language set for their terms.

[klonos]

I have given the PR sandbox a go and it works really good @herbdool 👍

Here are my comments:

• I just realised that all menus are automatically language-enabled. Now I see why you had that be the same for all vocabularies originally before I mentioned it. Will follow up on that in another issue, because a) consistency with content types, and now vocabularies, b) one option for site admins when configuring the menu - one less option for content editors when editing/creating menu items (if not intended to be multilingual).

• Visiting the "List terms" tab of the vocabulary always resets the language to English, which causes terms in other languages to be hidden. With menus, after adding a menu item, the user is redirected to the menu item list page, with the language of the item last added being passed as a filter argument. So for example after adding a menu item in Italian, the user is redirected to the Italian version of the menu list. But terms work differently in that after adding a term, we redirect the user to the "Add term" page so that they can add another term. Although in the frontend it might make sense to filter according to the selected language, I would expect the admin UI to be showing all languages by default, with the language filter at the top being the means to filter terms (and menu items) by a specific language. If we do not want to change the default behaviour (because lists of terms in all languages could get rather long), then we should at least make it more obvious that there are items in other languages that are hidden form the view. Perhaps some text at the top of the list saying "showing x out of z terms in this vocabulary".

• If the language drop-down is the only filter we'll be adding to the term listing page, then I would prefer it to be autosubmitting and filtering as I select a language.

• The "Terms can be assigned multiple languages" status text in the vertical tab is too wordy and "technically" inaccurate. If we wanted to be precise, it should be "A language may be assigned to each term" (or something like that). But I think we should just go with "Enabled"/"Disabled" if others agree.

• Disabling the multilingual option for a vocabulary retains all terms 👍 ...re-enabling it, remembers their languages 👍

• Vocabularies with multilingual support disabled should not have the "Language" column in the term listing page (nor should they have the "weight" column, but that's a separate issue).

• The "Language" field should not be shown in the "Manage fields" tab when multilingual is disabled.

I have not tested the tokens, nor deleting/adding languages or disabling the language module, nor how translation works.

The autocomplete thing is tough to answer because it is per use case 😑

Thanks for your work on this so far @herbdool 🙏

[olafgrabienski]

Thanks for your reply @herbdool!

1. Re keeping the language filter the same as the menu links filter due to consistency. A separate issue to support "only" options both for menus and taxonomy is a good idea.

2. In my opinion, the Autocomplete term widget should follow the selected language of the content item even if the interface is in a different language. I guess there are only few use cases where people want content of a specific language to be tagged with a term of a different language. Do you have an example use case for that approach in mind? At the same time, it's quite usual to use the editorial interface in only one language (e.g. English) even when people work with multilingual content. In that case, it would be counterproductive if terms would get the language of the interface.

3. Regarding tokens, do you mean [term:langcode] for URL alias? That seems to work generally, here are some examples:

• tags/und/test-tag-all-languages (All / Undefined)
• tags/en/test-term-english (English)
• tags/it/test-term-italian (Italian)

NB: the latter alias was only displayed in the respective field of the Edit form; when I viewed the term, it wasn't aliased (but was like taxonomy/term/30), so we've got a problem here.

[herbdool]

Thanks for the feedback. I'll try tweak things a bit more.

By the way, @olafgrabienski I took a look at the link you pasted https://www.drupal.org/project/i18n/issues/2506981. It looks like it addresses one part of it, which is the easy part: when saving the node it will set the language of new terms based on the node language (if it's set). However, it's missing the part of what existing terms to show in the autocomplete. That's the tricky part because the node language could change in the middle of selecting terms. I don't think there's any easy solution for this part. Alternatively, we could not even try: allow them to select existing terms regardless of language. Or, allow them to only select existing terms based on the interface. The latter seems too convoluted to me.

[klonos]

...I know I am not helping, but wanted to point out that we would not be having such issues if we were using translation sets. We would be using the "main" or "source" versions of the terms in the autocomplete if no translated versions available, and when rendering content, we'd link the translated version for the current language.

Just sayin'

[herbdool]

@klonos I'm not sure if that's true. This isn't as simple as showing a node; this is a reference between two entities. Showing the "translated version" in the autocomplete would mean that it would need to use the interface and not the chosen language of the node. And it seems it would mean that the term ids saved would always be the source id, or alternatively the term ids would switch depending on the interface. I'm not sure if we'd want that.

I think if it were that easy then D7 would have already had it figured out.

[klonos]

Hmmm, ...I don't claim to have deep knowledge of i18n and l10n; this is just how I remember things used to work. But it's been many years since last time I've touched a multi-lingual site and my memory is most likely failing me ...plus back then I would not even "dare" look at the code and try to understand it. ...it was just my first steps into Drupal-land and hesitant surface-scratching of php coding.

I guess what I am trying to say is that I have faith in you @herbdool ...please carry on 😄

[herbdool]

@klonos I'm not saying I am 100% certain but my feeling is that it's complicated.

I took a look at adding automatic help text to autocomplete tags on front end. I can't tell if it is possible or if it's recommended (e.g. how does a site builder remove the text if they need to?). On the post content type a tags field is created on install with some "Enter a comma-separated list of words..." text, but on any other term fields created there is no automatic help text. I can see the usefulness of this but looks like it is up to the site builder to ensure the field text on the front end reflects what the user needs.

[olafgrabienski]

@herbdool, thanks for updating the PR! I've seen that the setting "Follow interface language" isn't anymore on the Field settings but on the general field configuration page, and now it can be changed even if there is already data for the field in the database. I think that's a great improvement! (The title of backdrop/backdrop@bfc7cb4, "Move setting to widget", is a bit misleading though. Did you first consider to move it to the Widget type tab? That would be an even better place in my opinion but if it's difficult, the general tab seems also fine to me.)

we could partially change it to use the content language, but only for new terms

Thanks also for your thoughts on interface vs. content language above - #2934 (comment) - and on the PR page - backdrop/backdrop#2360 (comment). Being able to use content language as an alternative, would be great, even if only for new terms and not for term suggestions in the autocomplete widget. I see that it's tricky for term suggestions, and I have no idea how to handle it. (If others have ideas, we can still make follow-up improvements.)

Speaking of follow-ups: Do you plan to add a setting for using content language in this issue, or would you prefer a follow-up for content language related features?

[herbdool]

@olafgrabienski I can change it in here to follow content for new terms. It is much easier and simpler for new terms than for autocomplete.

It's not possible to show on the widget tab AFAIK. That is reserved solely for switching widgets. I moved the setting to hook_field_widget_settings_form thus my comment made sense to me since it is now depending on the widget.

[olafgrabienski]

@herbdool I've tested the updated PR, see backdrop/backdrop#2360 (comment). Apart of that, I guess there was a misunderstanding in the discussion about following interface vs. following content language. Based on the state of the PR where terms were following the interface language, I thought that following the content language would be an additional approach, leaving the interface approach as an option e.g. for people who upgrade from Drupal 7 / i18n and don't want to change the behaviour.

[klonos]

The "Follow content language" checkbox should not be available if the multilingual support is not enabled in the vocabulary. Right @herbdool?

[olafgrabienski]

Tested the PR again, it works generally good now! If the checkbox "Follow content language" on a content type is enabled, the language of new terms becomes the same as the content language (or "All", if no content language is set).

I agree with @klonos that the checkbox on the content type shouldn't be available if the vocabulary isn't configured for multilingual support. (Or, we could make it available but not editable in this case, and add a hint to the vocabulary setting.)

Speaking of vocabulary configuration: The span element with the setting summary for the "Multilingual support" tab is empty, irrespective if multilingual support is enabled or not.

[herbdool]

@klonos @olafgrabienski yes, it won't assign language to terms if it's not enabled on the vocabulary but it's not possible, or at least not easy, to make the checkbox depend on the vocabulary. That's because the vocabulary select is on the same form as the checkbox. It would have to be some fancy JS which I don't know if we use anywhere else since generally these need to be usable without JS.

We could just make a note on the help text.

[herbdool]

@olafgrabienski can you explain more what you mean by empty span?

[olafgrabienski]

@herbdool Sure, I'll try: it's the description text usually found in the vertical tabs which changes depending on the setting. Here's a screenshot where I've highlighted the element and its markup on the "Permissions" tab. You'll see that in comparison the text is missing on the "Multilingual support" tab:

[herbdool]

@olafgrabienski okay I've fixed that. @klonos I've edited the description.

If you're okay with it now (and my reasoning for why the follow_content_language can't be improved much further) then are you okay with RTBCing this?

[klonos]

Yep, RTBC 👍 ...I might take a crack at figuring out if there's an easy way to have the checkbox be disabled if multilingual is not enabled, but that can be a follow-up 😉

[olafgrabienski]

Yep, RTBC

I agree as well, and thanks for your work, @herbdool !

So, potential follow-ups would be:

• Disable the Follow content language checkbox for the autocomplete in the content type if Multilingual support isn't enabled in the vocabulary configuration.
• Make the suggestions of existing terms in the autocomplete widget aware of the content language. The tricky one, see Translate taxonomy terms #2934 (comment).
• Provide another option for the autocomplete in the content type to follow the interface language instead of the content language. Mainly for former Drupal 7 sites.

[herbdool]

@olafgrabienski thanks!

Your first and second follow-ups are both tricky. The first would require JS and I'm not sure if that's a good idea if it's hasn't been done somewhere else in field settings already. The vocabulary isn't readily available to the widget settings.

The last one I'm not sure is important to most people. You convinced me that following the content language was more intuitive and more likely to be assumed. But it could be a contrib module.

[quicksketch]

I've merged backdrop/backdrop#2360 into 1.x for 1.12.0! Thanks @herbdool, @olafgrabienski, @klonos, and @bjoern-st! You guys really rocked it to put this together!