Special characters in taxonomy and slugs #1180

nicolinuxfr · 2015-05-29T10:41:03Z

I'm trying Hugo for my personal blog which has a lot of taxonomies. And as I'm writing in French, many taxonomies have special characters in them, like an accentuated letter.

Right now, I'm using WordPress which has the perfect behavior on this matter. The taxonomy name can have any special characters (for example, "Gérard Depardieu"), the slug associated with it only has standard characters (gerard-depardieu). But when you display the taxonomy archive, you still have the special characters : so in this case, you would not have "Gerard Depardieu", but "Gérard Depardieu"). You can see the example live here : http://voiretmanger.fr/acteur/gerard-depardieu/.

I don't know if Hugo could do the same. I know WordPress has a database, so it's easier. But I can see some solutions (or hacks) to make it work : either look in the metadata associated with the post to display the name of the taxonomy on the archive page, or have a "table" (a YAML/TOML config file, I guess) with all correspondances.

An idea, to end my Gérard Depardieu example :

gerard-depardieu: "Gérard Depardieu"

I hope a solution will be feasible, because it's the main thing that would be keeping me out of Hugo and with WordPress. I think I can find a solution for every other problems I have…

Thanks anyway for your time !

The text was updated successfully, but these errors were encountered:

bep · 2015-05-29T19:00:51Z

This isn't hard to fix and I understand the motivation for it.

We already do some URL normalization of the taxonomies, but probably didn't think about monsieur Depardieu back then. This might be a breaking change (as someone will have some URLs changed), but it's the right thing to do.

So gerard-depardieu not gérard-depardieu etc. Fixes gohugoio#1180

nicolinuxfr · 2015-05-29T22:14:10Z

OK, great for the first and easy part ! Thanks :-)

There's one more problem though : I don't want the accent in the URL, but I want it on the archive page (like on the screenshot).

With nothing more, I don't see how it could work. Am I wrong ?

bep · 2015-05-29T22:41:37Z

You are wrong. The accents (and some others) are ONLY stripped for the paths (on disk and the URL presented to the user). The taxonomy name will be preserved as written.

I added the "Gérard Depardieu" tag to one of my posts to make sure. It has nothing to do with the actor, but I might publish it just to confuse people.

bep · 2015-05-29T22:45:21Z

OK, I retract the last I said above -- there is one more fix to do, will check on that tomorrow.

bep · 2015-05-29T23:34:58Z

I can get this to work in a hackish-kind-of way, but will have to look at this later -- to do a proper fix.

nicolinuxfr · 2015-05-30T05:55:39Z

Well thanks ! I'm impressed, we are definitely not on the WordPress pace here… :-)

So the taxonomy `Gérard Depardieu` gives paths on the form `gerard-depardieu`. Unfortunately this introduces two imports from the `golang.org/`, but Unicode-normalization isn't something we'd want to write from scratch. See https://blog.golang.org/normalization See gohugoio#1180

Before this commit, taxonomy names were hyphenated, lower-cased and normalized -- then fixed and titleized on the archive page. So what you entered in the front matter isn't necessarily what you got in the final site. To preserve backwards compability, `PreserveTaxonomyNames` is default `false`. Setting it to `true` will preserve what you type (the first characters is made toupper for titles), but normalized in URLs. This also means that, if you manually construct URLs to the archive pages, you will have to pass the Taxonomy names through the `urlize` func. Fixes gohugoio#1180

So the taxonomy `Gérard Depardieu` gives paths on the form `gerard-depardieu`. Unfortunately this introduces two imports from the `golang.org/`, but Unicode-normalization isn't something we'd want to write from scratch. See https://blog.golang.org/normalization See #1180

nicolinuxfr · 2015-06-01T15:47:52Z

Just a quick note to thank bep for his work… it works exactly as I wanted it ! So it's perfect as far as I am concerned. :-)

RickCogley · 2015-06-16T11:11:07Z

Interesting @bep, because when you "normalize" Japanese, and remove the "accent" from katakana, the meaning changes completely. In some cases it's unrecognizable or at least quite humorous.

bep · 2015-06-16T11:27:56Z

OK, so that part may have been a bad idea ... I can revert that if I'm convinced ... Hmm, languages. @nicolinuxfr

nicolinuxfr · 2015-06-16T11:33:33Z

Hum, not a bad idea for me anyway. I hope I will be able to keep this really important feature for me.

RickCogley · 2015-06-16T13:03:39Z

@bep, I can give you precise information about which characters in Japanese are losing their "accent" if that will help.

For instance:

ビ  going to ヒ

Please advise how I can assist in figuring it out.

bep · 2015-06-16T13:11:44Z

@nicolinuxfr yes, that was the input I wanted (how important is it). @RickCogley I think the solution is to add an option around this, default old behaviour.

I will fix this later tonight. BTW: This is just about the URLs/file paths.

nicolinuxfr · 2015-06-16T13:15:59Z

He he, it's not that we love these, but the meaning is completely different without accents… :-)

Thanks for trying to satisfy everyone here !

RickCogley · 2015-06-16T13:34:47Z

@bep, the bit of the character in Japanese that is getting stripped is called a "dakuten" https://en.wikipedia.org/wiki/Dakuten. There is one that looks like a double quote and one that looks like a circle. After rendering to public using hugo server, I see this:

rcogley@jrcmbp2015:~/dev/eSolia/public/ja/topics|rc-working-2⚡
⇒  ll
total 8
drwxr-xr-x  4 rcogley  staff   136 Jun 16 22:22 about
-rw-r--r--  1 rcogley  staff     0 Jun 16 22:23 index.html
-rw-r--r--  1 rcogley  staff  1928 Jun 16 22:23 index.xml
drwxr-xr-x  4 rcogley  staff   136 Jun 14 20:58 professional
drwxr-xr-x  4 rcogley  staff   136 Jun 14 20:58 お問い合わせ
drwxr-xr-x  4 rcogley  staff   136 Jun 16 22:25 はひふへほ
drwxr-xr-x  4 rcogley  staff   136 Jun 14 20:58 サーヒス
drwxr-xr-x  4 rcogley  staff   136 Jun 16 22:25 ハヒフヘホ

I'm using "topics" as a taxonomy here. The last 3 lines in the ll output have these marks, and are supposed to be:

topics:
  - About
  - ばびぶべぼ
  - バビブベボ
  - ぱぴぷぺぽ
  - パピプペポ

But Hugo strips the dakuten, and combines the four into two. That is, ba バ pa パ both become ハ.

bep · 2015-06-16T13:46:25Z

@RickCogley I know what we strip and how to not strip to them ... Will fix tonight.

bep · 2015-06-16T17:28:32Z

@nicolinuxfr please add RemovePathAccents = true to your config to keep the behavior you want.

nicolinuxfr · 2015-06-16T19:00:40Z

Great, thanks for keeping me satisfied along with everyone else ! :-)

Is it merged yet so I can try using home-brew or should I compile it manually ?

bep · 2015-06-16T19:32:19Z

Didn't you have some success with go get -u .... ? and yes its merged.

dunn · 2015-06-16T19:53:58Z

@nicolinuxfr you can install the absolute latest version with brew install --HEAD hugo

nicolinuxfr · 2015-06-16T20:05:08Z

@dunn are you sure ? I tried to upgrade that way and the build seems old :

Hugo Static Site Generator v0.14 BuildDate: 2015-05-26T18:46:46+02:00

EDIT : oh, it seems I have a failed build because of decencies. Well, it doesn't matter, the go get -u -v github.com/spf13/hugo worked perfectly.

@bep for me, everything is still fine !

RickCogley · 2015-06-16T21:20:24Z

@bep, thanks, after a recompile with go get -u -v github.com/spf13/hugo, I have proper Japanese again. :-)

@dunn, I stopped using brew install --HEAD hugo after finding it wonky a while back, but the above always works for me.

dunn · 2015-06-16T21:30:17Z

@RickCogley yeah, new dependencies have to be added manually, so it can break. I just opened Homebrew/legacy-homebrew#40794; thanks for the heads-up, @nicolinuxfr!

RickCogley · 2015-06-16T22:27:15Z

@dunn ah, I see. Thanks. I hadn't realized that.

So the taxonomy `Gérard Depardieu` gives paths on the form `gerard-depardieu`. Unfortunately this introduces two imports from the `golang.org/`, but Unicode-normalization isn't something we'd want to write from scratch. See https://blog.golang.org/normalization See gohugoio#1180

Before this commit, taxonomy names were hyphenated, lower-cased and normalized -- then fixed and titleized on the archive page. So what you entered in the front matter isn't necessarily what you got in the final site. To preserve backwards compability, `PreserveTaxonomyNames` is default `false`. Setting it to `true` will preserve what you type (the first characters is made toupper for titles), but normalized in URLs. This also means that, if you manually construct URLs to the archive pages, you will have to pass the Taxonomy names through the `urlize` func. Fixes gohugoio#1180

And default off. Fixes gohugoio#1180

7297c1172 Add note about caching for Hugo Pipes. c91be3403 minor markdown, capitalization and spelling fixes (#1183) fd4a103bf Fix several 404 errors (#1162) 69378bc20 Update related.md 28c24e95f Add note on setting baseURL 7b1502c99 minor typo fix (#1180) 33abeb4fe Update related.md 4887563f6 Update js.md ee5f1de2e Hugo 0.74.3 986ea0c8e releaser: Add release notes to /docs for release of 0.74.3 3299b44bd Fix Asciidoctor args bcb950347 resources/js: Add option for setting bundle format 3f8324918 resources/js: Add es5 build target git-subtree-dir: docs git-subtree-split: 7297c1172754078511ac1c10ca0dfd4cab629506

7297c1172 Add note about caching for Hugo Pipes. c91be3403 minor markdown, capitalization and spelling fixes (gohugoio#1183) fd4a103bf Fix several 404 errors (gohugoio#1162) 69378bc20 Update related.md 28c24e95f Add note on setting baseURL 7b1502c99 minor typo fix (gohugoio#1180) 33abeb4fe Update related.md 4887563f6 Update js.md ee5f1de2e Hugo 0.74.3 986ea0c8e releaser: Add release notes to /docs for release of 0.74.3 3299b44bd Fix Asciidoctor args bcb950347 resources/js: Add option for setting bundle format 3f8324918 resources/js: Add es5 build target git-subtree-dir: docs git-subtree-split: 7297c1172754078511ac1c10ca0dfd4cab629506

eugenioej · 2021-02-28T17:13:51Z

@nicolinuxfr please add RemovePathAccents = true to your config to keep the behavior you want.

Thanks!!!! This is what I was looking for my site on spanish :)

github-actions · 2022-01-23T01:50:00Z

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

bep added the Enhancement label May 29, 2015

bep added this to the v0.15 milestone May 29, 2015

bep self-assigned this May 29, 2015

bep added a commit to bep/hugo that referenced this issue May 29, 2015

Remove accents in URLs

beaa491

So gerard-depardieu not gérard-depardieu etc. Fixes gohugoio#1180

bep mentioned this issue May 29, 2015

WIP: Fix taxonomy titles #1183

Closed

bep removed their assignment May 30, 2015

bep mentioned this issue May 30, 2015

Fix taxonomy titles #1184

Closed

bep closed this as completed in be38acd May 31, 2015

bep reopened this Jun 16, 2015

bep closed this as completed in 4b7c134 Jun 16, 2015

bep mentioned this issue Aug 18, 2015

Enable proper titles for Nodes #1051

Closed

lolindrath mentioned this issue Oct 19, 2015

MakePath function in helpers/path.go removes unicode "mark"s #1488

Closed

moorereason mentioned this issue Dec 27, 2016

Deprecate preserveTaxonomyNames #2835

Closed

tychoish pushed a commit to tychoish/hugo that referenced this issue Aug 13, 2017

Make removal of accents in taxonomy and section paths optional

fb45361

And default off. Fixes gohugoio#1180

KillianKemps mentioned this issue Jan 29, 2019

Custom slug is appended to bundle directory name #4870

Closed

a-fortunato mentioned this issue May 31, 2019

Taxonomies names does not match urlized #5687

Closed

github-actions bot added the Outdated label Jan 23, 2022

github-actions bot locked as resolved and limited conversation to collaborators Jan 23, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Special characters in taxonomy and slugs #1180

Special characters in taxonomy and slugs #1180

nicolinuxfr commented May 29, 2015

bep commented May 29, 2015

nicolinuxfr commented May 29, 2015

bep commented May 29, 2015

bep commented May 29, 2015

bep commented May 29, 2015

nicolinuxfr commented May 30, 2015

nicolinuxfr commented Jun 1, 2015

RickCogley commented Jun 16, 2015

bep commented Jun 16, 2015

nicolinuxfr commented Jun 16, 2015

RickCogley commented Jun 16, 2015

bep commented Jun 16, 2015

nicolinuxfr commented Jun 16, 2015

RickCogley commented Jun 16, 2015

bep commented Jun 16, 2015

bep commented Jun 16, 2015

nicolinuxfr commented Jun 16, 2015

bep commented Jun 16, 2015

dunn commented Jun 16, 2015

nicolinuxfr commented Jun 16, 2015

RickCogley commented Jun 16, 2015

dunn commented Jun 16, 2015

RickCogley commented Jun 16, 2015

eugenioej commented Feb 28, 2021

github-actions bot commented Jan 23, 2022

Special characters in taxonomy and slugs #1180

Special characters in taxonomy and slugs #1180

Comments

nicolinuxfr commented May 29, 2015

bep commented May 29, 2015

nicolinuxfr commented May 29, 2015

bep commented May 29, 2015

bep commented May 29, 2015

bep commented May 29, 2015

nicolinuxfr commented May 30, 2015

nicolinuxfr commented Jun 1, 2015

RickCogley commented Jun 16, 2015

bep commented Jun 16, 2015

nicolinuxfr commented Jun 16, 2015

RickCogley commented Jun 16, 2015

bep commented Jun 16, 2015

nicolinuxfr commented Jun 16, 2015

RickCogley commented Jun 16, 2015

bep commented Jun 16, 2015

bep commented Jun 16, 2015

nicolinuxfr commented Jun 16, 2015

bep commented Jun 16, 2015

dunn commented Jun 16, 2015

nicolinuxfr commented Jun 16, 2015

RickCogley commented Jun 16, 2015

dunn commented Jun 16, 2015

RickCogley commented Jun 16, 2015

eugenioej commented Feb 28, 2021

github-actions bot commented Jan 23, 2022