Check hugo.Environment == “production” rather than HUGO_ENV=production environment variable

[mhansen]

Hi, this is forked off from #217 to track a systemic fix for how Docsy by-default sets all your pages to noindex. I think this is a fairly high-severity issue: sites migrating to Docsy might find themselves delisted from Google until Google reindexes.

https://discourse.gohugo.io/t/checking-env-variable-prevents-site-from-being-indexed-by-google-on-aws/23707
suggests that instead of

{{ if eq (getenv “HUGO_ENV”) “production” }} 

we should use

{{ if eq hugo.Environment “production” }}

Then running with

hugo will build for production config.

hugo.Environment is defined at https://gohugo.io/variables/hugo/

I wonder how much backwards-compatibility concerns there are here for people updating though.

[LisaFC]

Default no-index is also a Hugo default (I think), and I know a lot of users don't want their site indexed while they're actively working on it (eg testing a live deployment method) or sharing it as a preview.

We do tell users to switch to "production" in the hosting and deployment page but could possibly make it more prominent to ensure they see it?

[mhansen]

I can't find much by googling if hugo has noindex as the default. I *think* hugo doesn't ship with any default HTML templates (except maybe the sitemap.xml?) so it's up to the users to specify whether they want it to be indexed or not. I suppose this might be a tricky tradeoff if there are other users who want their site not indexed. On the other side, it looks like the current approach led to Graphviz and gRPC's pages being delisted from Google for a while (grpc/grpc.io#600). The docs about setting HUGO_ENV were under the "Deployment with Netlify" section and I wasn't deploying with Netlify (GitLab pages instead) so I didn't read them :-) https://www.docsy.dev/docs/deployment/#deployment-with-netlify

…

On Tue, 29 Jun 2021 at 04:45, LisaFC ***@***.***> wrote: Default no-index is also a Hugo default (I think), and I know a lot of users don't want their site indexed while they're actively working on it (eg testing a live deployment method) or sharing it as a preview. We do tell users to switch to "production" in the hosting and deployment page but could possibly make it more prominent to ensure they see it? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#611 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAZYOIIENIZFUGFLP25BIDTVC7NZANCNFSM47K2H2OA> .

[LisaFC]

How about we make the information more prominent on the deployment page, as I don't want to fix one set of problems ("oh no, my site is not indexed!") by creating new problems for other people ("oh no, my site is indexed!"). :)

I'll do a PR.

[LisaFC]

Though hmmm, I've looked at it more closely and I think your approach for the check might be right anyway - I think the fact that running "hugo" rather than "hugo server" gives you a production build is something that wasn't entirely evident (or possibly true) when we first built the template (also I am not sure why checking HUGO_ENV is potentially giving a different result from checking hugo.Environment or hugo.isProduction)

It does not help that the Hugo docs themselves are confusing. Let me investigate further and we will make sure we're doing the right thing!

[LisaFC]

Curioser and curioser.... gohugoio/hugo#6456, it seems they were actually different for a while. I'll try building/deploying some sites when I have a moment and see what behaviour I get...

[mhansen]

Wow, Hugo sure is full of surprises! Good digging

…

On Fri, 2 Jul 2021 at 20:53, LisaFC ***@***.***> wrote: Curioser and curioser.... gohugoio/hugo#6456 <gohugoio/hugo#6456>, it seems they *were* actually different for a while. I'll try building/deploying some sites when I have a moment and see what behaviour I get... — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#611 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAZYOJB7ICRS56RN6MFNMTTVWLEBANCNFSM47K2H2OA> .

[LisaFC]

A quick investigation:

• Deploy site from Netlify with HUGO_ENV= production and the current head.html -> site is indexable
• Deploy site from Netlify without HUGO_ENV=production and the current head.html -> site is not indexable
• Deploy site from Netlify without HUGO_ENV=production and the check changed to hugo.IsProduction -> site is indexable

I am now confused.

[LisaFC]

So from looking at the code it looks like they still are different things - the fix to that bug just meant that if it was set, Hugo would read it in and use it as its hugo.Environment value. Our check was for that specific environment variable, so given that I hadn't set it, the check was "false", even though hugo.IsProduction was "true". Funtimes!

What behaviour do we think makes most sense? Given that the default Hugo behaviour is actually to build a "production" site then maybe we should just update to check hugo.IsProduction?

It won't break anything for users who have set HUGO_ENV to "production" and we could always tell users to explicitly set HUGO_ENV to "something not production" if they don't want indexing.... Hmmm.

[chalin]

@LisaFC et al.: somehow #540 reverted #413. The latter introduced the use of hugo.IsProduction, in replacement of the environment-variable checking code. IMHO, use of hugo.IsProduction is the better (if not proper) way to determine the build environment, in particular since the build environment can be specified using the -e flag command line flag.

Related: #651

/cc @nate-double-u

[chalin]

Moved my previous comment to a separate issue (rather than burying it inside this one).

[LisaFC]

Fixed in #653