writeStats seems to miss some generated classes and IDs

[cyChop]

The writeStats feature and the contributing docs are awesome. Thanks a lot!

However, it seems that it misses some IDs and classes. I have yet failed to identify a pattern in the missed objects. PurgeCSS works fine but removes classes that are needed by the layout.

So far, it seems that the missing IDs and classes do not come from any .Content, only from layouts.

I have noticed, for instance, that classes .term, .tag and .category are missing (in my layout, they're used in conjunction, as in class="term tag" and class="term category").

I also miss the ID of my comment form (#staticman-form), though classes specific to that form are being detected (.replying-title).

I'm sorry to say that, as I don't see a pattern, I don't know how to reproduce or build a minimal case.
The sources for the website I use Hugo with are available at https://gitlab.com/keyboardplaying/keyboardplaying.org/ (branch develop is using Hugo 0.69).

The minimal approach to reproducing would be:

npm i
npm run build:hugo
cat site/hugo_stats.json

What version of Hugo are you using (hugo version)?

$ hugo version
Hugo Static Site Generator v0.69.0-4205844B/extended linux/amd64 BuildDate: 2020-04-10T09:16:58Z

Does this issue reproduce with the latest release?

Yes (using the latest version)

[bep]

Thanks. I tried to stay on the right side of false positives, but one never knows... But I have tested this on some of my sites.

In general I think that:

• The element parser is solid (someone from Google wrote that)
• I detect HTML elements in a streaming way (which is why it's so fast), and that is rather simple -- so I suspect the problem is here. I look for opening "<" and ending ">", but there is also some quote logic there to handle Vue etc. bindings (they can have "<" in them).

[cyChop]

Thanks for your explanation. This gave me an idea and, so far, it checks out: the tags and classes that are missing have some Hugo interpolated variables as an attribute.

For instance:

<form id="staticman-form" method="post" action="{{ .Site.Params.staticman.api }}">
<a href="{{ $termUrl }}/" class="term category" rel="tag">

I suppose it is possible the curly braces may mess with the < and > detection, don't you think? But this may have nothing to do with it, since everything passed in .Content is detected, so the parser works on the file after Hugo interpreted it.

[bep]

I suppose it is possible the curly braces may mess with the < and > detection, don't you think?

No, this is done on the final output , so those braces are gone. That was one of the motivations behind this, to also catch classes etc. generated by the templates.

[bep]

I quickly tested your site, but get lots of build ERRORs, which could explain missing stuff ...?

[cyChop]

That's surprising. I don't have any build mistake (I work on two separate computers and use Gitlab CI too).

I usually use npm run build (that triggers some additional builds, not only Hugo). The last CI's build today was fully successful. I tried doing it the minimalist way but couldn't reproduce.

Maybe you could copy the errors?

[bep]

... yes, "npm run build" builds it successfully. I assumed too much. I will look into this later.

[bep]

OK, I have run your site and I can find all of the "missing" classes and IDs you mention in the first post. Hmm....

[regisphilibert]

I was not able to find a pattern for this issue, but I can point to some examples:
Here: https://github.com/theNewDynamic/thenewdynamic.com/blob/e55857f6fa8a6fda2e7802503fd3f798bd5ab00d/layouts/partials/header.html#L19

The class px-3 and pb-1 are not added to hugo_stats.json. Whereas hover:text-gradient which is, just like the other two also only present in this template file, is added.

[bep]

@regisphilibert but your link give me a 404.

[regisphilibert]

Arf, I thought it was public. Will try and spin up a test repo for you today. In the mean time, in the off chance it might help, here's the content of that template file (culprit line is in the else clause) :

<header class="py-6 flex fixed z-50 w-full top-0 items-center bg-gray-100">
  <div class="container flex items-center justify-between">
    <a href="{{ site.Home.RelPermalink }}" class="flex items-center">
      <div class="w-6 md:w-8">{{ partial "svg/logo-compressed.svg" . }}</div>
      <div class="text-xl lg:text-xl md:text-xl ml-3">
        The New Dynamic
      </div>
    </a>
    <div class="lg:hidden flex flex-col justify-between">
      <div class="bg-gradient-horizontal h-1 w-10"></div>
      <div class="bg-gradient-horizontal h-1 w-10 my-1 ml-1"></div>
      <div class="bg-gradient-horizontal h-1 w-10"></div>
    </div>
    <nav class="hidden lg:block text-lg flex items-center">
      {{ range site.Menus.main }}
        {{ if eq "contact" .Page.File.BaseFileName }}
          <a class="text-base bg-black dark:bg-white text-white hover:bg-gradient pl-4 pr-3 pb-1 rounded inline-block lowercase" href="{{ .URL }}">{{ .Name }}</a>
        {{ else }}
          <a class="text-base  hover:text-gradient inline-block px-3 pb-1 rounded lowercase" href="{{ .URL }}">{{ .Name }}</a>
        {{ end }}
      {{ end }}
    </nav>
  </div>
</header>

[bep]

here's the content of that template file (culprit line is in the else clause) :

Are you really sure the "else clause" is ever invoked? Note that the "class list" is built in the content in /public -- not your templates.

[regisphilibert]

Are you really sure the "else clause" is ever invoked?

Yep.

The repo's now public, Webpack is not hooked on hugo_stats.json but the file is produced.

https://github.com/theNewDynamic/thenewdynamic.com/blob/e55857f6fa8a6fda2e7802503fd3f798bd5ab00d/layouts/partials/header.html#L19

[bep]

➜  thenewdynamic.com git:(master) ✗ grep px-3  hugo_stats.json && grep pb-1 hugo_stats.json
      "px-3",
      "pb-1",

I take a closer look at the source code see if there are some obvious reason why this works for me and not for you, the code in question should be rather simple. Can I ask the people in this thread what OS you run with?

[regisphilibert]

Hold on, I'm seeing px-3 now in my hugo_stats.json. Will have to run more tests and get back to you. Sorry about that.

[cyChop]

@bep I discovered the issue on Ubuntu and tested again this morning on Windows 10.

[bep]

I assume your build is without ERRORs?

[cyChop]

If you're asking me, yes it is without ERRORs.

[bep]

OK, now I have seen it myself on one of my sites on Netlify, so I at least know it's real. I'll try to fix it.

[bep]

... but I built that same site on my MacBook 200 times without any diffs. Hmm...

[bep]

OK ... so I found the culprit, I think. I can reproduce this with @regisphilibert 's site when running with hugo --minify.

[cyChop]

Seems promising, @bep! I tried removing the --minify option from my build script and the classes and IDs that were missing before reappeared in the hugo_stats.json

[bep]

Yes, I found the code logic flaw. The minfier is stripping quotes in hrefs so the home link turns into this:

<a class="text-base hover:text-gradient inline-block px-3 pb-1 rounded lowercase" href=/></a>

Which looks like a closing tag for my naive parser, so I just drop it (in retrospect, an obvious mistake).

[AlanBreck]

I'm having this same issue. Running Hugo v0.72.0/extended. An entire partial is being missed by writeStats. None of the ids or classes can be found in the resulting hugo_stats.json file.

[regisphilibert]

This issue has been closed, if it's isolated to one template file, this look like another problem. In any case you should start detailing your problem on the discourse and share a repo or more details there if you can.
Thanks a ton!

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.