Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WriteStat feature eat memory since map key was overwrite #7945

Closed
blackb1rd opened this issue Nov 11, 2020 · 2 comments
Closed

WriteStat feature eat memory since map key was overwrite #7945

blackb1rd opened this issue Nov 11, 2020 · 2 comments
Milestone

Comments

@blackb1rd
Copy link

blackb1rd commented Nov 11, 2020

What version of Hugo are you using (hugo version)?

$ hugo master branch

Does this issue reproduce with the latest release?

Yes.

issue came from #7320

At function

func (w *cssClassCollectorWriter) Write(p []byte) (n int, err error)

Assume string(w.buff.Bytes()) = "<br" and run into function at first loop

1. // First check if we have processed this element before.
2. w.collector.mu.RLock()
3. // See https://github.com/dominikh/go-tools/issues/723
4. //lint:ignore S1030 This construct avoids memory allocation for the string.
5. seen := w.collector.elementSet[string(w.buff.Bytes())]                              
6. w.collector.mu.RUnlock()
7. if seen {
8.     w.buff.Reset()
9.     continue
10. }
11. s := w.buff.String()
12. 
13. w.buff.Reset()
14. 
15. if strings.HasPrefix(s, "</") {
16.     continue
17. }
18. 
19. s, tagName := w.insertStandinHTMLElement(s)                            
20. el := parseHTMLElement(s)
21. el.Tag = tagName
22. 
23. w.collector.mu.Lock()
24. w.collector.elementSet[s] = true
25. if el.Tag != "" {
26.     w.collector.elements = append(w.collector.elements, el)
27. }
28. w.collector.mu.Unlock()

line 5: checking map key "<br"
line 19: key was overwrite from "<br" into "<div"
line 24: overwrite key was added to map "<div"

This is causing memory go to from 1.5GB to 4GB from my case.

@bep bep added the Bug label Nov 27, 2020
@bep bep added this to the v0.79 milestone Nov 27, 2020
@bep
Copy link
Member

bep commented Nov 27, 2020

Thanks for this detailed bug report, I will fix it.

@github-actions
Copy link

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Jan 26, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

2 participants