Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Headscale crash with unknown cause #1862

Closed
1 of 2 tasks
winterheart opened this issue Apr 9, 2024 · 1 comment · Fixed by #1895
Closed
1 of 2 tasks

Headscale crash with unknown cause #1862

winterheart opened this issue Apr 9, 2024 · 1 comment · Fixed by #1895
Labels
bug Something isn't working
Milestone
v0.23.0

Comments

@winterheart
Copy link
Contributor

Bug description

Periodically headscale crashes with dump in logs:

fatal error: concurrent map read and map write

goroutine 106672824 [running]:
github.com/juanfont/headscale/hscontrol.(*Headscale).handlePoll(0xc0002fc0e0, {0x2294590, _}, {_, _}, _, {0x57, {0xc007b4ba58, 0x4}, 0x1, ...})
        /home/runner/work/headscale/headscale/hscontrol/poll.go:455 +0x2169
github.com/juanfont/headscale/hscontrol.(*noiseServer).NoisePollNetMapHandler(0xc0018d1830, {0x2294590, 0xc00411c8b8}, 0xc0010626c0)
        /home/runner/work/headscale/headscale/hscontrol/poll_noise.go:95 +0x596
net/http.HandlerFunc.ServeHTTP(0xc0010625a0?, {0x2294590?, 0xc00411c8b8?}, 0xc0019a0000?)
        /nix/store/88y9r33p3j8f7bc8sqiy9jdlk7yqfrlg-go-1.22.0/share/go/src/net/http/server.go:2166 +0x29
github.com/gorilla/mux.(*Router).ServeHTTP(0xc0051800c0, {0x2294590, 0xc00411c8b8}, 0xc001991e60)
        /home/runner/go/pkg/mod/github.com/gorilla/[email protected]/mux.go:212 +0x1e2
golang.org/x/net/http2.(*serverConn).runHandler(0x4498fd?, 0x0?, 0x0?, 0xc0013a7400?)
        /home/runner/go/pkg/mod/golang.org/x/[email protected]/http2/server.go:2368 +0xbb
created by golang.org/x/net/http2.(*serverConn).scheduleHandler in goroutine 106672803
        /home/runner/go/pkg/mod/golang.org/x/[email protected]/http2/server.go:2303 +0x21d

Environment

  • OS: Gentoo Linux

  • Headscale version: v0.23.0-alpha5

  • Tailscale version: any

  • Headscale is behind a (reverse) proxy

  • Headscale runs in a container
@winterheart winterheart added the bug Something isn't working label Apr 9, 2024
@kradalby kradalby added this to the v0.23.0 milestone Apr 10, 2024
@kradalby
Copy link
Collaborator

Could you please try the newest alpha (https://github.com/juanfont/headscale/releases/tag/v0.23.0-alpha6) and report back?

kradalby added a commit to kradalby/headscale that referenced this issue Apr 21, 2024
@kradalby
use concurrency save connectedmap, more metrics
cd0dcdb 
Fixes juanfont#1862

Signed-off-by: Kristoffer Dalby <[email protected]>
@kradalby kradalby mentioned this issue Apr 21, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
v0.23.0
Development

Successfully merging a pull request may close this issue.

metrics, tuning in tests, db cleanups, fix concurrency issue kradalby/headscale
2 participants
@winterheart @kradalby