-
Notifications
You must be signed in to change notification settings - Fork 1.3k
gitserver: close packed-refs file in quickRevParseHead() #4026
Conversation
cmd/gitserver/server/server.go
Outdated
span.SetTag("args", cmd.Args) | ||
span.SetTag("dir", cmd.Dir) | ||
tr, ctx := trace.New(ctx, "runCommand", fmt.Sprintf("%v", cmd.Args)) | ||
defer tr.Finish() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Put this in the end of the defer func()
below instead:
- Then the reader doesn't have to think about the order in which defer is executed
defer
isn't free, so also worth not spawning two in such a frequently used code path
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we'll have to disagree on that. I like this pattern that's easy to copy and paste and I'm following the normal pattern of how to use defer
: for cleanup right after creating the resource.
See the benchmark below that shows defer
costs about 100ns, negligible for most purposes.
cmd/gitserver/server/server.go
Outdated
} | ||
args := strings.Join(req.Args, " ") | ||
tr, ctx := trace.New(ctx, "exec."+cmdStr, string(req.Repo)) | ||
defer tr.Finish() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
same with this one
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Same disagreement. :)
cmd/gitserver/server/server.go
Outdated
@@ -599,7 +599,14 @@ func (s *Server) handleExec(w http.ResponseWriter, r *http.Request) { | |||
// For searches over large repo sets (> 1k), this leads to too many child process execs, which can lead | |||
// to a persistent failure mode where every exec takes > 10s, which is disastrous for gitserver performance. | |||
if len(req.Args) == 2 && req.Args[0] == "rev-parse" && req.Args[1] == "HEAD" { | |||
if resolved, err := quickRevParseHead(dir); err == nil && git.IsAbsoluteRevision(resolved) { | |||
tr.LazyPrintf("rev-parse") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The trace is already named exec.rev-parse
, what is the reason this log is useful?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think I wanted something here to show when quickRevParseHead
was being called. Anyway, in the other branch I put a separate span into quickRevParseHead
. Adding a span takes about 7 microseconds according to this benchmark, so that's also nothing to worry about:
[ ~/src/github.com/sourcegraph/sourcegraph/bench ] go test -bench=NewTrace
goos: darwin
goarch: amd64
pkg: github.com/sourcegraph/sourcegraph/bench
BenchmarkNewTrace-4 1000000 7239 ns/op
PASS
ok github.com/sourcegraph/sourcegraph/bench 7.329s
func BenchmarkNewTrace(b *testing.B) {
ctx := context.Background()
for i := 0; i < b.N; i++ {
tr, _ := trace.New(ctx, "benchmark", "title")
tr.Finish()
}
}
cmd/gitserver/server/server.go
Outdated
@@ -599,7 +599,14 @@ func (s *Server) handleExec(w http.ResponseWriter, r *http.Request) { | |||
// For searches over large repo sets (> 1k), this leads to too many child process execs, which can lead | |||
// to a persistent failure mode where every exec takes > 10s, which is disastrous for gitserver performance. | |||
if len(req.Args) == 2 && req.Args[0] == "rev-parse" && req.Args[1] == "HEAD" { | |||
if resolved, err := quickRevParseHead(dir); err == nil && git.IsAbsoluteRevision(resolved) { | |||
tr.LazyPrintf("rev-parse") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
tr.LazyPrintf("rev-parse") | |
tr.LazyPrintf("HEAD fast-path") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
All this trace stuff is cross-talk from a different branch. I didn't mean to include it here. Cleaned it up.
cmd/gitserver/server/server.go
Outdated
if err != nil { | ||
tr.LazyPrintf("quickRevParseHead failed with %v", err) | ||
} | ||
tr.LazyPrintf("is %s an absolute revision? %v", resolved, git.IsAbsoluteRevision(resolved)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am unsure how performant /x/net/trace
is and have a slight concern that, given we are talking about these entire operations usually taking ~16ms or so, we may indirectly harm perf with these lazy logs. Thoughts?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The things that are harming performance for us are system calls that take milliseconds to run.
I wrote a couple of benchmarks that show there's nothing to worry about with defer
or tr.LazyPrintf
:
[ ~/src/github.com/sourcegraph/sourcegraph/bench ] go test -bench=.
goos: darwin
goarch: amd64
pkg: github.com/sourcegraph/sourcegraph/bench
BenchmarkDefer-4 20000000 100 ns/op
BenchmarkLazyPrintf-4 5000000 251 ns/op
PASS
ok github.com/sourcegraph/sourcegraph/bench 3.903s
// b_test.go
package main
import (
"context"
"testing"
"github.com/sourcegraph/sourcegraph/pkg/trace"
)
func BenchmarkDefer(b *testing.B) {
for i := 0; i < b.N; i++ {
defer func() {
// Do nothing.
}()
}
}
func BenchmarkLazyPrintf(b *testing.B) {
ctx := context.Background()
tr, ctx := trace.New(ctx, "benchmark", "title")
defer tr.Finish()
for i := 0; i < b.N; i++ {
tr.LazyPrintf("something something %d", i)
}
}
4dd8d99
to
7cb704a
Compare
This is meant to help with a customer-reported issue about search performance.
Test plan:
cd cmd/gitserver/server && go test -bench=BenchmarkQuickRevParseHead_packed_refs
Before this change the benchmark was failing with too many files open.
CC @beyang