Skip to content
This repository has been archived by the owner on Sep 30, 2024. It is now read-only.

gitserver: close packed-refs file in quickRevParseHead() #4026

Merged
merged 5 commits into from
May 16, 2019

Conversation

ijt
Copy link
Contributor

@ijt ijt commented May 15, 2019

This is meant to help with a customer-reported issue about search performance.

Test plan: cd cmd/gitserver/server && go test -bench=BenchmarkQuickRevParseHead_packed_refs

Before this change the benchmark was failing with too many files open.

CC @beyang

@ijt ijt requested review from slimsag and attfarhan May 15, 2019 03:30
CHANGELOG.md Outdated Show resolved Hide resolved
span.SetTag("args", cmd.Args)
span.SetTag("dir", cmd.Dir)
tr, ctx := trace.New(ctx, "runCommand", fmt.Sprintf("%v", cmd.Args))
defer tr.Finish()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Put this in the end of the defer func() below instead:

  • Then the reader doesn't have to think about the order in which defer is executed
  • defer isn't free, so also worth not spawning two in such a frequently used code path

Copy link
Contributor Author

@ijt ijt May 16, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we'll have to disagree on that. I like this pattern that's easy to copy and paste and I'm following the normal pattern of how to use defer: for cleanup right after creating the resource.

See the benchmark below that shows defer costs about 100ns, negligible for most purposes.

}
args := strings.Join(req.Args, " ")
tr, ctx := trace.New(ctx, "exec."+cmdStr, string(req.Repo))
defer tr.Finish()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same with this one

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same disagreement. :)

@@ -599,7 +599,14 @@ func (s *Server) handleExec(w http.ResponseWriter, r *http.Request) {
// For searches over large repo sets (> 1k), this leads to too many child process execs, which can lead
// to a persistent failure mode where every exec takes > 10s, which is disastrous for gitserver performance.
if len(req.Args) == 2 && req.Args[0] == "rev-parse" && req.Args[1] == "HEAD" {
if resolved, err := quickRevParseHead(dir); err == nil && git.IsAbsoluteRevision(resolved) {
tr.LazyPrintf("rev-parse")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The trace is already named exec.rev-parse, what is the reason this log is useful?

Copy link
Contributor Author

@ijt ijt May 16, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I wanted something here to show when quickRevParseHead was being called. Anyway, in the other branch I put a separate span into quickRevParseHead. Adding a span takes about 7 microseconds according to this benchmark, so that's also nothing to worry about:

[ ~/src/github.com/sourcegraph/sourcegraph/bench ] go test -bench=NewTrace
goos: darwin
goarch: amd64
pkg: github.com/sourcegraph/sourcegraph/bench
BenchmarkNewTrace-4   	 1000000	      7239 ns/op
PASS
ok  	github.com/sourcegraph/sourcegraph/bench	7.329s
func BenchmarkNewTrace(b *testing.B) {
	ctx := context.Background()
	for i := 0; i < b.N; i++ {
		tr, _ := trace.New(ctx, "benchmark", "title")
		tr.Finish()
	}
}

@@ -599,7 +599,14 @@ func (s *Server) handleExec(w http.ResponseWriter, r *http.Request) {
// For searches over large repo sets (> 1k), this leads to too many child process execs, which can lead
// to a persistent failure mode where every exec takes > 10s, which is disastrous for gitserver performance.
if len(req.Args) == 2 && req.Args[0] == "rev-parse" && req.Args[1] == "HEAD" {
if resolved, err := quickRevParseHead(dir); err == nil && git.IsAbsoluteRevision(resolved) {
tr.LazyPrintf("rev-parse")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
tr.LazyPrintf("rev-parse")
tr.LazyPrintf("HEAD fast-path")

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All this trace stuff is cross-talk from a different branch. I didn't mean to include it here. Cleaned it up.

if err != nil {
tr.LazyPrintf("quickRevParseHead failed with %v", err)
}
tr.LazyPrintf("is %s an absolute revision? %v", resolved, git.IsAbsoluteRevision(resolved))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am unsure how performant /x/net/trace is and have a slight concern that, given we are talking about these entire operations usually taking ~16ms or so, we may indirectly harm perf with these lazy logs. Thoughts?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The things that are harming performance for us are system calls that take milliseconds to run.

I wrote a couple of benchmarks that show there's nothing to worry about with defer or tr.LazyPrintf:

[ ~/src/github.com/sourcegraph/sourcegraph/bench ] go test -bench=.
goos: darwin
goarch: amd64
pkg: github.com/sourcegraph/sourcegraph/bench
BenchmarkDefer-4        	20000000	       100 ns/op
BenchmarkLazyPrintf-4   	 5000000	       251 ns/op
PASS
ok  	github.com/sourcegraph/sourcegraph/bench	3.903s
// b_test.go 
package main

import (
	"context"
	"testing"

	"github.com/sourcegraph/sourcegraph/pkg/trace"
)

func BenchmarkDefer(b *testing.B) {
	for i := 0; i < b.N; i++ {
		defer func() {
			// Do nothing.
		}()
	}
}

func BenchmarkLazyPrintf(b *testing.B) {
	ctx := context.Background()
	tr, ctx := trace.New(ctx, "benchmark", "title")
	defer tr.Finish()
	for i := 0; i < b.N; i++ {
		tr.LazyPrintf("something something %d", i)
	}
}

@ijt ijt force-pushed the search-perf-issue-take-2 branch from 4dd8d99 to 7cb704a Compare May 15, 2019 22:34
@ijt ijt merged commit 3e65372 into master May 16, 2019
@ijt ijt deleted the search-perf-issue-take-2 branch May 16, 2019 18:22
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants