x/vuln/cmd/govulncheck: TestCommand fails after golang.org/x dependency update

[dmitshur]

TestCommand is passing at commit golang/vuln@7335627, but failing at commit golang/vuln@d8d123b (CL 555515) with:

$ go test ./cmd/govulncheck
--- FAIL: TestCommand (1.65s)
    --- FAIL: TestCommand//Users/gopher/go/src/golang.org/x/vuln/cmd/govulncheck/testdata/testfiles/source-call/source_vuln_json (1.19s)
        cmdtest.go:336: /Users/gopher/go/src/golang.org/x/vuln/cmd/govulncheck/testdata/testfiles/source-call/source_vuln_json.ct:3: want=-, got=+
            $ govulncheck -C ${moddir}/vuln -json ./...
              []string{
              	... // 461 identical elements
              	`        "version": "v1.6.5",`,
              	`        "package": "github.com/tidwall/gjson"`,
            + 	"      }",
            + 	"    ]",
            + 	"  }",
            + 	"}",
            + 	"{",
            + 	`  "finding": {`,
            + 	`    "osv": "GO-2021-0054",`,
            + 	`    "fixed_version": "v1.6.6",`,
            + 	`    "trace": [`,
            + 	"      {",
            + 	`        "module": "github.com/tidwall/gjson",`,
            + 	`        "version": "v1.6.5",`,
            + 	`        "package": "github.com/tidwall/gjson",`,
            + 	`        "function": "ForEach",`,
            + 	`        "receiver": "Result"`,
            + 	"      },",
[...]
              	... // 163 identical elements
              }
            
            
    --- FAIL: TestCommand//Users/gopher/go/src/golang.org/x/vuln/cmd/govulncheck/testdata/testfiles/source-call/source_vuln_text (2.77s)
        cmdtest.go:336: /Users/gopher/go/src/golang.org/x/vuln/cmd/govulncheck/testdata/testfiles/source-call/source_vuln_text.ct:3: want=-, got=+
            $ govulncheck -C ${moddir}/vuln ./... --> FAIL 3
              []string{
              	... // 22 identical elements
              	"      #1: .../vuln.go:13:16: vuln.main calls language.Parse",
              	"",
            - 	"=== Informational ===",
            + 	"Vulnerability #3: GO-2021-0054",
            + 	"    Due to improper bounds checking, maliciously crafted JSON objects can cause",
            + 	"    an out-of-bounds panic. If parsing user input, this may be used as a denial",
            + 	"    of service vector.",
            + 	"  More info: https://pkg.go.dev/vuln/GO-2021-0054",
            + 	"  Module: github.com/tidwall/gjson",
            + 	"    Found in: github.com/tidwall/[email protected]",
            + 	"    Fixed in: github.com/tidwall/[email protected]",
            + 	"    Example traces found:",
[...]           
            /Users/gopher/go/src/golang.org/x/vuln/cmd/govulncheck/testdata/testfiles/source-call/source_vuln_text.ct:71: want=-, got=+
            $ govulncheck -C ${moddir}/vuln -show=traces ./... --> FAIL 3
[...]

FAIL
FAIL	golang.org/x/vuln/cmd/govulncheck	7.890s
FAIL

(full details are on the post-submit dashboard here)

CL 555515 is a change generated by the automated tagging workflow, that updates to the latest versions of golang.org/x dependencies. This test runs only on longtest builders, and x/vuln trybots only has short builders by default, so it wasn't caught in pre-submit.

A short term fix is to revert the CL, or fix forward. @golang/vulndb What would you prefer?

That said, these kinds of CLs are expected to happen continuously, so there might be more to do to make the test okay with continuous golang.org/x dependency updates. If that's not viable, then let's discuss if something should change on the side of the monthly tagging workflow or the builder configuration. Thanks.

[gopherbot]

Change https://go.dev/cl/555655 mentions this issue: cmd/govulncheck: update test data

[dmitshur]

I think CL 555655 should fix the immediate problem.

[zpavlinovic]

This is fixed now.