Skip to content

Commit

Permalink
Browse files Browse the repository at this point in the history
61353: util/tracing,sql: add builtin to set trace spans' verbosity r=irfansharif,erikgrinaker a=angelapwen

Resolves #61180.

Previously there was no way to change a span's verbosity via the
SQL shell. We want to be able to set a specific long-running
span's verbosity on to retrieve its recordings. This patch adds a
builtin, `crdb_internal.set_trace_verbose` that takes in a
trace ID and a bool representing verbose or not
verbose. It sets the verbosity of all spans in this trace.

Note that we would prefer to toggle individual span verbosity,
but this would require a registry of Span objects that
is not added to the 21.1 release. If this Span registry were added
in the future, we could access a Span given its span ID.

Release justification: Adds a crdb_internal tool meant for on-call
engineers, TSEs, etc to debug.

Release note (sql change): Adds a new builtin that sets the verbosity
of all spans in a given trace. Syntax:
crdb_internal.set_trace_verbose($traceID,$verbosityAsBool).

61514: builtins: implement ST_AddMeasure r=otan,rafiss a=andyyang890

This patch implements the geometry builtin `ST_AddMeasure`.

Resolves #60873.

Release justification: low-risk update to new functionality
Release note (sql change): The `ST_AddMeasure` function is now
available for use.

61573: kvnemesis: increase step count during TeamCity nightlies r=nvanbenschoten a=erikgrinaker

The nightly TeamCity stress tests currently run kvnemesis tests for 1
hour. However, each test run only has 50 steps (operations), which
completes in seconds and has limited complexity.

This commit makes the step count configurable via the environment
variable `COCKROACH_KVNEMESIS_STEPS` (default 50), and sets it to
10000 for the nightly TeamCity tests (takes about 3 minutes to run locally).

Resolves #61054.

Release justification: non-production code changes
Release note: None

Co-authored-by: angelapwen <[email protected]>
Co-authored-by: Andy Yang <[email protected]>
Co-authored-by: Erik Grinaker <[email protected]>
  • Loading branch information
4 people committed Mar 5, 2021
4 parents 90d6ba4 + d4f9b02 + f73dae7 + faf7500 commit 15a1856
Show file tree
Hide file tree
Showing 19 changed files with 889 additions and 238 deletions.
4 changes: 4 additions & 0 deletions docs/generated/sql/functions.md
Original file line number Diff line number Diff line change
Expand Up @@ -1271,6 +1271,8 @@ the locality flag on node startup. Returns an error if no region is set.</p>
</span></td></tr>
<tr><td><a name="postgis_wagyu_version"></a><code>postgis_wagyu_version() &rarr; <a href="string.html">string</a></code></td><td><span class="funcdesc"><p>Compatibility placeholder function with PostGIS. Returns a fixed string based on PostGIS 3.0.1, with minor edits.</p>
</span></td></tr>
<tr><td><a name="st_addmeasure"></a><code>st_addmeasure(geometry: geometry, start: <a href="float.html">float</a>, end: <a href="float.html">float</a>) &rarr; geometry</code></td><td><span class="funcdesc"><p>Returns a copy of a LineString or MultiLineString with measure coordinates linearly interpolated between the specified start and end values. Any existing M coordinates will be overwritten.</p>
</span></td></tr>
<tr><td><a name="st_addpoint"></a><code>st_addpoint(line_string: geometry, point: geometry) &rarr; geometry</code></td><td><span class="funcdesc"><p>Adds a Point to the end of a LineString.</p>
</span></td></tr>
<tr><td><a name="st_addpoint"></a><code>st_addpoint(line_string: geometry, point: geometry, index: <a href="int.html">int</a>) &rarr; geometry</code></td><td><span class="funcdesc"><p>Adds a Point to a LineString at the given 0-based index (-1 to append).</p>
Expand Down Expand Up @@ -2706,6 +2708,8 @@ SELECT * FROM crdb_internal.check_consistency(true, ‘\x02’, ‘\x04’)</p>
</span></td></tr>
<tr><td><a name="crdb_internal.round_decimal_values"></a><code>crdb_internal.round_decimal_values(val: <a href="decimal.html">decimal</a>[], scale: <a href="int.html">int</a>) &rarr; <a href="decimal.html">decimal</a>[]</code></td><td><span class="funcdesc"><p>This function is used internally to round decimal array values during mutations.</p>
</span></td></tr>
<tr><td><a name="crdb_internal.set_trace_verbose"></a><code>crdb_internal.set_trace_verbose(trace_id: <a href="int.html">int</a>, verbosity: <a href="bool.html">bool</a>) &rarr; <a href="bool.html">bool</a></code></td><td><span class="funcdesc"><p>Returns true if root span was found and verbosity was set, false otherwise.</p>
</span></td></tr>
<tr><td><a name="crdb_internal.set_vmodule"></a><code>crdb_internal.set_vmodule(vmodule_string: <a href="string.html">string</a>) &rarr; <a href="int.html">int</a></code></td><td><span class="funcdesc"><p>Set the equivalent of the <code>--vmodule</code> flag on the gateway node processing this request; it affords control over the logging verbosity of different files. Example syntax: <code>crdb_internal.set_vmodule('recordio=2,file=1,gfs*=3')</code>. Reset with: <code>crdb_internal.set_vmodule('')</code>. Raising the verbosity can severely affect performance.</p>
</span></td></tr>
<tr><td><a name="crdb_internal.trace_id"></a><code>crdb_internal.trace_id() &rarr; <a href="int.html">int</a></code></td><td><span class="funcdesc"><p>Returns the current trace ID or an error if no trace is open.</p>
Expand Down
11 changes: 6 additions & 5 deletions pkg/cmd/teamcity-trigger/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -81,11 +81,16 @@ func runTC(queueBuild func(string, map[string]string)) {
// halve these values.
parallelism := 4

// Conditionally override stressflags.
opts := map[string]string{
"env.PKG": importPath,
}

// Conditionally override settings.
switch importPath {
case baseImportPath + "kv/kvnemesis":
// Disable -maxruns for kvnemesis. Run for the full 1h.
maxRuns = 0
opts["env.COCKROACH_KVNEMESIS_STEPS"] = "10000"
case baseImportPath + "sql/logictest":
// Stress logic tests with reduced parallelism (to avoid overloading the
// machine, see https://github.com/cockroachdb/cockroach/pull/10966).
Expand All @@ -95,10 +100,6 @@ func runTC(queueBuild func(string, map[string]string)) {
maxTime = 3 * time.Hour
}

opts := map[string]string{
"env.PKG": importPath,
}

opts["env.TESTTIMEOUT"] = testTimeout.String()

// Run non-race build.
Expand Down
2 changes: 2 additions & 0 deletions pkg/cmd/teamcity-trigger/main_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -64,11 +64,13 @@ func Example_runTC() {

// Output:
// github.com/cockroachdb/cockroach/pkg/kv/kvnemesis
// env.COCKROACH_KVNEMESIS_STEPS: 10000
// env.GOFLAGS: -parallel=4
// env.STRESSFLAGS: -maxruns 0 -maxtime 1h0m0s -maxfails 1 -p 4
// env.TESTTIMEOUT: 40m0s
//
// github.com/cockroachdb/cockroach/pkg/kv/kvnemesis
// env.COCKROACH_KVNEMESIS_STEPS: 10000
// env.GOFLAGS: -race -parallel=4
// env.STRESSFLAGS: -maxruns 0 -maxtime 1h0m0s -maxfails 1 -p 1
// env.TESTTIMEOUT: 40m0s
Expand Down
2 changes: 2 additions & 0 deletions pkg/geo/geomfn/BUILD.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ load("@io_bazel_rules_go//go:def.bzl", "go_library", "go_test")
go_library(
name = "geomfn",
srcs = [
"add_measure.go",
"affine_transforms.go",
"angle.go",
"azimuth.go",
Expand Down Expand Up @@ -59,6 +60,7 @@ go_test(
name = "geomfn_test",
size = "small",
srcs = [
"add_measure_test.go",
"affine_transforms_test.go",
"angle_test.go",
"azimuth_test.go",
Expand Down
131 changes: 131 additions & 0 deletions pkg/geo/geomfn/add_measure.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,131 @@
// Copyright 2021 The Cockroach Authors.
//
// Use of this software is governed by the Business Source License
// included in the file licenses/BSL.txt.
//
// As of the Change Date specified in that file, in accordance with
// the Business Source License, use of this software will be governed
// by the Apache License, Version 2.0, included in the file
// licenses/APL.txt.

package geomfn

import (
"github.com/cockroachdb/cockroach/pkg/geo"
"github.com/cockroachdb/errors"
"github.com/twpayne/go-geom"
)

// AddMeasure takes a LineString or MultiLineString and linearly interpolates measure values for each line.
func AddMeasure(geometry geo.Geometry, start float64, end float64) (geo.Geometry, error) {
t, err := geometry.AsGeomT()
if err != nil {
return geometry, err
}

switch t := t.(type) {
case *geom.LineString:
newLineString, err := addMeasureToLineString(t, start, end)
if err != nil {
return geometry, err
}
return geo.MakeGeometryFromGeomT(newLineString)
case *geom.MultiLineString:
newMultiLineString, err := addMeasureToMultiLineString(t, start, end)
if err != nil {
return geometry, err
}
return geo.MakeGeometryFromGeomT(newMultiLineString)
default:
// Ideally we should return NULL here, but following PostGIS on this.
return geometry, errors.Newf("input geometry must be LINESTRING or MULTILINESTRING")
}
}

// addMeasureToMultiLineString takes a MultiLineString and linearly interpolates measure values for each component line.
func addMeasureToMultiLineString(
multiLineString *geom.MultiLineString, start float64, end float64,
) (*geom.MultiLineString, error) {
newMultiLineString :=
geom.NewMultiLineString(augmentLayoutWithM(multiLineString.Layout())).SetSRID(multiLineString.SRID())

// Create a copy of the MultiLineString with measures added to each component LineString.
for i := 0; i < multiLineString.NumLineStrings(); i++ {
newLineString, err := addMeasureToLineString(multiLineString.LineString(i), start, end)
if err != nil {
return multiLineString, err
}
err = newMultiLineString.Push(newLineString)
if err != nil {
return multiLineString, err
}
}

return newMultiLineString, nil
}

// addMeasureToLineString takes a LineString and linearly interpolates measure values.
func addMeasureToLineString(
lineString *geom.LineString, start float64, end float64,
) (*geom.LineString, error) {
newLineString := geom.NewLineString(augmentLayoutWithM(lineString.Layout())).SetSRID(lineString.SRID())

if lineString.Empty() {
return newLineString, nil
}

// Extract the line's current points.
lineCoords := lineString.Coords()

// Compute the length of the line as the sum of the distances between each pair of points.
// Also, fill in pointMeasures with the partial sums.
prevPoint := lineCoords[0]
lineLength := float64(0)
pointMeasures := make([]float64, lineString.NumCoords())
for i := 0; i < lineString.NumCoords(); i++ {
curPoint := lineCoords[i]
distBetweenPoints := coordNorm(coordSub(prevPoint, curPoint))
lineLength += distBetweenPoints
pointMeasures[i] = lineLength
prevPoint = curPoint
}

// Compute the measures for each point.
for i := 0; i < lineString.NumCoords(); i++ {
// Handle special case where line is zero length.
if lineLength == 0 {
pointMeasures[i] = start + (end-start)*(float64(i)/float64(lineString.NumCoords()-1))
} else {
pointMeasures[i] = start + (end-start)*(pointMeasures[i]/lineLength)
}
}

// Replace M value if it exists, otherwise append it to each Coord.
for i := 0; i < lineString.NumCoords(); i++ {
if lineString.Layout().MIndex() == -1 {
lineCoords[i] = append(lineCoords[i], pointMeasures[i])
} else {
lineCoords[i][lineString.Layout().MIndex()] = pointMeasures[i]
}
}

// Create a new LineString with the measures tacked on.
_, err := newLineString.SetCoords(lineCoords)
if err != nil {
return lineString, err
}

return newLineString, nil
}

// augmentLayoutWithM takes a layout and returns a layout with the M dimension added.
func augmentLayoutWithM(layout geom.Layout) geom.Layout {
switch layout {
case geom.XY, geom.XYM:
return geom.XYM
case geom.XYZ, geom.XYZM:
return geom.XYZM
default:
return layout
}
}
Loading

0 comments on commit 15a1856

Please sign in to comment.