Skip to content

Commit

Permalink
Merge #28072 #28129
Browse files Browse the repository at this point in the history
28072: rangefeed: create new rangefeed package r=nvanbenschoten a=nvanbenschoten

This is the first "mergable" part of #27285.

The change introduces a new rangefeed package, which exports a `Processor`
that manages a set of rangefeed registrations and handles the routing of
logical updates to these registrations. While routing logical updates to
rangefeed registrations, the processor performs two important tasks:
1. it translates logical updates into rangefeed events. These logical
   updates are provided to the Processor through its `ConsumeLogicalOps`
   method.
2. it transforms a range-level closed timestamp to a rangefeed-level resolved
   timestamp. Closed timestamp updates are passed to the Processor through
   its `ForwardClosedTS` method.

In order to perform its second task, a `Processor` hold on to a `resolvedTimestamp`
object. A rangefeed's "resolved timestamp" is defined as the timestamp at
which no future updates will be emitted to the feed at or before. The timestamp
is monotonically increasing and is communicated through RangeFeedCheckpoint
notifications whenever it changes. The `resolvedTimestamp` itself holds on
to a `unresolvedIntentQueue` (described in #26782) which allows it to efficiently
track unresolved intents that it observes in logical updates provided to the
Processor.

There are two major pieces of this package missing. The first is catch up
scans for new registrations and the second is a mechanism to push intents
that are old enough to be holding back the resolved timestamp.

The change doesn't plug anything in yet, but a rough draft of the plumbing
can be seen in #27285.

Release note: None

28129: sql: disambiguate float,decimal formatting r=jordanlewis a=jordanlewis

Previously, the FmtParsable directive wouldn't produce a parsable
representation for some floats and decimals, in the context of other
expressions. In particular, negative floats and decimals would format
like `-10.0:::FLOAT`. This seems fine, but since the `:::` operator
binds more closely than the `-` operator, an expression like
`(-10.0:::FLOAT)::STRING` would get formatted to
`-10.0:::FLOAT::STRING`, which is of course invalid as there's no unary
negative operator for the string datatype.

Now, floats and decimals are marked as unambiguously formatted datatypes
and take care of disambiguating themselves with a type annotation if
necessary, as well as further-disambiguating parentheses if the numbers
are nuegative.

Release note (bug fix): correct the round-trip formatting of negative
floats and decimals in the context of other expressions when executing
in a distributed flow.

Co-authored-by: Nathan VanBenschoten <[email protected]>
Co-authored-by: Jordan Lewis <[email protected]>
  • Loading branch information
3 people committed Aug 1, 2018
3 parents 5195e36 + 1f91fa7 + 22cbf6b commit 6e2bcfe
Show file tree
Hide file tree
Showing 20 changed files with 8,779 additions and 938 deletions.
1,800 changes: 1,785 additions & 15 deletions c-deps/libroach/protos/storage/engine/enginepb/mvcc3.pb.cc

Large diffs are not rendered by default.

2,713 changes: 2,256 additions & 457 deletions c-deps/libroach/protos/storage/engine/enginepb/mvcc3.pb.h

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion pkg/cli/dump.go
Original file line number Diff line number Diff line change
Expand Up @@ -536,7 +536,7 @@ func dumpTableData(w io.Writer, conn *sqlConn, clusterTS string, bmd basicMetada
g.GoCtx(func(ctx context.Context) error {
// Convert SQL rows into VALUE strings.
defer close(stringsCh)
f := tree.NewFmtCtxWithBuf(tree.FmtParsable)
f := tree.NewFmtCtxWithBuf(tree.FmtParsableNumerics)
defer f.Close()
for vals := range valsCh {
f.Reset()
Expand Down
2 changes: 1 addition & 1 deletion pkg/cli/testdata/dump/row
Original file line number Diff line number Diff line change
Expand Up @@ -84,7 +84,7 @@ CREATE TABLE t (
);

INSERT INTO t (i, f, s, b, d, t, ts, n, o, e, u, ip, j, ary, tz, e1, e2, s1) VALUES
(1, 2.3, 'striiing', '\x613162326333', '2016-03-26', '01:02:03.456', '2016-01-25 10:10:10+00:00', '2h30m30s', true, 1.2345, 'e9716c74-2638-443d-90ed-ffde7bea7d1d', '192.168.0.1', '{"a": "b"}', ARRAY['hello':::STRING,'world':::STRING], '2016-01-25 10:10:10+00:00', 3, 4.5, 's'),
(1, 2.3, 'striiing', '\x613162326333', '2016-03-26', '01:02:03.456', '2016-01-25 10:10:10+00:00', '2h30m30s', true, 1.2345, 'e9716c74-2638-443d-90ed-ffde7bea7d1d', '192.168.0.1', '{"a": "b"}', ARRAY['hello','world'], '2016-01-25 10:10:10+00:00', 3, 4.5, 's'),
(NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL),
(NULL, '+Inf', NULL, NULL, NULL, NULL, NULL, NULL, NULL, 'Infinity', NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL),
(NULL, '-Inf', NULL, NULL, NULL, NULL, NULL, NULL, NULL, '-Infinity', NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL),
Expand Down
Loading

0 comments on commit 6e2bcfe

Please sign in to comment.