more descriptive test failure issues #34
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
the issues will now have a list of failed tests in the subject. each failing test has a bright red label created and assigned to its issues.
tbg
added a commit
that referenced
this pull request
Aug 31, 2016
We have been seeing long startup times which disappear spontaneously. During a
restart of the beta cluster, the following goroutine was observed, which suggests
that we were spending a lot of time GCing replicas on startup.
engine ??:0 _Cfunc_DBIterNext(cockroachdb#324, cockroachdb#323, 0, 0, 0, 0, 0, 0, 0)
engine rocksdb.go:1135 (*rocksDBIterator).Next(cockroachdb#235)
storage replica_data_iter.go:104 (*ReplicaDataIterator).Next(cockroachdb#316)
storage store.go:1748 (*Store).destroyReplicaData(#109, cockroachdb#317, 0, 0)
storage store.go:841 (*Store).Start.func2(0x101b, cockroachdb#300, 0x36, 0x40, cockroachdb#301, 0x36, 0x40, cockroachdb#315, 0x3, 0x4, ...)
storage store.go:734 IterateRangeDescriptors.func1(cockroachdb#306, 0x40, 0x41, cockroachdb#307, 0x92, 0x92, cockroachdb#341, 0, 0x186c, 0x4000, ...)
engine mvcc.go:1593 MVCCIterate(cockroachdb#329, #68, #47, #81, cockroachdb#232, 0x9, 0x10, cockroachdb#233, 0xb, 0x10, ...)
storage store.go:738 IterateRangeDescriptors(cockroachdb#330, cockroachdb#196, #47, #81, cockroachdb#195, #179, #110)
storage store.go:867 (*Store).Start(#109, cockroachdb#330, cockroachdb#196, #179, cockroachdb#185, 0x1)
server node.go:405 (*Node).initStores(#78, cockroachdb#330, cockroachdb#196, #98, 0x1, 0x1, #179, 0, #55)
server node.go:330 (*Node).start(#78, cockroachdb#330, cockroachdb#196, #42, #129, #98, 0x1, 0x1, 0, 0, ...)
server server.go:431 (*Server).Start(#5, cockroachdb#330, cockroachdb#196, #95, 0x1)
cli start.go:368 runStart(#34, #178, 0, 0x9, 0, 0)
cobra command.go:599 (*Command).execute(#34, #177, 0x9, 0x9, #34, #177)
cobra command.go:689 (*Command).ExecuteC(#33, #70, cockroachdb#343, #72)
cobra command.go:648 (*Command).Execute(#33, #71, cockroachdb#343)
cli cli.go:96 Run(#64, 0xa, 0xa, 0, 0)
main main.go:37 main()
tbg
added a commit
that referenced
this pull request
Aug 31, 2016
We have been seeing long startup times which disappear spontaneously. During a
restart of the beta cluster, the following goroutine was observed, which suggests
that we were spending a lot of time GCing replicas on startup.
engine ??:0 _Cfunc_DBIterNext(cockroachdb#324, cockroachdb#323, 0, 0, 0, 0, 0, 0, 0)
engine rocksdb.go:1135 (*rocksDBIterator).Next(cockroachdb#235)
storage replica_data_iter.go:104 (*ReplicaDataIterator).Next(cockroachdb#316)
storage store.go:1748 (*Store).destroyReplicaData(#109, cockroachdb#317, 0, 0)
storage store.go:841 (*Store).Start.func2(0x101b, cockroachdb#300, 0x36, 0x40, cockroachdb#301, 0x36, 0x40, cockroachdb#315, 0x3, 0x4, ...)
storage store.go:734 IterateRangeDescriptors.func1(cockroachdb#306, 0x40, 0x41, cockroachdb#307, 0x92, 0x92, cockroachdb#341, 0, 0x186c, 0x4000, ...)
engine mvcc.go:1593 MVCCIterate(cockroachdb#329, #68, #47, #81, cockroachdb#232, 0x9, 0x10, cockroachdb#233, 0xb, 0x10, ...)
storage store.go:738 IterateRangeDescriptors(cockroachdb#330, cockroachdb#196, #47, #81, cockroachdb#195, #179, #110)
storage store.go:867 (*Store).Start(#109, cockroachdb#330, cockroachdb#196, #179, cockroachdb#185, 0x1)
server node.go:405 (*Node).initStores(#78, cockroachdb#330, cockroachdb#196, #98, 0x1, 0x1, #179, 0, #55)
server node.go:330 (*Node).start(#78, cockroachdb#330, cockroachdb#196, #42, #129, #98, 0x1, 0x1, 0, 0, ...)
server server.go:431 (*Server).Start(#5, cockroachdb#330, cockroachdb#196, #95, 0x1)
cli start.go:368 runStart(#34, #178, 0, 0x9, 0, 0)
cobra command.go:599 (*Command).execute(#34, #177, 0x9, 0x9, #34, #177)
cobra command.go:689 (*Command).ExecuteC(#33, #70, cockroachdb#343, #72)
cobra command.go:648 (*Command).Execute(#33, #71, cockroachdb#343)
cli cli.go:96 Run(#64, 0xa, 0xa, 0, 0)
main main.go:37 main()
tbg
pushed a commit
that referenced
this pull request
May 17, 2017
**tl;dr** use `\set display_format raw` before a query to get the strings unmodified
Background
----------
One of the inconveniences of SHOW CREATE TABLE is that its output, by
default, encloses the generated CREATE TABLE statement in an ASCII art
table, like this:
````
root@:26257/> SHOW CREATE TABLE t.kv;
+-------+------------------------------------------------+
| Table | CreateTable |
+-------+------------------------------------------------+
| t.kv | CREATE TABLE kv ( |
| | k INT NOT NULL, |
| | v INT NULL, |
| | CONSTRAINT "primary" PRIMARY KEY (k ASC), |
| | FAMILY "primary" (k, v) |
| | ) |
+-------+------------------------------------------------+
(1 row)
```
This prevents copy-pasting the output directly onto some other
shell or text file.
Meanwhile, our SQL CLI shell *does* provide alternative ways to print
out the results, that can be selected with `\set display_format ...`;
however, none of the formats defined so far are able to print out the
above CREATE TABLE statement **in a way that can be copy-pasted
elsewhere**; for example:
```
root@:26257/> \set display_format html
root@:26257/> SHOW CREATE TABLE t.kv;
<table>
<thead><tr><th>Table</th><th>CreateTable</th></tr></head>
<tbody>
<tr><td>t.kv</td><td>CREATE TABLE kv (<br/> k INT NOT NULL,<br/> v INT NULL,<br/>
CONSTRAINT "primary" PRIMARY KEY (k ASC),<br/> FAMILY "primary" (k, v)<br/>)</td></tr>
</tbody>
</table>
```
(This replaces all special characters)
```
root@:26257/> \set display_format csv
root@:26257/> SHOW CREATE TABLE t.kv;
1 row
Table,CreateTable
t.kv,"CREATE TABLE kv (
k INT NOT NULL,
v INT NULL,
CONSTRAINT ""primary"" PRIMARY KEY (k ASC),
FAMILY ""primary"" (k, v)
)"
```
(This escapes the double quotes by doubling them, which renders the SQL invalid)
And so forth.
Solution: new display format
----------------------------
The `cockroach sql` CLI shell (as well as any built-in command that
emits tables) can use a custom display format configurable either with
`--format` on the command-line, or `\set display_format` in the shell.
This patch adds a new format `raw` which causes the string
representation of every record to be printed as-is, starting on its
own line, so that it can be copy-pasted from a terminal.
This format makes an extra mile to ensure the data can be further
processed: each record is prefixed by an annotation of the form `# N`
followed by a newline character, where *N* is the number of characters
in the record, starting with the first character following the
newline. A subsequent automated processing tool can use this
information to determine record boundaries.
Application
-----------
```
root@:26257/> select createtable from [show create table t.kv];
...
CREATE TABLE kv (
k INT NOT NULL,
v INT NULL,
CONSTRAINT "primary" PRIMARY KEY (k ASC),
FAMILY "primary" (k, v)
)
...
```
Tada!
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
the issues will now have a list of failed tests in the subject.
each failing test has a bright red label created and assigned to
its issues.