-
Notifications
You must be signed in to change notification settings - Fork 3.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
cli: add --{from,to} to tsdump
command
#69491
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewed 1 of 2 files at r1, all commit messages.
Reviewable status: complete! 0 of 0 LGTMs obtained (waiting on @tbg)
pkg/cli/debug.go, line 1569 at r1 (raw file):
f.Var(&debugTimeSeriesDumpOpts.format, "format", "output format (text, csv, tsv, raw)") f.Var(&debugTimeSeriesDumpOpts.from, "from", "oldest timestamp to include") f.Var(&debugTimeSeriesDumpOpts.to, "to", "newest timestamp to include")
might be worth specifying whether they are inclusive or exclusive
pkg/cli/tsdump.go, line 78 at r1 (raw file):
// Stdout does not support Sync in all situations, for example when piping // out from roachprod via `roachprod ssh foo:1 -- [...] tsdump > foo.bar`, // so ignore the error.
Can you explain this a bit more? I am pretty sure that sync is supported in that case, and more generally we want that write errors on the output stream translate into a process error especially when redirecting to a file locally.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewable status: complete! 0 of 0 LGTMs obtained (waiting on @knz and @tbg)
pkg/cli/tsdump.go, line 78 at r1 (raw file):
Previously, knz (kena) wrote…
Can you explain this a bit more? I am pretty sure that sync is supported in that case, and more generally we want that write errors on the output stream translate into a process error especially when redirecting to a file locally.
I don't have much more to explain, look here's a repro:
roachprod create -n 1 $USER-ouch
roachprod stage $USER-ouch release v21.1.7
roachprod start $USER-ouch
roachprod ssh $USER-ouch -- ./cockroach debug tsdump --format=raw --insecure > foo.txt
ERROR: sync /dev/stdout: invalid argument
Failed running "debug tsdump"
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewable status: complete! 0 of 0 LGTMs obtained (waiting on @knz and @tbg)
pkg/cli/tsdump.go, line 78 at r1 (raw file):
Previously, tbg (Tobias Grieger) wrote…
I don't have much more to explain, look here's a repro:
roachprod create -n 1 $USER-ouch roachprod stage $USER-ouch release v21.1.7 roachprod start $USER-ouch roachprod ssh $USER-ouch -- ./cockroach debug tsdump --format=raw --insecure > foo.txt ERROR: sync /dev/stdout: invalid argument Failed running "debug tsdump"
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewable status: complete! 0 of 0 LGTMs obtained (waiting on @tbg)
pkg/cli/tsdump.go, line 78 at r1 (raw file):
Previously, tbg (Tobias Grieger) wrote…
oh my, I had misread this entire time.
We never ever cared for a Sync() here. What we want is a Flush(). That should always work, and if it doesn't that's an error we want to see.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewable status: complete! 0 of 0 LGTMs obtained (waiting on @knz and @tbg)
pkg/cli/tsdump.go, line 78 at r1 (raw file):
Previously, knz (kena) wrote…
oh my, I had misread this entire time.
We never ever cared for a Sync() here. What we want is a Flush(). That should always work, and if it doesn't that's an error we want to see.
os.Stdout
is a *os.File
, which does not have Flush()
.
8019c78
to
640e6b2
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewable status: complete! 0 of 0 LGTMs obtained (waiting on @knz)
pkg/cli/debug.go, line 1569 at r1 (raw file):
Previously, knz (kena) wrote…
might be worth specifying whether they are inclusive or exclusive
Done.
pkg/cli/tsdump.go, line 78 at r1 (raw file):
Previously, tbg (Tobias Grieger) wrote…
os.Stdout
is a*os.File
, which does not haveFlush()
.
Updated the comment.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewed 2 of 2 files at r2, all commit messages.
Reviewable status: complete! 0 of 0 LGTMs obtained (waiting on @tbg)
pkg/cli/tsdump.go, line 78 at r1 (raw file):
Previously, tbg (Tobias Grieger) wrote…
Updated the comment.
I had forgotten that go's os.File are not buffered.
IMHO we can drop the Sync (and the comment) entirely. It does not buy us anything.
Separately, it may be worth adding a bufferedwriter on this. Writing to the file row by row via the CSV writer probably results in terrible write performance. (And then add a Flush and check the error)
This allows restricting which range of datapoints is pulled by `./cockroach debug tsdump` via the `--from` and `--to` flags. This command also touches up the `tsdump` command a little bit. Release justification: low-risk observability change Release note (cli change): The `debug tsdump` command now accepts `--from` and `--to` flags that limit for which dates timeseries are exported.
640e6b2
to
ddfa4db
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewable status: complete! 0 of 0 LGTMs obtained (waiting on @knz)
pkg/cli/tsdump.go, line 78 at r1 (raw file):
Previously, knz (kena) wrote…
I had forgotten that go's os.File are not buffered.
IMHO we can drop the Sync (and the comment) entirely. It does not buy us anything.Separately, it may be worth adding a bufferedwriter on this. Writing to the file row by row via the CSV writer probably results in terrible write performance. (And then add a Flush and check the error)
Added write buffering, PTAL.
bors r=knz |
Build succeeded: |
blathers backport 21.1 |
Encountered an error creating backports. Some common things that can go wrong:
You might need to create your backport manually using the backport tool. error creating merge commit from ddfa4db to blathers/backport-release-21.1-69491: POST https://api.github.com/repos/cockroachdb/cockroach/merges: 409 Merge conflict [] you may need to manually resolve merge conflicts with the backport tool. Backport to branch 21.1 failed. See errors above. 🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is otan. |
Looked at this, there are a few annoying conflicts, so I'm going to abandon the attempt to backport this, since we don't have a concrete use case yet and 21.2 is around the corner. |
This allows restricting which range of datapoints is pulled by
./cockroach debug tsdump
via the--from
and--to
flags.This command also touches up the
tsdump
command a little bit.Release justification: low-risk observability change
Release note (cli change): The
debug tsdump
command now accepts--from
and--to
flags that limit for which dates timeseriesare exported.