Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

*: standardize all sub-benchmarks on key=value format #16830

Merged
merged 2 commits into from
Jul 5, 2017
Merged

*: standardize all sub-benchmarks on key=value format #16830

merged 2 commits into from
Jul 5, 2017

Conversation

tamird
Copy link
Contributor

@tamird tamird commented Jun 30, 2017

No description provided.

@cockroach-teamcity
Copy link
Member

This change is Reviewable

Copy link
Member

@tbg tbg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Skimmed this only.

@tamird
Copy link
Contributor Author

tamird commented Jul 1, 2017

I'd like to get everyone's "ack" on this, at least, since new benchmarks should be written in this style as well.

@petermattis
Copy link
Collaborator

:lgtm:

Might want to drop a page on the github wiki explaining (briefly) why this is useful/important.


Review status: 0 of 9 files reviewed at latest revision, 1 unresolved discussion, all commit checks successful.


pkg/sql/distsqlrun/sorter_test.go, line 245 at r1 (raw file):

	for _, inputSize := range []int{0, 1 << 2, 1 << 4, 1 << 8, 1 << 12, 1 << 16} {
		b.Run(fmt.Sprintf("InputSize=%d", inputSize), func(b *testing.B) {

Moving the b.Run call might affect the timing. I imagine the timer only starts when b.Run is called. I think I'd revert this part of the change unless you had a strong reason for doing so.

PS We're not consistent in capitalization for these "keys". Does that matter?


Comments from Reviewable

@tamird
Copy link
Contributor Author

tamird commented Jul 2, 2017

Review status: 0 of 9 files reviewed at latest revision, 1 unresolved discussion, all commit checks successful.


pkg/sql/distsqlrun/sorter_test.go, line 245 at r1 (raw file):

Previously, petermattis (Peter Mattis) wrote…

Moving the b.Run call might affect the timing. I imagine the timer only starts when b.Run is called. I think I'd revert this part of the change unless you had a strong reason for doing so.

PS We're not consistent in capitalization for these "keys". Does that matter?

Ah, I moved the call to b.Run so that failures here would only tank the sub-bench rather than the top-level one.

Looking at this more, are we intentionally measuring the time taken by rowSource.Reset? @arjunravinarayan @asubiotto


Comments from Reviewable

@petermattis
Copy link
Collaborator

Review status: 0 of 9 files reviewed at latest revision, 1 unresolved discussion, all commit checks successful.


pkg/sql/distsqlrun/sorter_test.go, line 245 at r1 (raw file):

Previously, tamird (Tamir Duberstein) wrote…

Ah, I moved the call to b.Run so that failures here would only tank the sub-bench rather than the top-level one.

Looking at this more, are we intentionally measuring the time taken by rowSource.Reset? @arjunravinarayan @asubiotto

rowSource.Reset is trivial (it sets an index). I doubt the b.Fatal ever fired. I'd move the placement of b.Run back in this PR and let @arjunravinarayan and @asubiotto sort it out in a follow-on.


Comments from Reviewable

@tamird
Copy link
Contributor Author

tamird commented Jul 3, 2017

Review status: 0 of 9 files reviewed at latest revision, 1 unresolved discussion.


pkg/sql/distsqlrun/sorter_test.go, line 245 at r1 (raw file):

Previously, petermattis (Peter Mattis) wrote…

rowSource.Reset is trivial (it sets an index). I doubt the b.Fatal ever fired. I'd move the placement of b.Run back in this PR and let @arjunravinarayan and @asubiotto sort it out in a follow-on.

I went with a more explicit call to b.ResetTimer(); PTAL.


Comments from Reviewable

@petermattis
Copy link
Collaborator

Review status: 0 of 9 files reviewed at latest revision, 1 unresolved discussion, all commit checks successful.


pkg/sql/distsqlrun/sorter_test.go, line 245 at r1 (raw file):

We're not consistent in capitalization for these "keys". Does that matter?

Ping.


Comments from Reviewable

@tamird
Copy link
Contributor Author

tamird commented Jul 5, 2017 via email

@rjnn
Copy link
Contributor

rjnn commented Jul 5, 2017

Skimmed, new convention looks fine. :lgtm:
Did not look closely.


Reviewed 1 of 9 files at r2, 3 of 9 files at r3.
Review status: 3 of 9 files reviewed at latest revision, 1 unresolved discussion, all commit checks successful.


pkg/sql/distsqlrun/sorter_test.go, line 245 at r1 (raw file):

Previously, petermattis (Peter Mattis) wrote…

We're not consistent in capitalization for these "keys". Does that matter?

Ping.

I'll let @asubiotto authoritatively answer this.


Comments from Reviewable

@asubiotto
Copy link
Contributor

:lgtm: Might be nice to make capitalization of the keys consistent but I have no strong feelings about it.


Review status: 3 of 9 files reviewed at latest revision, 1 unresolved discussion, all commit checks successful.


pkg/sql/distsqlrun/sorter_test.go, line 245 at r1 (raw file):

Previously, arjunravinarayan (Arjun Narayan) wrote…

I'll let @asubiotto authoritatively answer this.

This call to ResetTimer is fine. I don't think there's a need to have this change in a separate PR.


Comments from Reviewable

@tamird tamird merged commit 8ebb3ec into cockroachdb:master Jul 5, 2017
@tamird tamird deleted the bench-numbers branch July 5, 2017 14:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants