Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cli: add timestamp as tag in tsdump datadog upload #135158

Merged

Conversation

aa-joshi
Copy link
Contributor

Previously, we updated upload_id identifier to include timestamp as part of based on uploadID. This change adds upload year, month & day tags as part of tsdump upload.

Epic: None
Part of: CRDB-44379
Release note: None

@aa-joshi aa-joshi requested review from a team as code owners November 14, 2024 10:20
@aa-joshi aa-joshi requested review from angles-n-daemons and arjunmahishi and removed request for a team November 14, 2024 10:20
@cockroach-teamcity
Copy link
Member

This change is Reviewable

@aa-joshi aa-joshi force-pushed the add_timestamp_as_tag_in_tsdump_upload branch from 223bec0 to c02cd3b Compare November 14, 2024 10:30
Copy link
Contributor

@arjunmahishi arjunmahishi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @aa-joshi and @angles-n-daemons)


pkg/cli/tsdump_upload.go line 95 at r1 (raw file):

	if debugTimeSeriesDumpOpts.clusterLabel != "" {
		clusterTagValue = debugTimeSeriesDumpOpts.clusterLabel
	} else if serverCfg.ClusterName != "" {

Will this ever be true? I think this is generating a "default cluster settings" object. I traced back the code. I don't think ClusterName is being set anywhere. I could be wrong. But worth verifying once to trim down this function.


pkg/cli/tsdump_upload.go line 98 at r1 (raw file):

		clusterTagValue = serverCfg.ClusterName
	} else {
		clusterTagValue = fmt.Sprintf("cluster-debug-%d", timeutil.Now().Unix())

The timestamp will anyway be appended in the newUploadID function right? We should skip the -%d time in this else case


pkg/cli/tsdump_upload.go line 99 at r1 (raw file):

	} else {
		clusterTagValue = fmt.Sprintf("cluster-debug-%d", timeutil.Now().Unix())
	}

NIT: We can shorten the if-else ladder if we handle the default cluster name first.

Suggestion:

	clusterTagValue := fmt.Sprintf("cluster-debug-%d", timeutil.Now().Unix())
	if debugTimeSeriesDumpOpts.clusterLabel != "" {
		clusterTagValue = debugTimeSeriesDumpOpts.clusterLabel
	} else if serverCfg.ClusterName != "" {
		clusterTagValue = serverCfg.ClusterName
	}

pkg/cli/tsdump_upload.go line 225 at r1 (raw file):

	tags = append(tags, makeDDTag(uploadIDTag, d.uploadID))

	year, month, day := timeutil.Now().Date()

Should this be using d.uploadTime instead? (for consistence)


pkg/cli/tsdump_test.go line 192 at r1 (raw file):

	defer leaktest.AfterTest(t)()
	defer log.Scope(t).Close(t)
	defer testutils.TestingHook(&newUploadID, func(cluster string, uploadTime time.Time) string {

I don't think we need this hook anymore. newUploadID is a pure function now. The output is determined by getCurrentTime. Since that is already taken care of, I think we can skip this hook. I think we can make the function a non-variable also now.


pkg/cli/zip_upload_test.go line 111 at r1 (raw file):

	defer leaktest.AfterTest(t)()
	defer log.Scope(t).Close(t)
	defer testutils.TestingHook(&newUploadID, func(string, time.Time) string {

Same as above. We can skip this change in this PR if you want.

@aa-joshi aa-joshi force-pushed the add_timestamp_as_tag_in_tsdump_upload branch from c02cd3b to 4b908f1 Compare November 15, 2024 10:50
Copy link
Contributor Author

@aa-joshi aa-joshi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @angles-n-daemons and @arjunmahishi)


pkg/cli/tsdump_upload.go line 95 at r1 (raw file):

Previously, arjunmahishi (Arjun Mahishi) wrote…

Will this ever be true? I think this is generating a "default cluster settings" object. I traced back the code. I don't think ClusterName is being set anywhere. I could be wrong. But worth verifying once to trim down this function.

Done.


pkg/cli/tsdump_upload.go line 98 at r1 (raw file):

Previously, arjunmahishi (Arjun Mahishi) wrote…

The timestamp will anyway be appended in the newUploadID function right? We should skip the -%d time in this else case

Done.


pkg/cli/tsdump_upload.go line 225 at r1 (raw file):

Previously, arjunmahishi (Arjun Mahishi) wrote…

Should this be using d.uploadTime instead? (for consistence)

Thanks for Pointing it out!. Done


pkg/cli/tsdump_test.go line 192 at r1 (raw file):

Previously, arjunmahishi (Arjun Mahishi) wrote…

I don't think we need this hook anymore. newUploadID is a pure function now. The output is determined by getCurrentTime. Since that is already taken care of, I think we can skip this hook. I think we can make the function a non-variable also now.

Done.


pkg/cli/zip_upload_test.go line 111 at r1 (raw file):

Previously, arjunmahishi (Arjun Mahishi) wrote…

Same as above. We can skip this change in this PR if you want.

Here it is needed because we are calling timeutil.Now() in method invocation

Copy link
Contributor

@arjunmahishi arjunmahishi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm:

Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained (waiting on @angles-n-daemons)

Previously, we updated `upload_id` to include timestamp as part of cockroachdb#131340.
This was inadequate because it is hard to filter out uploads based on uploadID.
This change adds upload year, month & day tags as part of tsdump upload.

Epic: None
Part of: CRDB-44379
Release note: None
@aa-joshi aa-joshi force-pushed the add_timestamp_as_tag_in_tsdump_upload branch from 4b908f1 to 0d5a5a9 Compare November 18, 2024 05:47
@aa-joshi
Copy link
Contributor Author

TFTR!

bors r+

@craig craig bot merged commit 3aaab0a into cockroachdb:master Nov 18, 2024
23 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants