Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bulkio: use correct context in processor initialization #60958

Merged
merged 2 commits into from
Feb 23, 2021

Conversation

pbardea
Copy link
Contributor

@pbardea pbardea commented Feb 22, 2021

This commit ensures that producer goroutines created during the
initialization of bulk processors use the same context as bp.Ctx.

This does change the context that these goroutines were using,
but that shouldn't actually have any effects on the processors, other
than missing tracing capabilities.

Closes #60940.

Closes #60984
Closes #60983
Closes #60977
Closes #60990
Closes #60989
Closes #60979
Closes #60987
Closes #60986
Closes #60985
Closes #60981
Closes #60980
Closes #60978
Closes #60978
Closes #60976
Closes #60975

Release note: None

@pbardea pbardea requested review from yuzefovich, adityamaru and a team February 22, 2021 22:33
@cockroach-teamcity
Copy link
Member

This change is Reviewable

@@ -176,14 +172,14 @@ func (sip *streamIngestionProcessor) Start(ctx context.Context) {
startTime := timeutil.Unix(0 /* sec */, sip.spec.StartTime.WallTime)
eventChs := make(map[streamingccl.PartitionAddress]chan streamingccl.Event)
for _, partitionAddress := range sip.spec.PartitionAddresses {
eventCh, err := sip.client.ConsumePartition(sip.Ctx, partitionAddress, startTime)
eventCh, err := sip.client.ConsumePartition(ctx, partitionAddress, startTime)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@yuzefovich Just wanted to check if there was a convention to use ctx here rather than sip.Ctx?

Copy link
Member

@yuzefovich yuzefovich left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 1 of 4 files at r1.
Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @adityamaru and @pbardea)


pkg/ccl/streamingccl/streamingest/stream_ingestion_processor.go, line 175 at r1 (raw file):

Previously, pbardea (Paul Bardea) wrote…

@yuzefovich Just wanted to check if there was a convention to use ctx here rather than sip.Ctx?

I think we tend to use ctx in most places (the only exceptions are sampler and sample_aggregator, I think), so this change LGTM.

@ajstorm
Copy link
Collaborator

ajstorm commented Feb 23, 2021

Just a heads-up that #60928 is also blocked on this issue. You can see the test case failure it's getting here.

This commit ensures that producer goroutines created during the
initialization of bulk processors use the same context as bp.Ctx.

Release note: None
@pbardea
Copy link
Contributor Author

pbardea commented Feb 23, 2021

Ack, this has been hitting a lot of flakes. Rebased in case any tests have been updated on master/flakes addressed recently.

@pbardea
Copy link
Contributor Author

pbardea commented Feb 23, 2021

@pbardea
Copy link
Contributor Author

pbardea commented Feb 23, 2021

TFTRs, merging with high priority to address the flakes
bors r=adityamaru p=9999 single on

@craig
Copy link
Contributor

craig bot commented Feb 23, 2021

Build failed:

@pbardea
Copy link
Contributor Author

pbardea commented Feb 23, 2021

Hit a couple of flakes
bors r=adityamaru p=9999 single on

@craig
Copy link
Contributor

craig bot commented Feb 23, 2021

Build failed:

Skipping on master while flake is resolved.

Release note: None
@pbardea
Copy link
Contributor Author

pbardea commented Feb 23, 2021

@ajstorm this flaked on #60773 in https://teamcity.cockroachdb.com/buildConfiguration/Cockroach_UnitTests/2709929?. Unsure what's happening, but I added a commit that skips the entire test. We've been seeing some imports hand due to KV changes, which may be related to these restores hanging.

Could I get a stamp for the second commit?

@pbardea
Copy link
Contributor Author

pbardea commented Feb 23, 2021

bors r=adityamaru

@craig
Copy link
Contributor

craig bot commented Feb 23, 2021

Build failed:

@pbardea
Copy link
Contributor Author

pbardea commented Feb 23, 2021

Got a nil pointer deref on
TestReplicateRemovedNodeDisruptiveElection in ^

@pbardea
Copy link
Contributor Author

pbardea commented Feb 23, 2021

bors r=adityamaru

@craig
Copy link
Contributor

craig bot commented Feb 23, 2021

Build succeeded:

@craig craig bot merged commit 0edd7e8 into cockroachdb:master Feb 23, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment