Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

changefeedccl: Fix flaky tests. #65867

Merged
merged 1 commit into from
Jun 1, 2021
Merged

Conversation

miretskiy
Copy link
Contributor

@miretskiy miretskiy commented May 28, 2021

Fix flaky test and re-enable it to run under stress.
The problem was that the transaction executed by the table feed can
be restarted. If that happens, then we would see the same keys again,
but because we had side effects inside transaction (marking the keys
seen), we would not emit those keys causing the test to be hung.
The stress race was failing because of both transaction restarts and
the 10ms resolved timestamp frequency (with so many resolved timestamps
being generated, the table feed transaction was always getting
restarted).

Fixes #57754
Fixes #65168

Release Notes: None

@miretskiy miretskiy requested review from stevendanna and HonoreDB May 28, 2021 21:47
@cockroach-teamcity
Copy link
Member

This change is Reviewable

Copy link
Contributor

@HonoreDB HonoreDB left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 2 of 2 files at r1.
Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @stevendanna)

@miretskiy miretskiy force-pushed the frontier branch 4 times, most recently from 9ca981a to 2452788 Compare May 30, 2021 16:02
Fix flaky test and re-enable it to run under stress.
The problem was that the transaction executed by the table feed can
be restarted.  If that happens, then we would see the same keys again,
but because we had side effects inside transaction (marking the keys
seen), we would not emit those keys causing the test to be hung.
The stress race was failing because of both transaction restarts and
the 10ms resolved timestamp frequency (with so many resolved timestamps
being generated, the table feed transaction was always getting
restarted).

Fixes cockroachdb#57754
Fixes cockroachdb#65168

Release Notes: None
@miretskiy
Copy link
Contributor Author

@HonoreDB fyi: I removed changed to the resumer -- keeping only changes to make test non-flaky.
Going to address resumer changes later (if needed) since it's quite a bit more involved than I thought initially.

@miretskiy
Copy link
Contributor Author

bors r+

@craig craig bot merged commit f5b21cf into cockroachdb:master Jun 1, 2021
@craig
Copy link
Contributor

craig bot commented Jun 1, 2021

Build succeeded:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

unknown: (unknown) failed changefeedccl: skipped TestChangefeedInitialScan under race because of timeout
3 participants