Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix the requeue logic #230

Merged
merged 1 commit into from
May 21, 2020
Merged

Fix the requeue logic #230

merged 1 commit into from
May 21, 2020

Conversation

xing-yang
Copy link
Collaborator

@xing-yang xing-yang commented Jan 2, 2020 •
edited
Loading

What type of PR is this?
/kind bug

What this PR does / why we need it:
Add requeue to the sidecar controller.

Which issue(s) this PR fixes:
Part of #225

Special notes for your reviewer:

Does this PR introduce a user-facing change?:

This fixes the re-queue logic so a failed snapshot operation will be added back to a rate limited queue for retries.

@k8s-ci-robot k8s-ci-robot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. release-note-none Denotes a PR that doesn't merit a release note. kind/bug Categorizes issue or PR as related to a bug. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels Jan 2, 2020
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: xing-yang

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added approved Indicates a PR has been approved by an approver from all required OWNERS files. size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Jan 2, 2020
@xing-yang xing-yang force-pushed the requeue_sidecar branch 3 times, most recently from 3145f7a to 67b5fcb Compare January 3, 2020 03:00
@k8s-ci-robot k8s-ci-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Jan 3, 2020
@xing-yang xing-yang changed the title WIP: Add requeue in sidecar WIP: Do Not Merge Jan 3, 2020
@xing-yang xing-yang changed the title WIP: Do Not Merge WIP: Fix the requeue logic Mar 31, 2020
@xing-yang xing-yang mentioned this pull request Mar 31, 2020
Merged
@xing-yang xing-yang mentioned this pull request Apr 29, 2020
Merged
@xing-yang xing-yang force-pushed the requeue_sidecar branch 2 times, most recently from 1e5bed9 to 7a5aafa Compare May 9, 2020 20:32
@xing-yang xing-yang changed the title WIP: Fix the requeue logic Fix the requeue logic May 10, 2020
@k8s-ci-robot k8s-ci-robot added release-note Denotes a PR that will be considered when it comes time to generate release notes. and removed do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. release-note-none Denotes a PR that doesn't merit a release note. labels May 10, 2020
@humblec
Copy link
Contributor

humblec commented May 15, 2020 •
edited
Loading

unless I missed something, the requeue logic ( forgetting on success and immediate retry on failure) here looks good to me. However we will wait for @bswartz as the issue is reproducible in the setup and can have some real validation. considering this is on cache and sync code path, its better to get validated.

@bswartz
Copy link
Contributor

bswartz commented May 16, 2020

/assign

@bswartz
Copy link
Contributor

bswartz commented May 17, 2020

I downloaded this patch and ran it in my torture tests and I don't see a single slow snapshot anymore. Unfortunately my torture test eventually crashed after 35 snapshots due to bugs in other components, but I am confident that this patch solves my problem.

Thanks @xing-yang !

@xing-yang
Copy link
Collaborator Author

Thanks @bswartz for testing this!

@bswartz
Copy link
Contributor

bswartz commented May 18, 2020

Okay I fixed the bug in my torture test, and re-ran it with this patch. I got up to 278 snapshots with no hangs and no crashes, before I decided to end the test.

@xing-yang
Copy link
Collaborator Author

That's awesome. Thanks @bswartz!

@xing-yang
Copy link
Collaborator Author

/assign @yuxiangqian

@xing-yang xing-yang mentioned this pull request May 20, 2020
Closed
@humblec
Copy link
Contributor

humblec commented May 21, 2020

/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label May 21, 2020
@k8s-ci-robot k8s-ci-robot merged commit 82ef6e6 into kubernetes-csi:master May 21, 2020
mowangdk added a commit to mowangdk/external-snapshotter that referenced this pull request Aug 6, 2023
@mowangdk
Squashed 'release-tools/' changes from 5489de6..670bb0e
06fd1a6 
670bb0e Merge pull request kubernetes-csi#229 from marosset/fix-codespell-errors
35d5e78 Merge pull request kubernetes-csi#219 from yashsingh74/update-registry
63473cc Merge pull request kubernetes-csi#231 from coulof/bump-go-version-1.20.5
29a5c76 Merge pull request kubernetes-csi#228 from mowangdk/chore/adopt_kubernetes_recommand_labels
8dd2821 Update cloudbuild image with go 1.20.5
1df23db Merge pull request kubernetes-csi#230 from msau42/prow
1f92b7e Add ginkgo timeout to e2e tests to help catch any stuck tests
2b8b80e fixing some codespell errors
c10b678 Merge pull request kubernetes-csi#227 from coulof/check-sidecar-supported-versions
72984ec chore: adopt kubernetes recommand label
b055535 Header
bd0a10b typo
c39d73c Add comments
f6491af Script to verify EOL sidecar version
4133d1d Merge pull request kubernetes-csi#226 from msau42/cloudbuild
8d519d2 Pin buildkit to v0.10.6 to workaround v0.11 bug with docker manifest
6e04a03 Merge pull request kubernetes-csi#224 from msau42/cloudbuild
26fdfff Update cloudbuild image
6613c39 Merge pull request kubernetes-csi#223 from sunnylovestiramisu/update
0e7ae99 Update k8s image repo url
77e47cc Merge pull request kubernetes-csi#222 from xinydev/fix-dep-version
155854b Fix dep version mismatch
8f83905 Merge pull request kubernetes-csi#221 from sunnylovestiramisu/go-update
1d3f94d Update go version to 1.20 to match k/k v1.27
e322ce5 Merge pull request kubernetes-csi#220 from andyzhangx/fix-golint-error
b74a512 test: fix golint error
901bcb5 Update registry k8s.gcr.io -> registry.k8s.io
aa61bfd Merge pull request kubernetes-csi#218 from xing-yang/update_csi_driver
7563d19 Update CSI_PROW_DRIVER_VERSION to v1.11.0
a2171be Merge pull request kubernetes-csi#216 from msau42/process
cb98782 Merge pull request kubernetes-csi#217 from msau42/owners
a11216e add new reviewers and remove inactive reviewers
dd98675 Add step for checking builds
b66c082 Merge pull request kubernetes-csi#214 from pohly/junit-fixes
b9b6763 filter-junit.go: fix loss of testcases when parsing Ginkgo v2 JUnit
d427783 filter-junit.go: preserve system error log
38e1146 prow.sh: publish individual JUnit files as separate artifacts
78c0fb7 Merge pull request kubernetes-csi#208 from jsafrane/skip-selinux
36e433e Skip SELinux tests in CI by default
348d4a9 Merge pull request kubernetes-csi#207 from RaunakShah/reviewers
1efc272 Merge pull request kubernetes-csi#206 from RaunakShah/update-prow
7d410d8 Changes to csi prow to run e2e tests in sidecars
cfa5a75 Merge pull request kubernetes-csi#203 from humblec/test-vendor
4edd1d8 Add RaunakShah to CSI reviewers group
7ccc959 release tools update to 1.19
d24254f Merge pull request kubernetes-csi#202 from xing-yang/kind_0.14.0
0faa3fc Update to Kind v0.14.0 images
ef4e1b2 Merge pull request kubernetes-csi#201 from xing-yang/add_1.24_image
4ddce25 Add 1.24 Kind image
7fe5149 Merge pull request kubernetes-csi#200 from pohly/bump-kubernetes-version
70915a8 prow.sh: update snapshotter version
31a3f38 Merge pull request kubernetes-csi#199 from pohly/bump-kubernetes-version
7577454 prow.sh: bump Kubernetes to v1.22.0
d29a2e7 Merge pull request kubernetes-csi#198 from pohly/csi-test-5.0.0
41cb70d prow.sh: sanity testing with csi-test v5.0.0
c85a63f Merge pull request kubernetes-csi#197 from pohly/fix-alpha-testing
b86d8e9 support Kubernetes 1.25 + Ginkgo v2
ab0b0a3 Merge pull request kubernetes-csi#192 from andyzhangx/patch-1
7bbab24 Merge pull request kubernetes-csi#196 from humblec/non-alpha
e51ff2c introduce control variable for non alpha feature gate configuration
ca19ef5 Merge pull request kubernetes-csi#195 from pohly/fix-alpha-testing
3948331 fix testing with latest Kubernetes
e4dab7f Merge pull request kubernetes-csi#194 from yselkowitz/registry-k8s-io
84a4d5a Move from k8s.gcr.io to registry.k8s.io
9a0260c fix boilerplate header
37d1104 Merge pull request kubernetes-csi#191 from pohly/go-1.18
db917f5 update to Go 1.18
335339f Merge pull request kubernetes-csi#187 from mauriciopoppe/remove-eol-windows-versions
890b87a Merge pull request kubernetes-csi#188 from pwschuurman/update-release-notes-docs
274bc9b Update Sidecar Release Process documentation to reference latest syntax for release-notes tool
87b6c37 Merge pull request kubernetes-csi#185 from Garima-Negi/fix-OWNERS-files
f1de2c6 Fix OWNERS file - squashed commits
59ae38b Remove EOL windows versions from BUILD_PLATFORMS
5d66471 Merge pull request kubernetes-csi#186 from humblec/sp
d066f1b Correct prow.sh typo and make codespell linter pass
762e22d Merge pull request kubernetes-csi#184 from pohly/image-publishing-troubleshooting
81e26c3 SIDECAR_RELEASE_PROCESS.md: add troubleshooting for image publishing
31aa44d Merge pull request kubernetes-csi#182 from chrishenzie/csi-sanity-version
f49e141 Update csi-sanity test suite to v4.3.0
d9815c2 Merge pull request kubernetes-csi#181 from mauriciopoppe/armv7-support
05c1801 Add support to build arm/v7 images
4aedf35 Merge pull request kubernetes-csi#178 from xing-yang/timeout
2b9897e Increase build timeout
51d3702 Merge pull request kubernetes-csi#177 from mauriciopoppe/kind-image-1.23
a30efea Add kind image for 1.23
a6a1a79 Merge pull request kubernetes-csi#176 from pohly/go-1.17.3
0a2cf63 prow.sh: bump Go to 1.17.3
fc29fdd Merge pull request kubernetes-csi#141 from pohly/prune-replace-optional
5b9a1e0 Merge pull request kubernetes-csi#175 from jimdaga/patch-1
5eeb602 images: use k8s-staging-test-infra/gcb-docker-gcloud
b46691a go-get-kubernetes.sh: make replace statement pruning optional

git-subtree-dir: release-tools
git-subtree-split: 670bb0e
mowangdk added a commit to mowangdk/external-snapshotter that referenced this pull request Aug 6, 2023
@mowangdk
Squashed 'release-tools/' changes from 4133d1d..670bb0e
4ef8c7f 
670bb0e Merge pull request kubernetes-csi#229 from marosset/fix-codespell-errors
35d5e78 Merge pull request kubernetes-csi#219 from yashsingh74/update-registry
63473cc Merge pull request kubernetes-csi#231 from coulof/bump-go-version-1.20.5
29a5c76 Merge pull request kubernetes-csi#228 from mowangdk/chore/adopt_kubernetes_recommand_labels
8dd2821 Update cloudbuild image with go 1.20.5
1df23db Merge pull request kubernetes-csi#230 from msau42/prow
1f92b7e Add ginkgo timeout to e2e tests to help catch any stuck tests
2b8b80e fixing some codespell errors
c10b678 Merge pull request kubernetes-csi#227 from coulof/check-sidecar-supported-versions
72984ec chore: adopt kubernetes recommand label
b055535 Header
bd0a10b typo
c39d73c Add comments
f6491af Script to verify EOL sidecar version
901bcb5 Update registry k8s.gcr.io -> registry.k8s.io

git-subtree-dir: release-tools
git-subtree-split: 670bb0e
mowangdk added a commit to mowangdk/external-snapshotter that referenced this pull request Aug 15, 2023
@mowangdk
Squashed 'release-tools/' changes from 4133d1d..de2fba8
3653079 
de2fba8 Merge pull request kubernetes-csi#233 from andyzhangx/andyzhangx-patch-1
cee895e remove windows 20H2 build since it's EOL long time ago
670bb0e Merge pull request kubernetes-csi#229 from marosset/fix-codespell-errors
35d5e78 Merge pull request kubernetes-csi#219 from yashsingh74/update-registry
63473cc Merge pull request kubernetes-csi#231 from coulof/bump-go-version-1.20.5
29a5c76 Merge pull request kubernetes-csi#228 from mowangdk/chore/adopt_kubernetes_recommand_labels
8dd2821 Update cloudbuild image with go 1.20.5
1df23db Merge pull request kubernetes-csi#230 from msau42/prow
1f92b7e Add ginkgo timeout to e2e tests to help catch any stuck tests
2b8b80e fixing some codespell errors
c10b678 Merge pull request kubernetes-csi#227 from coulof/check-sidecar-supported-versions
72984ec chore: adopt kubernetes recommand label
b055535 Header
bd0a10b typo
c39d73c Add comments
f6491af Script to verify EOL sidecar version
901bcb5 Update registry k8s.gcr.io -> registry.k8s.io

git-subtree-dir: release-tools
git-subtree-split: de2fba8
@mowangdk mowangdk mentioned this pull request Aug 17, 2023
Merged
andyzhangx pushed a commit to andyzhangx/external-snapshotter that referenced this pull request Jun 5, 2024
@k8s-ci-robot
Merge pull request kubernetes-csi#230 from msau42/prow
1df23db 
Add ginkgo timeout to e2e tests to help catch any stuck tests
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jingxu97 jingxu97 Awaiting requested review from jingxu97

@lpabon lpabon Awaiting requested review from lpabon

Assignees

@bswartz bswartz

@humblec humblec

@yuxiangqian yuxiangqian

Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/bug Categorizes issue or PR as related to a bug. lgtm "Looks good to me", indicates that a PR is ready to be merged. release-note Denotes a PR that will be considered when it comes time to generate release notes. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@xing-yang @k8s-ci-robot @humblec @bswartz @yuxiangqian