Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Restart zero worker if there is still work to do #18658

Merged
merged 5 commits into from
Feb 8, 2022

Conversation

zeripath
Copy link
Contributor

@zeripath zeripath commented Feb 7, 2022

It is possible for the zero worker to timeout before all the work is finished.
This may mean that work may take a long time to complete because a worker will only
be induced on repushing.

Also ensure that requested count is reset after pulls and push mirror sync requests and add some more trace logging to the queue push.

Fix #18607

Signed-off-by: Andrew Thornton [email protected]

It is possible for the zero worker to timeout before all the work is finished.
This may mean that work may take a long time to complete because a worker will only
be induced on repushing.

Fix go-gitea#18607

Signed-off-by: Andrew Thornton <[email protected]>
@GiteaBot GiteaBot added the lgtm/need 1 This PR needs approval from one additional maintainer to be merged. label Feb 7, 2022
@codecov-commenter
Copy link

Codecov Report

Merging #18658 (b1dd300) into main (9712f7d) will increase coverage by 0.37%.
The diff coverage is 56.52%.

Impacted file tree graph

@@            Coverage Diff             @@
##             main   #18658      +/-   ##
==========================================
+ Coverage   46.31%   46.69%   +0.37%     
==========================================
  Files         846      846              
  Lines      121244   121259      +15     
==========================================
+ Hits        56155    56619     +464     
+ Misses      58271    57762     -509     
- Partials     6818     6878      +60     
Impacted Files Coverage Δ
models/migrate.go 40.00% <ø> (+0.73%) ⬆️
modules/queue/workerpool.go 57.22% <22.22%> (+0.23%) ⬆️
services/cron/tasks.go 42.30% <40.00%> (-0.12%) ⬇️
modules/translation/translation.go 60.95% <100.00%> (+3.66%) ⬆️
modules/charset/charset.go 69.74% <0.00%> (-4.21%) ⬇️
modules/log/file.go 71.52% <0.00%> (-1.99%) ⬇️
models/user/user.go 58.03% <0.00%> (+0.14%) ⬆️
routers/web/repo/pull.go 30.97% <0.00%> (+0.34%) ⬆️
models/repo_list.go 76.15% <0.00%> (+0.48%) ⬆️
models/pull.go 54.86% <0.00%> (+0.92%) ⬆️
... and 17 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3a91f84...b1dd300. Read the comment docs.

@GiteaBot GiteaBot added lgtm/done This PR has enough approvals to get merged. There are no important open reservations anymore. and removed lgtm/need 1 This PR needs approval from one additional maintainer to be merged. labels Feb 8, 2022
@zeripath
Copy link
Contributor Author

zeripath commented Feb 8, 2022

make lgtm work

@zeripath zeripath merged commit df44017 into go-gitea:main Feb 8, 2022
@zeripath zeripath deleted the repeat-boost-worker branch February 8, 2022 14:02
zeripath added a commit to zeripath/gitea that referenced this pull request Feb 8, 2022
Backport go-gitea#18658

It is possible for the zero worker to timeout before all the work is finished.
This may mean that work may take a long time to complete because a worker will only
be induced on repushing.

Also ensure that requested count is reset after pulls and push mirror sync requests and add some more trace logging to the queue push.

Fix go-gitea#18607

Signed-off-by: Andrew Thornton <[email protected]>
zeripath added a commit to zeripath/gitea that referenced this pull request Feb 8, 2022
go-gitea#18658 has added a check to see if we need to boost because there is still work to do
however the check is slightly complex and not ideal. There's no point boosting if
the queue is paused or can't scale. Therefore merge the two selects into one and add
a check to p.paused.

Signed-off-by: Andrew Thornton <[email protected]>
techknowlogick pushed a commit that referenced this pull request Feb 8, 2022
* Simplify Boost/Pause logic

#18658 has added a check to see if we need to boost because there is still work to do
however the check is slightly complex and not ideal. There's no point boosting if
the queue is paused or can't scale. Therefore merge the two selects into one and add
a check to p.paused.

Signed-off-by: Andrew Thornton <[email protected]>

* And on resume add a zeroboost if necessary

Signed-off-by: Andrew Thornton <[email protected]>

* simplify

Signed-off-by: Andrew Thornton <[email protected]>

Co-authored-by: Lauris BH <[email protected]>
lafriks pushed a commit that referenced this pull request Feb 8, 2022
* Restart zero worker if there is still work to do (#18658)

Backport #18658

It is possible for the zero worker to timeout before all the work is finished.
This may mean that work may take a long time to complete because a worker will only
be induced on repushing.

Also ensure that requested count is reset after pulls and push mirror sync requests and add some more trace logging to the queue push.

Fix #18607

Signed-off-by: Andrew Thornton <[email protected]>

* Update modules/queue/workerpool.go
zjjhot added a commit to zjjhot/gitea that referenced this pull request Feb 9, 2022
* giteaofficial/main:
  Prevent security failure due to bad APP_ID (go-gitea#18678)
  [skip ci] Updated translations via Crowdin
  Let `MinUnitAccessMode` return correct perm (go-gitea#18675)
  Simplify Boost/Pause logic (go-gitea#18673)
  update the comparison documents (go-gitea#18669)
  Restart zero worker if there is still work to do (go-gitea#18658)
firephreek pushed a commit to firephreek/gitea that referenced this pull request Feb 9, 2022
@wxiaoguang wxiaoguang added the backport/done All backports for this PR have been created label Feb 11, 2022
Chianina pushed a commit to Chianina/gitea that referenced this pull request Mar 28, 2022
* Restart zero worker if there is still work to do

It is possible for the zero worker to timeout before all the work is finished.
This may mean that work may take a long time to complete because a worker will only
be induced on repushing.

Also ensure that requested count is reset after pulls and push mirror sync requests and add some more trace logging to the queue push.

Fix go-gitea#18607

Signed-off-by: Andrew Thornton <[email protected]>
Chianina pushed a commit to Chianina/gitea that referenced this pull request Mar 28, 2022
* Simplify Boost/Pause logic

go-gitea#18658 has added a check to see if we need to boost because there is still work to do
however the check is slightly complex and not ideal. There's no point boosting if
the queue is paused or can't scale. Therefore merge the two selects into one and add
a check to p.paused.

Signed-off-by: Andrew Thornton <[email protected]>

* And on resume add a zeroboost if necessary

Signed-off-by: Andrew Thornton <[email protected]>

* simplify

Signed-off-by: Andrew Thornton <[email protected]>

Co-authored-by: Lauris BH <[email protected]>
@go-gitea go-gitea locked and limited conversation to collaborators Apr 28, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
backport/done All backports for this PR have been created lgtm/done This PR has enough approvals to get merged. There are no important open reservations anymore. type/bug
Projects
None yet
Development

Successfully merging this pull request may close these issues.

cron.update_mirrors broken in Gitea 1.16.0
6 participants