Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reduce unnecessary DB queries for Actions tasks #25199

Merged
merged 48 commits into from
Jul 24, 2023

Conversation

sillyguodong
Copy link
Contributor

@sillyguodong sillyguodong commented Jun 12, 2023

Close #24544

Changes:

  • Create action_tasks_version table to store the latest version of each scope (global, org and repo).
  • When a job with the status of waiting is created, the tasks version of the scopes it belongs to will increase.
  • When the status of a job already in the database is updated to waiting, the tasks version of the scopes it belongs to will increase.
  • On Gitea side, in FeatchTask(), will try to query the action_tasks_version record of the scope of the runner that call FetchTask(). If the record does not exist, will insert a row. Then, Gitea will compare the version passed from runner to Gitea with the version in database, if inconsistent, try pick task. Gitea always returns the latest version from database to the runner.

Related:

@GiteaBot GiteaBot added the lgtm/need 2 This PR needs two approvals by maintainers to be considered for merging. label Jun 12, 2023
@pull-request-size pull-request-size bot added the size/M Denotes a PR that changes 30-99 lines, ignoring generated files. label Jun 12, 2023
@pull-request-size pull-request-size bot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Jun 29, 2023
@sillyguodong sillyguodong marked this pull request as ready for review June 30, 2023 02:53
@silverwind silverwind added the topic/gitea-actions related to the actions of Gitea label Jun 30, 2023
@silverwind silverwind added the performance/speed performance issues with slow downs label Jun 30, 2023
models/actions/tasks_version.go Show resolved Hide resolved
models/actions/tasks_version.go Outdated Show resolved Hide resolved
models/actions/tasks_version.go Outdated Show resolved Hide resolved
models/actions/tasks_version.go Outdated Show resolved Hide resolved
models/migrations/v1_21/v265.go Outdated Show resolved Hide resolved
models/actions/tasks_version.go Outdated Show resolved Hide resolved
routers/api/actions/runner/runner.go Outdated Show resolved Hide resolved
services/actions/notifier_helper.go Outdated Show resolved Hide resolved
@lunny
Copy link
Member

lunny commented Jul 22, 2023

Looks like it didn't work

图片

Copy link
Member

@lunny lunny left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It doesn't work.

@GiteaBot GiteaBot added lgtm/blocked A maintainer has reservations with the PR and thus it cannot be merged and removed lgtm/done This PR has enough approvals to get merged. There are no important open reservations anymore. labels Jul 22, 2023
@lunny
Copy link
Member

lunny commented Jul 23, 2023

图片 It's my fault to not enable actions. But it's still check database every 2 seconds?

@yardenshoham yardenshoham removed the reviewed/wait-merge This pull request is part of the merge queue. It will be merged soon. label Jul 23, 2023
@GiteaBot GiteaBot added lgtm/done This PR has enough approvals to get merged. There are no important open reservations anymore. and removed lgtm/blocked A maintainer has reservations with the PR and thus it cannot be merged labels Jul 23, 2023
@wolfogre
Copy link
Member

It's my fault to not enable actions. But it's still check database every 2 seconds?

Yes, but it's lighter than start a transaction and filter jobs. And caches for versions could be added in another PR.

@lunny lunny added the reviewed/wait-merge This pull request is part of the merge queue. It will be merged soon. label Jul 24, 2023
@lunny lunny enabled auto-merge (squash) July 24, 2023 05:52
@lunny lunny merged commit f5c7d4c into go-gitea:main Jul 24, 2023
@GiteaBot GiteaBot removed the reviewed/wait-merge This pull request is part of the merge queue. It will be merged soon. label Jul 24, 2023
zjjhot added a commit to zjjhot/gitea that referenced this pull request Jul 25, 2023
* giteaofficial/main: (23 commits)
  Avoid writing config file if not installed (go-gitea#26107)
  Implement auto-cancellation of concurrent jobs if the event is push (go-gitea#25716)
  [skip ci] Updated translations via Crowdin
  doc guide the user to create the appropriate level runner (go-gitea#26091)
  Fix handling of Debian files with trailing slash (go-gitea#26087)
  fix Missing 404 swagger response docs for /admin/users/{username} (go-gitea#26086)
  Allow the use of alternative net.Listener implementations by downstreams (go-gitea#25855)
  Add missing default value for some Bool cli flags (go-gitea#26082)
  Reduce unnecessary DB queries for Actions tasks (go-gitea#25199)
  Use stderr as fallback if the log file can't be opened (go-gitea#26074)
  Make organization redirect warning more clear (go-gitea#26077)
  Replace gogs/cron with go-co-op/gocron (go-gitea#25977)
  Remove `db.DefaultContext` in `routers/` and `cmd/` (go-gitea#26076)
  Categorize admin settings sidebar panel (go-gitea#26030)
  [skip ci] Updated translations via Crowdin
  Fix duplicated url prefix on issue context menu (go-gitea#26066)
  Add context parameter to some database functions (go-gitea#26055)
  Fix branch list auth (go-gitea#26041)
  Fix the truncate and alignment problem for some admin tables (go-gitea#26042)
  Update secrets.en-us.md (go-gitea#26057)
  ...
@go-gitea go-gitea locked as resolved and limited conversation to collaborators Oct 22, 2023
@sillyguodong sillyguodong deleted the feature/fetch_with_task_index branch February 29, 2024 03:30
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
lgtm/done This PR has enough approvals to get merged. There are no important open reservations anymore. performance/speed performance issues with slow downs size/L Denotes a PR that changes 100-499 lines, ignoring generated files. topic/gitea-actions related to the actions of Gitea
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Proposal] Reduce unnecessary DB queries for Actions tasks
7 participants