Logview threadpool #861
Merged
Logview threadpool #861
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sorting jobs only by priority causes a situation where low priority jobs can get starved by a constant flow of high priority jobs. The new formula adds a modifier to the sorting rank to take into account the number of cores the job is requesting and also the number of days the job is waiting on the queue. Priorities numbers over 200 will mostly override the formula and work as a priority only based scheduling. sort = priority + (100 * (1 - (job.cores/job.int_min_cores))) + (age in days) Besides that, also take layer_int_cores_min into account when filtering folder_resourse limitations to avoid allocating more cores than the folder limits. (cherry picked from commit 566411aeeddc60983a30eabe121fd03263d05525)
This reverts commit 2eb4936
(cherry picked from commit 351e210a9f0d49bfe61e9a57bba349bf1951af4e)
DiegoTavares
requested review from
bcipriano,
gregdenton,
jrray,
larsbijl and
smith1511
as code owners
|
We're working on the pending SLA for Fermi (owner of the original commit for this issue) |
For some ui process that especially involves multi threading or concurrent update of widget data, some warnings can happen due to obsolete function calls. The code has been updated to suppress a couple of known warnings (cherry picked from commit dd0c4f900fb75d25d63897b6aacd2d0b02e230f5)
(cherry picked from commit 09446bbe5f0c631c74525cbc78b0e60ce114a2e2)
donalm
added a commit
to donalm/OpenCue
that referenced
this pull request
Oct 2, 2021
…when using the log viewer.
donalm
added a commit
to donalm/OpenCue
that referenced
this pull request
Oct 2, 2021
… freeze when using the log viewer." This reverts commit fe799f3.
Signed-off-by: Diego Tavares da Silva <[email protected]>
bcipriano
requested changes
Jan 29, 2024
DiegoTavares
commented
Jan 29, 2024
Signed-off-by: Diego Tavares da Silva <[email protected]>
|
@bcipriano Ready for review |
bcipriano
approved these changes
Jan 30, 2024
| finally: | ||
| QtCore.QTimer.singleShot(5000, self._display_log_content) | ||
|
|
||
| def _update_log(self): | ||
| # pylint: disable=no-self-use |
There was a problem hiding this comment.
nit: If there's no self use, can this be a static method? If not, add a comment explaining why.
DiegoTavares
merged commit c1dcad9
into
AcademySoftwareFoundation:master
12 checks passed
carlosfelgarcia
pushed a commit
to carlosfelgarcia/OpenCue
that referenced
this pull request
May 22, 2024
* Update dispatchQuery to use min_cores Sorting jobs only by priority causes a situation where low priority jobs can get starved by a constant flow of high priority jobs. The new formula adds a modifier to the sorting rank to take into account the number of cores the job is requesting and also the number of days the job is waiting on the queue. Priorities numbers over 200 will mostly override the formula and work as a priority only based scheduling. sort = priority + (100 * (1 - (job.cores/job.int_min_cores))) + (age in days) Besides that, also take layer_int_cores_min into account when filtering folder_resourse limitations to avoid allocating more cores than the folder limits. (cherry picked from commit 566411aeeddc60983a30eabe121fd03263d05525) * Revert "Update dispatchQuery to use min_cores" This reverts commit 2eb4936 * Add threadpool logic for handling incoming log update requests (cherry picked from commit 351e210a9f0d49bfe61e9a57bba349bf1951af4e) * Suppress known ui warning messages For some ui process that especially involves multi threading or concurrent update of widget data, some warnings can happen due to obsolete function calls. The code has been updated to suppress a couple of known warnings (cherry picked from commit dd0c4f900fb75d25d63897b6aacd2d0b02e230f5) * Fix logview unit tests to work around the threading logic (cherry picked from commit 09446bbe5f0c631c74525cbc78b0e60ce114a2e2) * FIx unit tests * Fix python lint * Fix python lint * Fix lint Signed-off-by: Diego Tavares da Silva <[email protected]> --------- Signed-off-by: Diego Tavares da Silva <[email protected]> Co-authored-by: Fermi Perumal <[email protected]>
carlosfelgarcia
pushed a commit
to carlosfelgarcia/OpenCue
that referenced
this pull request
May 22, 2024
* Update dispatchQuery to use min_cores Sorting jobs only by priority causes a situation where low priority jobs can get starved by a constant flow of high priority jobs. The new formula adds a modifier to the sorting rank to take into account the number of cores the job is requesting and also the number of days the job is waiting on the queue. Priorities numbers over 200 will mostly override the formula and work as a priority only based scheduling. sort = priority + (100 * (1 - (job.cores/job.int_min_cores))) + (age in days) Besides that, also take layer_int_cores_min into account when filtering folder_resourse limitations to avoid allocating more cores than the folder limits. (cherry picked from commit 566411aeeddc60983a30eabe121fd03263d05525) * Revert "Update dispatchQuery to use min_cores" This reverts commit 2eb4936 * Add threadpool logic for handling incoming log update requests (cherry picked from commit 351e210a9f0d49bfe61e9a57bba349bf1951af4e) * Suppress known ui warning messages For some ui process that especially involves multi threading or concurrent update of widget data, some warnings can happen due to obsolete function calls. The code has been updated to suppress a couple of known warnings (cherry picked from commit dd0c4f900fb75d25d63897b6aacd2d0b02e230f5) * Fix logview unit tests to work around the threading logic (cherry picked from commit 09446bbe5f0c631c74525cbc78b0e60ce114a2e2) * FIx unit tests * Fix python lint * Fix python lint * Fix lint Signed-off-by: Diego Tavares da Silva <[email protected]> --------- Signed-off-by: Diego Tavares da Silva <[email protected]> Co-authored-by: Fermi Perumal <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Link the Issue(s) this Pull Request is related to.
Users using the LogView were experiencing slowness on all opened widget when big log files were being streamed.
Summarize your change.
Add
QtCore.QThreadPoollogic for handling incoming log update requests.