Remove job configs #145
Remove job configs #145
Conversation
mickmister
added
2: Dev Review
Codecov Report
@@ Coverage Diff @@
## master #145 +/- ##
==========================================
- Coverage 24.20% 23.40% -0.81%
==========================================
Files 67 63 -4
Lines 2702 2594 -108
==========================================
- Hits 654 607 -47
+ Misses 1964 1909 -55
+ Partials 84 78 -6
Continue to review full report at Codecov.
|
| func (jm *JobManager) Close() error { | ||
| jm.env.Logger.Debugf("Deactivating all jobs due to plugin deactivation.") | ||
| jm.activeJobs.Range(func(k interface{}, v interface{}) bool { | ||
| job := v.(*activeJob) | ||
| err := jm.deactivateJob(job.RegisteredJob) | ||
| if err != nil { | ||
| jm.env.Logger.Debugf("Failed to deactivate job: %v", err) | ||
| jm.env.Logger.Warnf("Failed to deactivate %s job: %v", job.id, err) | ||
| } |
There was a problem hiding this comment.
@lieut-data We are experiencing an issue testing the cluster jobs in HA. We are testing with a fresh cloud server with --size miniHA, which spins up a cluster with two nodes.
I will be referring to the jobs as a singular "job" (cluster.Job), as all the jobs are showing the same behavior. The job activates properly on plugin activation. The successful activation message is logged by both nodes by the jm.activateJob method. The job runs its first execution with no issues. However, the job does not run again when its configured interval comes around.
Disabling the plugin results in the call to job.Close() to hang. Tracing the logs provided by this PR, the Deactivating all jobs due to plugin deactivation message is logged once (due to the node blocking on disable, and not allowing the second node to begin disabling), and does not log the debug message in jm.deactivateJob, nor the warning in this function. Therefore the job.Close() method must be blocking.
I'm not sure where to begin with debugging this. I'd like to debug locally, but I haven't set up an HA environment on my local machine. How do you advise moving forward with this?
There was a problem hiding this comment.
To be clear, this behavior was occurring before this PR. Part of the reason for this PR is to simplify the logic, to minimize the surface area for errors like the ones described above to occur.
There was a problem hiding this comment.
(For anyone following this thread, see https://community-daily.mattermost.com/core/pl/auyr1cd1q7nu9ytmtb9a5fod8a for the resolution.)
|
Tested and passed
|
#192) * Revert "Update main.go (#154)" This reverts commit be4a281d0cc791d10e6e5ae917b325b2f054e475. * Revert "[MM-33506] Use embed package to include plugin manifest (#145)" This reverts commit ca9ee3c17c6920a636a33f378e17395afd6f329f. * Revert "Don't generate manifest.ts (#127)" This reverts commit 18d30b50bc7ba800c9f05bfd82970781db0aea3e. * install-go-tools target, adopt gotestsum * bring back make apply + automatic versioning * Update build/manifest/main.go Co-authored-by: Michael Kochell <[email protected]> * suppress git describe error when no tags match * make version/release notes opt-in * fix whitespace in Makefile * document version management options --------- Co-authored-by: Michael Kochell <[email protected]>
* Sync with playbooks: install-go-tools, gotestsum, and dynamic versions (#192) * Revert "Update main.go (#154)" This reverts commit be4a281d0cc791d10e6e5ae917b325b2f054e475. * Revert "[MM-33506] Use embed package to include plugin manifest (#145)" This reverts commit ca9ee3c17c6920a636a33f378e17395afd6f329f. * Revert "Don't generate manifest.ts (#127)" This reverts commit 18d30b50bc7ba800c9f05bfd82970781db0aea3e. * install-go-tools target, adopt gotestsum * bring back make apply + automatic versioning * Update build/manifest/main.go Co-authored-by: Michael Kochell <[email protected]> * suppress git describe error when no tags match * make version/release notes opt-in * fix whitespace in Makefile * document version management options --------- Co-authored-by: Michael Kochell <[email protected]> * Fetch plugin logs from server (#193) Co-authored-by: Jesse Hallam <[email protected]> * [MM-231]: Fixed merge conflicts * [MM-231] Fix flaky tests --------- Co-authored-by: Jesse Hallam <[email protected]> Co-authored-by: Michael Kochell <[email protected]> Co-authored-by: Ben Schumacher <[email protected]> Co-authored-by: kshitij <[email protected]>
Summary
Scheduled jobs are no longer able to be disabled. They are still synchronized via the cluster job locks.
Ticket Link
Fixes #144
Fixes #135