Skip to content

Commit

Permalink
fix: Set MAX_DB_WORKER limit in local_run
Browse files Browse the repository at this point in the history
We allow the user to set the number of concurrent jobs using the
`--concurrency` argument and use this to update the worker limit, but we
didn't update the database worker limit.

The distinction is meaningless for local jobs in any case because these
jobs are just generating dummy data, but it meant that jobs ran with
less parallelism than expected. It also lead to "Waiting on available
database workers" messages appearing in the log output which caused
confusion and anxiety.
  • Loading branch information
evansd committed Oct 1, 2024
1 parent 4d08e64 commit e9be162
Showing 1 changed file with 1 addition and 0 deletions.
1 change: 1 addition & 0 deletions jobrunner/cli/local_run.py
Original file line number Diff line number Diff line change
Expand Up @@ -259,6 +259,7 @@ def create_and_run_jobs(
config.USING_DUMMY_DATA_BACKEND = True
config.CLEAN_UP_DOCKER_OBJECTS = clean_up_docker_objects
config.MAX_WORKERS = concurrency
config.MAX_DB_WORKERS = concurrency
config.DEFAULT_JOB_MEMORY_LIMIT = memory
config.DEFAULT_JOB_CPU_COUNT = cpus

Expand Down

0 comments on commit e9be162

Please sign in to comment.