Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a burst_buffer.conf #26

Merged
merged 3 commits into from
Dec 8, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions src/burst_buffer/burst_buffer.conf
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# The string that must be used by a job to request a DWS workflow. This
# string must be immediately preceded by a single '#' character.
# See https://slurm.schedmd.com/burst_buffer.conf.html
Directive=DW

2 changes: 1 addition & 1 deletion src/burst_buffer/burst_buffer.lua
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@

-- The directive used in job scripts. This is the same value specified in the
-- burst_buffer.conf "Directive" configuration parameter.
local DIRECTIVE = "BB_LUA"
local DIRECTIVE = "DW"

-- A placeholder for the WLM ID. This value is not used by Slurm, but is
-- used by other WLMs, so the DWS library expects it.
Expand Down
5 changes: 1 addition & 4 deletions testsuite/integration/slurm/README.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,6 @@
# Slurm Docker Cluster

This is a multi-container Slurm v22 cluster using docker-compose. The compose file
creates named volumes for persistent storage of MySQL data files as well as
Slurm state and log directories. The burst_buffer.lua script is mounted in the
slurmctld container and is available to be tested using `#BB_LUA` directives.
This is a multi-container Slurm v22 cluster using docker-compose. The compose file creates named volumes for persistent storage of MySQL data files as well as Slurm state and log directories. The burst_buffer.lua script and burst_buffer.conf are mounted in the slurmctld container and the burst_buffer script is available to be tested using `#DW` directives.

## Containers and Volumes

Expand Down
3 changes: 3 additions & 0 deletions testsuite/integration/slurm/docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,9 @@ services:
- type: bind
source: ../../../src/burst_buffer/burst_buffer.lua
target: /etc/slurm/burst_buffer.lua
- type: bind
source: ../../../src/burst_buffer/burst_buffer.conf
target: /etc/slurm/burst_buffer.conf
- ../kubeconfig:/home/slurm/.kube/config
expose:
- "6817"
Expand Down
2 changes: 1 addition & 1 deletion testsuite/integration/slurm/jobs/test-bb.sh
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@
#

#SBATCH --output=/jobs/slurm-%j.out
#BB_LUA
#DW
/bin/hostname
srun -l /bin/hostname
srun -l /bin/pwd
20 changes: 10 additions & 10 deletions testsuite/unit/src/burst_buffer/dws-test.lua
Original file line number Diff line number Diff line change
Expand Up @@ -198,8 +198,8 @@ describe("The dws library initializer", function()

it("can handle a non-empty array value for dwd", function()
local dwd = {}
dwd[1] = "#BB_LUA line 1"
dwd[2] = "#BB_LUA line 2"
dwd[1] = "#DW line 1"
dwd[2] = "#DW line 2"
fill_template(workflow, wlmID, jobID, userID, groupID, dwd)
assert.is_not_nil(string.find(workflow.yaml, "dwDirectives:\n %- " .. dwd[1] .. "\n" .. " %- " .. dwd[2] .. "\n"))
end)
Expand Down Expand Up @@ -666,8 +666,8 @@ describe("Burst buffer helpers", function()

it("finds directives in job script", function()
local in_dwd = {}
in_dwd[1] = "#BB_LUA pool=pool1 capacity=1K"
in_dwd[2] = "#BB_LUA pool=pool2 capacity=1K"
in_dwd[1] = "#DW pool=pool1 capacity=1K"
in_dwd[2] = "#DW pool=pool2 capacity=1K"
local job_script = "#!/bin/bash\n" .. in_dwd[1] .. "\n" .. in_dwd[2] .. "\nsrun application.sh\n"

write_job_script(job_script_name, job_script)
Expand Down Expand Up @@ -771,14 +771,14 @@ describe("Burst buffer helpers", function()

it("can create workflow from job script with directives", function()
local in_dwd = {}
in_dwd[1] = "#BB_LUA pool=pool1 capacity=1K"
in_dwd[2] = "#BB_LUA pool=pool2 capacity=1K"
in_dwd[1] = "#DW pool=pool1 capacity=1K"
in_dwd[2] = "#DW pool=pool2 capacity=1K"
local job_script = "#!/bin/bash\n" .. in_dwd[1] .. "\n" .. in_dwd[2] .. "\nsrun application.sh\n"

write_job_script(job_script_name, job_script)

-- The DWS environment does not have a ruleset for
-- the BB_LUA directives, so we should expect an error.
-- the #DW directives, so we should expect an error.
-- We'll look for only a small piece of the error
-- message here.
local result_wanted = "unable to find ruleset"
Expand Down Expand Up @@ -907,13 +907,13 @@ describe("Slurm API", function()

it("slurm_bb_job_process can validate workflow from job script with directives", function()
local in_dwd = {}
in_dwd[1] = "#BB_LUA pool=pool1 capacity=1K"
in_dwd[2] = "#BB_LUA pool=pool2 capacity=1K"
in_dwd[1] = "#DW pool=pool1 capacity=1K"
in_dwd[2] = "#DW pool=pool2 capacity=1K"
local job_script = "#!/bin/bash\n" .. in_dwd[1] .. "\n" .. in_dwd[2] .. "\nsrun application.sh\n"
write_job_script(job_script_name, job_script)

-- The DWS environment does not have a ruleset for
-- the BB_LUA directives, so we should expect an error.
-- the #DW directives, so we should expect an error.
-- We'll look for only a small piece of the error
-- message here.
local result_wanted = "unable to find ruleset"
Expand Down