Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[develop] Setup custom munge key #2443

Closed
wants to merge 28 commits into from
Closed
Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
47743f7
Custom munge key setup, Add logic to the head node slurm configuratio…
hehe7318 Sep 1, 2023
cbb3fbd
refine, use resource to manage munge key
hehe7318 Sep 1, 2023
b8f5cfa
use lazy to dig munge key secret arn
hehe7318 Sep 1, 2023
d2eb7a1
test changing position of the arn manager
hehe7318 Sep 1, 2023
dcc5deb
set correct ownership to the munge key
hehe7318 Sep 5, 2023
76ef5b6
refine
hehe7318 Sep 5, 2023
3595f63
Add region
hehe7318 Sep 5, 2023
ad81ea2
add error handler
hehe7318 Sep 5, 2023
f5b51a1
Merge branch 'develop' into wip/munge-key-setup
hehe7318 Sep 6, 2023
a07c220
Add kitchen tests
hehe7318 Sep 6, 2023
49dfda0
Add kitchen test of check error message
hehe7318 Sep 6, 2023
b2c1bb9
Merge branch 'develop' into wip/munge-key-setup
hehe7318 Sep 7, 2023
9e19ef0
commit for test update
hehe7318 Sep 7, 2023
5ee2022
Add update logic and a unused update resource for now.
hehe7318 Sep 8, 2023
7cff2cc
Correct the update script resource path
hehe7318 Sep 8, 2023
5626997
change back to lazy dig instead of using ruby block
hehe7318 Sep 8, 2023
c5cbf8d
change update logic, add kitchen test config
hehe7318 Sep 8, 2023
dc5bb79
kitchen test configuration
hehe7318 Sep 8, 2023
334920d
Merge branch 'develop' into wip/munge-key-setup
hehe7318 Sep 8, 2023
2ce20d7
Add rotation script update logic
hehe7318 Sep 8, 2023
dc8dc8f
Add restart munge and share munge key logic in rotation script and up…
hehe7318 Sep 8, 2023
d88429b
Correct an error in update logic
hehe7318 Sep 8, 2023
6f71cbe
Use restart munge instead of enable and start munge in update logic
hehe7318 Sep 9, 2023
e7eeffa
Add cluster user variable in rotation script
hehe7318 Sep 10, 2023
59be5b5
Correct the logic in rotation script
hehe7318 Sep 10, 2023
e0e6f23
Add kitchen tests
hehe7318 Sep 11, 2023
45224cd
Modify the kitchen tests tags
hehe7318 Sep 11, 2023
9af2561
Add some prints in the rotation script
hehe7318 Sep 11, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 4 additions & 13 deletions cookbooks/aws-parallelcluster-slurm/libraries/helpers.rb
Original file line number Diff line number Diff line change
Expand Up @@ -66,19 +66,10 @@ def enable_munge_service
end

def setup_munge_head_node
# Generate munge key
bash 'generate_munge_key' do
not_if { ::File.exist?('/etc/munge/munge.key') }
user node['cluster']['munge']['user']
group node['cluster']['munge']['group']
cwd '/tmp'
code <<-HEAD_CREATE_MUNGE_KEY
set -e
# Generates munge key in /etc/munge/munge.key
/usr/sbin/mungekey --verbose
# Enforce correct permission on the key
chmod 0600 /etc/munge/munge.key
HEAD_CREATE_MUNGE_KEY
munge_key_manager 'manage_munge_key' do
munge_key_secret_arn lazy {
node['cluster']['config'].dig(:DevSettings, :SlurmSettings, :MungeKeySecretArn)
}
end

enable_munge_service
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
# frozen_string_literal: true

#
# Cookbook:: aws-parallelcluster-slurm
# Recipe:: config_head_node
#
# Copyright:: 2013-2021 Amazon.com, Inc. or its affiliates. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"). You may not use this file except in compliance with the
# License. A copy of the License is located at
#
# http://aws.amazon.com/apache2.0/
#
# or in the "LICENSE.txt" file accompanying this file. This file is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES
# OR CONDITIONS OF ANY KIND, express or implied. See the License for the specific language governing permissions and
# limitations under the License.

resource_name :munge_key_manager
provides :munge_key_manager
unified_mode true

property :munge_key_secret_arn, String

default_action :manage

action :manage do
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we use a more descriptive action e.g. :setup_munge_key?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure!

if new_resource.munge_key_secret_arn
# This block will fetch the munge key from Secrets Manager
bash 'fetch_and_decode_munge_key' do
user 'root'
group 'root'
cwd '/tmp'
code <<-FETCH_AND_DECODE
# Get encoded munge key from secrets manager and decode it
encoded_key=$(aws secretsmanager get-secret-value --secret-id #{new_resource.munge_key_secret_arn} --query 'SecretString' --output text)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What if there's an error fetching the secret? This will not return the value.
Can you check if encoded_key has a value and possibly echo and error & exit early before base64 decoding it?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, done!

echo $encoded_key | base64 -d > /etc/munge/munge.key
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here, what if base64 decoding fails? We need to add the base64 decoded key to /etc/munge/munge.key only if base64 decoding succeeds, otherwise, echo an error and exit with non-zero.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done!

# Set ownership on the key
chown #{node['cluster']['munge']['user']}:#{node['cluster']['munge']['group']} /etc/munge/munge.key
# Enforce correct permission on the key
chmod 0600 /etc/munge/munge.key
FETCH_AND_DECODE
end
else
# This block will generate a munge key if it doesn't exist
bash 'generate_munge_key' do
not_if { ::File.exist?('/etc/munge/munge.key') }
user node['cluster']['munge']['user']
group node['cluster']['munge']['group']
cwd '/tmp'
code <<-GENERATE_KEY
set -e
/usr/sbin/mungekey --verbose
chmod 0600 /etc/munge/munge.key
GENERATE_KEY
end
end
end