Skip to content

Commit

Permalink
Deployed 40cfafd with MkDocs version: 1.5.3
Browse files Browse the repository at this point in the history
  • Loading branch information
@cartalla
cartalla committed May 13, 2024
1 parent c999cd6 commit 52a307b
Show file tree
Hide file tree
Showing 6 changed files with 226 additions and 15 deletions.
215 changes: 208 additions & 7 deletions deployment-prerequisites/index.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -140,6 +140,22 @@
</li>
</ul>
</li>
<li class="nav-item" data-level="2"><a href="#security-groups-for-login-nodes" class="nav-link">Security Groups for Login Nodes</a>
<ul class="nav flex-column">
<li class="nav-item" data-level="3"><a href="#slurm-submitter-security-group" class="nav-link">Slurm Submitter Security Group</a>
<ul class="nav flex-column">
</ul>
</li>
<li class="nav-item" data-level="3"><a href="#slurm-head-node-security-group" class="nav-link">Slurm Head Node Security Group</a>
<ul class="nav flex-column">
</ul>
</li>
<li class="nav-item" data-level="3"><a href="#slurm-compute-node-security-group" class="nav-link">Slurm Compute Node Security Group</a>
<ul class="nav flex-column">
</ul>
</li>
</ul>
</li>
<li class="nav-item" data-level="2"><a href="#create-configuration-file" class="nav-link">Create Configuration File</a>
<ul class="nav flex-column">
<li class="nav-item" data-level="3"><a href="#configure-the-compute-instances" class="nav-link">Configure the Compute Instances</a>
Expand Down Expand Up @@ -226,13 +242,204 @@ <h3 id="install-cloud-development-kit-cdk-optional">Install Cloud Development Ki
<p>Note that the version of aws-cdk changes frequently.
The version that has been tested is in the CDK_VERSION variable in the install script.</p>
<p>The install script will try to install the prerequisites if they aren't already installed.</p>
<h2 id="security-groups-for-login-nodes">Security Groups for Login Nodes</h2>
<p>If you want to allow instances like remote desktops to use the cluster directly, you must define
three security groups that allow connections between the instance, the Slurm head node, and the Slurm compute nodes.
We call the instance that is connecting to the Slurm cluster a login node or a submitter instance.</p>
<p>I'll call the three security groups the following names, but they can be whatever you want.</p>
<ul>
<li>SlurmSubmitterSG</li>
<li>SlurmHeadNodeSG</li>
<li>SlurmComputeNodeSG</li>
</ul>
<h3 id="slurm-submitter-security-group">Slurm Submitter Security Group</h3>
<p>The SlurmSubmitterSG will be attached to your login nodes, such as your virtual desktops.</p>
<p>It needs at least the following inbound rules:</p>
<table>
<thead>
<tr>
<th>Type</th>
<th>Port range</th>
<th>Source</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td>TCP</td>
<td>1024-65535</td>
<td>SlurmHeadNodeSG</td>
<td>SlurmHeadNode ephemeral</td>
</tr>
<tr>
<td>TCP</td>
<td>1024-65535</td>
<td>SlurmComputeNodeSG</td>
<td>SlurmComputeNode ephemeral</td>
</tr>
<tr>
<td>TCP</td>
<td>6000-7024</td>
<td>SlurmComputeNodeSG</td>
<td>SlurmComputeNode X11</td>
</tr>
</tbody>
</table>
<p>It needs the following outbound rules.</p>
<table>
<thead>
<tr>
<th>Type</th>
<th>Port range</th>
<th>Destination</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td>TCP</td>
<td>2049</td>
<td>SlurmHeadNodeSG</td>
<td>SlurmHeadNode NFS</td>
</tr>
<tr>
<td>TCP</td>
<td>6818</td>
<td>SlurmComputeNodeSG</td>
<td>SlurmComputeNode slurmd</td>
</tr>
<tr>
<td>TCP</td>
<td>6819</td>
<td>SlurmHeadNodeSG</td>
<td>SlurmHeadNode slurmdbd</td>
</tr>
<tr>
<td>TCP</td>
<td>6820-6829</td>
<td>SlurmHeadNodeSG</td>
<td>SlurmHeadNode slurmctld</td>
</tr>
<tr>
<td>TCP</td>
<td>6830</td>
<td>SlurmHeadNodeSG</td>
<td>SlurmHeadNode slurmrestd</td>
</tr>
</tbody>
</table>
<h3 id="slurm-head-node-security-group">Slurm Head Node Security Group</h3>
<p>The SlurmHeadNodeSG will be specified in your configuration file for the slurm/SlurmCtl/AdditionalSecurityGroups parameter.</p>
<p>It needs at least the following inbound rules:</p>
<table>
<thead>
<tr>
<th>Type</th>
<th>Port range</th>
<th>Source</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td>TCP</td>
<td>2049</td>
<td>SlurmSubmitterSG</td>
<td>SlurmSubmitter NFS</td>
</tr>
<tr>
<td>TCP</td>
<td>6819</td>
<td>SlurmSubmitterSG</td>
<td>SlurmSubmitter slurmdbd</td>
</tr>
<tr>
<td>TCP</td>
<td>6820-6829</td>
<td>SlurmSubmitterSG</td>
<td>SlurmSubmitter slurmctld</td>
</tr>
<tr>
<td>TCP</td>
<td>6830</td>
<td>SlurmSubmitterSG</td>
<td>SlurmSubmitter slurmrestd</td>
</tr>
</tbody>
</table>
<p>It needs the following outbound rules.</p>
<table>
<thead>
<tr>
<th>Type</th>
<th>Port range</th>
<th>Destination</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td>TCP</td>
<td>1024-65535</td>
<td>SlurmSubmitterSG</td>
<td>SlurmSubmitter ephemeral</td>
</tr>
</tbody>
</table>
<h3 id="slurm-compute-node-security-group">Slurm Compute Node Security Group</h3>
<p>The SlurmComputeNodeSG will be specified in your configuration file for the slurm/InstanceConfig/AdditionalSecurityGroups parameter.</p>
<p>It needs at least the following inbound rules:</p>
<table>
<thead>
<tr>
<th>Type</th>
<th>Port range</th>
<th>Source</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td>TCP</td>
<td>6818</td>
<td>SlurmSubmitterSG</td>
<td>SlurmSubmitter slurmd</td>
</tr>
</tbody>
</table>
<p>It needs the following outbound rules.</p>
<table>
<thead>
<tr>
<th>Type</th>
<th>Port range</th>
<th>Destination</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td>TCP</td>
<td>1024-65535</td>
<td>SlurmSubmitterSG</td>
<td>SlurmSubmitter ephemeral</td>
</tr>
<tr>
<td>TCP</td>
<td>6000-7024</td>
<td>SlurmSubmitterSG</td>
<td>SlurmSubmitter X11</td>
</tr>
</tbody>
</table>
<h2 id="create-configuration-file">Create Configuration File</h2>
<p>Before you deploy a cluster you need to create a configuration file.
A default configuration file is found in <a href="https://github.com/aws-samples/aws-eda-slurm-cluster/blob/main/source/resources/config/default_config.yml">source/resources/config/default_config.yml</a>.
You should create a new config file and update the parameters for your cluster.
Ideally you should version control this file so you can keep track of changes.</p>
<p>The schema for the config file along with its default values can be found in <a href="https://github.com/aws-samples/aws-eda-slurm-cluster/blob/main/source/cdk/config_schema.py#L230-L445">source/cdk/config_schema.py</a>.
The schema is defined in python, but the actual config file should be in yaml format.</p>
The schema is defined in python, but the actual config file should be in yaml format.
See <a href="../config/">Configuration File Format</a> for documentation on all of the parameters.</p>
<p>The following are key parameters that you will need to update.
If you do not have the required parameters in your config file then the installer script will fail unless you specify the <code>--prompt</code> option.
You should save your selections in the config file.</p>
Expand Down Expand Up @@ -277,12 +484,6 @@ <h2 id="create-configuration-file">Create Configuration File</h2>
<td>None</td>
</tr>
<tr>
<td><a href="https://github.com/aws-samples/aws-eda-slurm-cluster/blob/main/source/cdk/config_schema.py#L480-L485">slurm/SubmitterSecurityGroupIds</a></td>
<td>Existing security groups that can submit to the cluster. For SOCA this is the ComputeNodeSG* resource.</td>
<td>sg-*</td>
<td>None</td>
</tr>
<tr>
<td><a href="https://github.com/aws-samples/aws-eda-slurm-cluster/blob/main/source/cdk/config_schema.py#L379-L380">ErrorSnsTopicArn</a></td>
<td>ARN of an SNS topic that will be notified of errors</td>
<td><code>arn:aws:sns:{{region}}:{AccountId}:{TopicName}</code></td>
Expand Down
2 changes: 1 addition & 1 deletion index.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -309,5 +309,5 @@ <h4 class="modal-title" id="keyboardModalLabel">Keyboard Shortcuts</h4>

<!--
MkDocs version : 1.5.3
Build Date UTC : 2024-05-13 22:39:01.372323+00:00
Build Date UTC : 2024-05-13 23:38:16.510384+00:00
-->
11 changes: 8 additions & 3 deletions res_integration/index.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -146,9 +146,14 @@ <h1 id="res-integration">RES Integration</h1>
<td>subnet-xxxxx</td>
</tr>
<tr>
<td>SubmitterSecurityGroupIds</td>
<td>The security group names and ids used by RES VDIs. The name will be something like <em>EnvironmentName</em>-vdc-dcv-host-security-group</td>
<td><em>EnvironmentName</em>-<em>VDISG</em>: sg-xxxxxxxx</td>
<td>slurm/SlurmCtl/AdditionalSecurityGroups</td>
<td>Security group ids that give desktop instances access to the head node and that give the head node access to VPC resources such as file systems.</td>
<td></td>
</tr>
<tr>
<td>slurm/InstanceConfig/AdditionalSecurityGroups</td>
<td>Security group ids that give desktop instances access to the compute nodes and that give compute nodes access to VPC resources such as file systems.</td>
<td></td>
</tr>
<tr>
<td>SubmitterInstanceTags</td>
Expand Down
2 changes: 1 addition & 1 deletion search/search_index.json
View file

Large diffs are not rendered by default.

Binary file modified sitemap.xml.gz
View file
Binary file not shown.
11 changes: 8 additions & 3 deletions soca_integration/index.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -140,9 +140,14 @@ <h1 id="soca-integration">SOCA Integration</h1>
<td>vpc-xxxxxx</td>
</tr>
<tr>
<td>SubmitterSecurityGroupIds</td>
<td>The ComputeNode security group name and id</td>
<td><em>cluster-id</em>-<em>ComputeNodeSG</em>: sg-xxxxxxxx</td>
<td>slurm/SlurmCtl/AdditionalSecurityGroups</td>
<td>Security group ids that give desktop instances access to the head node and that give the head node access to VPC resources such as file systems.</td>
<td></td>
</tr>
<tr>
<td>slurm/InstanceConfig/AdditionalSecurityGroups</td>
<td>Security group ids that give desktop instances access to the compute nodes and that give compute nodes access to VPC resources such as file systems.</td>
<td></td>
</tr>
<tr>
<td>ExtraMounts</td>
Expand Down

0 comments on commit 52a307b

Please sign in to comment.