improvements to documentation and install.py for Linux performance tw…

…eaks idaholab#495

docs/host-config-linux.md

@@ -58,6 +58,13 @@ vm.dirty_background_ratio=40
 vm.dirty_ratio=80
 ```
 
+* In addition, the [some suggest](https://www.elastic.co/guide/en/elasticsearch/reference/current/system-config-tcpretries.html) lowering the TCP retransmission timeout to `5`. However, if your host communicates with other systems over a low-quality network, this low of a setting may be detrimental to those communications. To set this value, add the following to `/etc/sysctl.conf`:
+
+```
+# maximum number of TCP retransmissions
+net.ipv4.tcp_retries2=5
+```
+
 * Depending on your distribution, create **either** the file `/etc/security/limits.d/limits.conf` containing:
 
 ```
@@ -86,7 +93,9 @@ DefaultLimitMEMLOCK=infinity
 blockdev --setra 512 /dev/sda
 ```
 
-* Change the I/O scheduler to `deadline` or `noop`. Again, this can be done in a variety of ways. The simplest is to add `elevator=deadline` to the arguments in `GRUB_CMDLINE_LINUX` in `/etc/default/grub`, then running `sudo update-grub2`
+* Change the I/O scheduler to `deadline` or `noop`. Again, this can be done in a variety of ways. The simplest is to add `elevator=deadline` to the arguments in `GRUB_CMDLINE_LINUX` in `/etc/default/grub`, then running `sudo update-grub`.
+
+* Enable cgroup accounting for memory and swap space. This can be done by adding `cgroup_enable=memory swapaccount=1 cgroup.memory=nokmem` to the arguments in `GRUB_CMDLINE_LINUX` in `/etc/default/grub`, then running `sudo update-grub`.
 
 * If you are planning on using very large data sets, consider formatting the drive containing the `opensearch` volume as XFS.
 

docs/quickstart.md

@@ -23,7 +23,13 @@ The `build.sh` script can build Malcolm's images from scratch. See [Building fro
 The scripts to control Malcolm require Python 3. The [`install.py`](malcolm-config.md#ConfigAndTuning) script requires the [dotenv](https://github.com/theskumar/python-dotenv), [requests](https://docs.python-requests.org/en/latest/) and [ruamel.yaml](https://yaml.readthedocs.io/en/latest/) modules for Python 3, and will make use of the [pythondialog](https://pythondialog.sourceforge.io/) module for user interaction (on Linux) if it is available.
 
 You must run [`auth_setup`](authsetup.md#AuthSetup) prior to pulling Malcolm's images. You should also ensure your system configuration and Malcolm settings are tuned by running `./scripts/install.py` and `./scripts/configure` (see [Malcolm Configuration](malcolm-config.md#ConfigAndTuning)).
-
+
+Users may wish to read the documentation on platform-specific host configuration:
+
+* [Linux host system configuration](host-config-linux.md#HostSystemConfigLinux)
+* [macOS host system configuration](host-config-macos.md#HostSystemConfigMac)
+* [Windows host system configuration](host-config-windows.md#HostSystemConfigWindows)
+
 ### Pull Malcolm's Container images
 
 Malcolm's images are periodically built and hosted on [GitHub](https://github.com/orgs/idaholab/packages?repo_name=Malcolm). If you already have [Docker](https://www.docker.com/) and [Docker Compose](https://docs.docker.com/compose/), these prebuilt images can be pulled by navigating into the Malcolm directory (containing the `docker-compose.yml` file) and running `docker compose --profile malcolm pull` like this:

hedgehog-iso/config/hooks/normal/0991-security-performance.hook.chroot

@@ -92,6 +92,7 @@ net.ipv4.conf.default.accept_source_route=0
 net.ipv4.conf.default.send_redirects=0
 net.ipv4.icmp_echo_ignore_broadcasts=1
 net.ipv4.ip_forward=0
+net.ipv4.tcp_retries2=5
 net.ipv6.conf.all.accept_source_route=0
 net.ipv6.conf.all.accept_ra=0
 net.ipv6.conf.default.accept_ra=0

malcolm-iso/config/hooks/normal/0991-security-performance.hook.chroot

@@ -88,6 +88,7 @@ net.ipv4.conf.default.accept_source_route=0
 net.ipv4.conf.default.send_redirects=0
 net.ipv4.icmp_echo_ignore_broadcasts=1
 net.ipv4.ip_forward=1
+net.ipv4.tcp_retries2=5
 net.ipv6.conf.all.accept_source_route=0
 net.ipv6.conf.all.accept_ra=0
 net.ipv6.conf.default.accept_ra=0

scripts/install.py

@@ -3324,6 +3324,13 @@ def tweak_system_files(self):
                 'vm.dirty_ratio defines the maximum percentage of dirty system memory before committing everything',
                 ['# maximum % of dirty system memory before committing everything', 'vm.dirty_ratio=80'],
             ),
+            ConfigLines(
+                [],
+                '/etc/sysctl.conf',
+                'net.ipv4.tcp_retries2=',
+                'net.ipv4.tcp_retries2 defines the maximum number of TCP retransmissions',
+                ['# maximum number of TCP retransmissions', 'net.ipv4.tcp_retries2=5'],
+            ),
             ConfigLines(
                 ['centos', 'core'],
                 '/etc/systemd/system.conf.d/limits.conf',
@@ -3393,6 +3400,36 @@ def tweak_system_files(self):
                         privileged=True,
                     )
 
+        # tweak other kernel parameters
+
+        # cgroup accounting in GRUB_CMDLINE_LINUX in /etc/default/grub
+        if (
+            (grubFileName := '/etc/default/grub')
+            and os.path.isfile(grubFileName)
+            and (not [line.rstrip('\n') for line in open(grubFileName) if 'cgroup' in line.lower()])
+            and InstallerYesOrNo(
+                f'\ncgroup parameters appear to be missing from {grubFileName}, set them?',
+                default=True,
+            )
+        ):
+            err, out = self.run_process(
+                [
+                    'bash',
+                    '-c',
+                    f'sed -i \'s/^GRUB_CMDLINE_LINUX="/&cgroup_enable=memory swapaccount=1 cgroup.memory=nokmem /\' {grubFileName}',
+                ],
+                privileged=True,
+            )
+            if err == 0:
+                if which('update-grub', debug=self.debug):
+                    err, out = self.run_process(['update-grub'], privileged=True)
+                elif which('update-grub2', debug=self.debug):
+                    err, out = self.run_process(['update-grub2'], privileged=True)
+                else:
+                    InstallerDisplayMessage(
+                        f"{grubFileName} has been modified, consult your distribution's documentation generate new grub config file"
+                    )
+
 
 ###################################################################################################
 class MacInstaller(Installer):