-
Notifications
You must be signed in to change notification settings - Fork 2
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Showing
25 changed files
with
577 additions
and
487 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -239,22 +239,55 @@ Change the target branch of the blocked PR to ``develop`` and remove the ``chain | |
| label from that PR. Remove the ``base`` label from the blocking PR. Lastly, remove the blocking | ||
| relationship. | ||
|
|
||
| Updating the AMI for GitLab instances | ||
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | ||
|
|
||
| Once a month, operators must check for updates to the AMI for the root volume of | ||
| the EC2 instance running GitLab. There are ways to dynamically determine the | ||
| latest AMI for Amazon Linux 2 but in the spirit of reproducible builds, we would | ||
| rather pin the AMI ID and adopt updates at our own discretion to avoid | ||
| unexpected failures. To obtain the latest compatible AMI ID, select the desired | ||
| ``….gitlab`` component, say, ``_select dev.gitlab`` and run | ||
|
|
||
| :: | ||
|
|
||
| aws ssm get-parameters \ | ||
| --names \ | ||
| $(aws ssm get-parameters-by-path \ | ||
| --path /aws/service/ami-amazon-linux-latest \ | ||
| --query "Parameters[].Name" \ | ||
| | jq -r .[] \ | ||
| | grep -F amzn2 \ | ||
| | grep -Fv minimal \ | ||
| | grep -Fv kernel-5.10 \ | ||
| | grep -F x86_64 \ | ||
| | grep -F ebs) \ | ||
| | jq -r .Parameters[].Value | ||
|
|
||
| This will print the ID of the most recent Amazon Linux 2 AMI. Update the value | ||
| of the ``ami_id`` variable in ``terraform/gitlab/gitlab.tf.json.template.py``. | ||
| The variable holds a dictionary with one entry per region, because AMIs are | ||
| specific to a region. If there are ``….gitlab`` components in more than one AWS | ||
| region (uncommon), you need to select at least one ``….gitlab`` component in | ||
| each of these regions, rerun the command above for each such component, and add | ||
| or update the ``ami_id`` entry for the respective region. | ||
|
|
||
| Upgrading GitLab & ClamAV | ||
| ^^^^^^^^^^^^^^^^^^^^^^^^^ | ||
|
|
||
| Operators must check for updates to GitLab and ClamAV on a monthly basis in | ||
| addition to triaging GitLab security releases that occur during the month. | ||
| An email notification is sent to ``[email protected]`` when a GitLab security | ||
| Operators must check for updates to the Docker images for GitLab and ClamAV at | ||
| least once a month, and whenever a GitLab security releases requires it. An | ||
| email notification is sent to ``[email protected]`` when a GitLab security | ||
| release is available. Discuss with the lead the **Table of Fixes** referenced in | ||
| the release blog post to determine the urgency of the update. An email | ||
| notification should also be received when ClamAV releases become available. The | ||
| current version of GitLab installed can be found on the ``/help`` endpoint of | ||
| `GitLab dev`_, and the available releases can be found on the | ||
| `GitLab Docker image`_ page. When updating the GitLab instance, check if there | ||
| are applicable updates to the `GitLab runner image`_. Use the latest runner | ||
| image whose major and minor version match that of the GitLab image. Similarly, | ||
| check for available releases to ClamAV in the `ClamAV image`_. The current | ||
| version of ClamAV image being used can be found by running:: | ||
| `GitLab dev`_, and the available releases can be found on the `GitLab Docker | ||
| image`_ page. When updating the GitLab instance, check if there are applicable | ||
| updates to the `GitLab runner image`_. Use the latest runner image whose major | ||
| and minor version match that of the GitLab image. Similarly, check for available | ||
| releases to ClamAV in the `ClamAV image`_. The current version of ClamAV image | ||
| being used can be found by running:: | ||
|
|
||
| cat $project_root/terraform/gitlab/gitlab.tf.json.template.py | grep 'clamav_image =' | ||
|
|
||
|
|
@@ -313,6 +346,19 @@ For GitLab or ClamAV updates, use the ``--no-restart`` flag in order to leave | |
| the instance stopped after the snapshot has been created. There is no point in | ||
| starting the instance only to have the update terminate it again. | ||
|
|
||
| Updating software packages on GitLab instances | ||
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | ||
|
|
||
| Once a week, operators must update all Linux packages installed on the root | ||
| volume of each GitLab instance. SSH access to the instances is necessary to | ||
| perform these instructions but on production instances this access is | ||
| unavailable, even to operators. In these cases the operator must request the | ||
| help of the system administrator via Slack to perform these steps. | ||
|
|
||
| SSH into the instance, and run ``sudo yum update`` followed by ``sudo reboot``. | ||
| Wait for the GitLab web application to become available again and perform a | ||
| ``git pull`` from one of the Git repositories hosted on that instance. | ||
|
|
||
| Adding snapshots to ``dev`` | ||
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^ | ||
|
|
||
|
|
||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,89 @@ | ||
| """ | ||
| Count the number of subgraphs per configured source and produce output to | ||
| expedite updated source configurations. | ||
| """ | ||
|
|
||
| import argparse | ||
| import sys | ||
|
|
||
| import attr | ||
|
|
||
| from azul import ( | ||
| config, | ||
| ) | ||
| from azul.args import ( | ||
| AzulArgumentHelpFormatter, | ||
| ) | ||
| from azul.azulclient import ( | ||
| AzulClient, | ||
| ) | ||
| from azul.indexer import ( | ||
| Prefix, | ||
| ) | ||
| from azul.openapi import ( | ||
| format_description, | ||
| ) | ||
| from azul.terra import ( | ||
| TDRSourceSpec, | ||
| ) | ||
|
|
||
|
|
||
| @attr.s(auto_attribs=True, frozen=True, kw_only=True) | ||
| class SourceSpecArgs: | ||
| project: str | ||
| snapshot: str | ||
| subgraph_count: int | ||
|
|
||
| def __str__(self) -> str: | ||
| return f'mksrc({self.project!r}, {self.snapshot!r}, {self.subgraph_count!r})' | ||
|
|
||
|
|
||
| def generate_sources(catalog: str) -> list[SourceSpecArgs]: | ||
| plugin = AzulClient().repository_plugin(catalog) | ||
| sources = [] | ||
| for spec in plugin.sources: | ||
| spec: TDRSourceSpec = attr.evolve(spec, prefix=Prefix.of_everything) | ||
| ref = plugin.resolve_source(str(spec)) | ||
| partitions = plugin.list_partitions(ref) | ||
| sources.append(SourceSpecArgs(project=spec.project, | ||
| snapshot=spec.name, | ||
| subgraph_count=sum(partitions.values()))) | ||
| return sources | ||
|
|
||
|
|
||
| def main(args: list[str]): | ||
| parser = argparse.ArgumentParser(description=__doc__, | ||
| formatter_class=AzulArgumentHelpFormatter) | ||
|
|
||
| parser.add_argument('--catalogs', | ||
| nargs='+', | ||
| metavar='NAME', | ||
| default=[ | ||
| c for c in config.catalogs | ||
| if c not in config.integration_test_catalogs | ||
| ], | ||
| help='The names of the catalogs to determine source specs for.') | ||
|
|
||
| args = parser.parse_args(args) | ||
|
|
||
| for catalog in args.catalogs: | ||
| print(catalog) | ||
| print('-' * len(catalog)) | ||
| spec_args_list = generate_sources(catalog) | ||
| spec_args_list.sort(key=lambda spec_args: spec_args.snapshot) | ||
| print(',\n'.join(map(str, spec_args_list)), end='\n\n') | ||
|
|
||
| print(format_description(''' | ||
| ----------------- | ||
| !!! IMPORTANT !!! | ||
| ----------------- | ||
| This script does *not* populate the `ma` or `pop` flags for the source | ||
| specs. Do not copy/paste the above output without checking whether these | ||
| flags should be applied. If `mksrc` generates a common prefix, manual | ||
| adjustment of the generated common prefix may be required. | ||
| ''')) | ||
|
|
||
|
|
||
| if __name__ == '__main__': | ||
| main(sys.argv[1:]) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.