MacOS agent AMI creation using packer. (#164)

* parent 83a6200 author pgodithi <[email protected]> 1651769732 -0400 committer pgodithi <[email protected]> 1658274990 -0400 gpgsig -----BEGIN PGP SIGNATURE----- iQIzBAABCAAdFiEEuaPMeSek3p9IWNzDMSDf0RO9/aIFAmLXRK4ACgkQMSDf0RO9 /aKKpQ/9FQ92spUtBEdt2lf4Kz6c2x7+eunr5kzzxZPB8fj28AFWBtGP/NA6a/WK BjRgtxZ4060PCDUtbcnfFy8mg8PdQa03FpofQKr6exSfDTI+TNISJAkcI2HIWzqx ehiFRDGWG577fgrnk67xxd6LxD8sO8EHWBJzZWKrYA3z7f0Q05n1q2osm/jhDYgK cEDQBHZJVi4htO2sBGJO3caqCbqt0EphxYy+uua5On4a2ZgWlBUxjqoaZ6X/LrnR 6IoN8PYVUtyrkbWkkXB+0T4PPDYP8DwnCrCsDQxQyYQ7tBqnR3hQ6nfga2d4Dry5 Dz9oqPoWZjDJNBAf+B6NAmwmyIZHn2lnVL/b8GvwGjSOixyUqaptRKVrg/PwIT+I i2CeMuGIxIG8pycC+wruSNoNqV26Vm7rFSOC4VSvOXneQUStMrAEM+LjcCDtG8bF 0837RaB4Vj37QfkXJyQl2JLTvw6TUKbZuCXqSTLF6oy4nfKN1wBPhfmgjRt146o3 VKt9iZ/q5kOEAPhEHJkHWoC/C+jaMRkVk0ZV5CoujQcIPZX2HfA88RogNZe0Ommy ayc+gLw8oBiRBeKkWuQLrLtrBYCLKrIK5qjEEj9MqLxbLSlDRXJjvZvPSUGqS2wg a8KKD0i9faPpRIuViJRqHztQBXVsv0UsYFMeeid00lJm2F4dmBk= =c8LN -----END PGP SIGNATURE----- fixed IAM roles and code cleanup Signed-off-by: pgodithi <[email protected]> Fix tests Signed-off-by: pgodithi <[email protected]> Added assume role feature Signed-off-by: pgodithi <[email protected]> Added assume role feature Signed-off-by: pgodithi <[email protected]> Added assume role feature Signed-off-by: pgodithi <[email protected]> Added assume role feature Signed-off-by: pgodithi <[email protected]> Added assume role feature Signed-off-by: pgodithi <[email protected]> Added 2 new plugins Signed-off-by: pgodithi <[email protected]> Add readme doc Signed-off-by: pgodithi <[email protected]> Added docker resources Signed-off-by: pgodithi <[email protected]> Added docker resources Signed-off-by: pgodithi <[email protected]> Bump minor version Signed-off-by: pgodithi <[email protected]> change docker tag to 2.332.3-lts Signed-off-by: pgodithi <[email protected]> added jenkinsOpts Signed-off-by: pgodithi <[email protected]> added jenkinsOpts Signed-off-by: pgodithi <[email protected]> added jenkinsOpts Signed-off-by: pgodithi <[email protected]> Added IAM required imports Signed-off-by: pgodithi <[email protected]> Added agent node unit tests Signed-off-by: pgodithi <[email protected]> Added agent node unit tests Signed-off-by: pgodithi <[email protected]> Added Managed SSM Policy to Agent instance role (#117) Signed-off-by: Rishabh Singh <[email protected]> fix agentAssumeRole Signed-off-by: pgodithi <[email protected]> Ecr public Signed-off-by: pgodithi <[email protected]> Added AmazonSSMManagedInstanceCore Signed-off-by: pgodithi <[email protected]> Added AmazonSSMManagedInstanceCore policy to agents. (#124) * fixed IAM roles and code cleanup Signed-off-by: pgodithi <[email protected]> * Fix tests Signed-off-by: pgodithi <[email protected]> * Added assume role feature Signed-off-by: pgodithi <[email protected]> * Added assume role feature Signed-off-by: pgodithi <[email protected]> * Added assume role feature Signed-off-by: pgodithi <[email protected]> * Added assume role feature Signed-off-by: pgodithi <[email protected]> * Added assume role feature Signed-off-by: pgodithi <[email protected]> * Added 2 new plugins Signed-off-by: pgodithi <[email protected]> * Add readme doc Signed-off-by: pgodithi <[email protected]> * Added docker resources Signed-off-by: pgodithi <[email protected]> * Added docker resources Signed-off-by: pgodithi <[email protected]> * Bump minor version Signed-off-by: pgodithi <[email protected]> * change docker tag to 2.332.3-lts Signed-off-by: pgodithi <[email protected]> * added jenkinsOpts Signed-off-by: pgodithi <[email protected]> * added jenkinsOpts Signed-off-by: pgodithi <[email protected]> * added jenkinsOpts Signed-off-by: pgodithi <[email protected]> * Added IAM required imports Signed-off-by: pgodithi <[email protected]> * Added agent node unit tests Signed-off-by: pgodithi <[email protected]> * Added agent node unit tests Signed-off-by: pgodithi <[email protected]> * fix agentAssumeRole Signed-off-by: pgodithi <[email protected]> * Ecr public Signed-off-by: pgodithi <[email protected]> * Added AmazonSSMManagedInstanceCore Signed-off-by: pgodithi <[email protected]> Added new jenkins agents and logging bug-fix (#126) Signed-off-by: Rishabh Singh <[email protected]> Added new output parameters and updated jenkins image tag (#128) Signed-off-by: Rishabh Singh <[email protected]> use jenkins-2.332.3 jdk8 base image (#129) Signed-off-by: Rishabh Singh <[email protected]> added support for QEMU emulators (#131) Signed-off-by: Rishabh Singh <[email protected]> Add Docker restart policy (#137) * fixed IAM roles and code cleanup Signed-off-by: pgodithi <[email protected]> * Fix tests Signed-off-by: pgodithi <[email protected]> * Added assume role feature Signed-off-by: pgodithi <[email protected]> * Added assume role feature Signed-off-by: pgodithi <[email protected]> * Added assume role feature Signed-off-by: pgodithi <[email protected]> * Added assume role feature Signed-off-by: pgodithi <[email protected]> * Added assume role feature Signed-off-by: pgodithi <[email protected]> * Added 2 new plugins Signed-off-by: pgodithi <[email protected]> * Add readme doc Signed-off-by: pgodithi <[email protected]> * Added docker resources Signed-off-by: pgodithi <[email protected]> * Added docker resources Signed-off-by: pgodithi <[email protected]> * Bump minor version Signed-off-by: pgodithi <[email protected]> * change docker tag to 2.332.3-lts Signed-off-by: pgodithi <[email protected]> * added jenkinsOpts Signed-off-by: pgodithi <[email protected]> * added jenkinsOpts Signed-off-by: pgodithi <[email protected]> * added jenkinsOpts Signed-off-by: pgodithi <[email protected]> * Added IAM required imports Signed-off-by: pgodithi <[email protected]> * Added agent node unit tests Signed-off-by: pgodithi <[email protected]> * Added agent node unit tests Signed-off-by: pgodithi <[email protected]> * fix agentAssumeRole Signed-off-by: pgodithi <[email protected]> * Ecr public Signed-off-by: pgodithi <[email protected]> * Added AmazonSSMManagedInstanceCore Signed-off-by: pgodithi <[email protected]> * added docker-compose restart policy Signed-off-by: pgodithi <[email protected]> Add markup formatter as safe/html disable syntax highlighting to allow html (#138) Signed-off-by: Peter Zhu <[email protected]> Set cache to false for aws secret manager plugin (#140) Signed-off-by: Peter Zhu <[email protected]> Add environment variables using configuration as code (#136) * Add environment variables using configuration as code Signed-off-by: Sayali Gaikawad <[email protected]> Install workflow-api plugin to get latest version and remove lockable permissions (#141) * Install workflow-api plugin to get latest version Signed-off-by: Sayali Gaikawad <[email protected]> * Remove lockable permissions Signed-off-by: Sayali Gaikawad <[email protected]> [Bug-fix]: Handle multiple colons and space (#142) * Handle multiple colons and space Signed-off-by: Sayali Gaikawad <[email protected]> * Handle multiple space Signed-off-by: Sayali Gaikawad <[email protected]> Move env variabled to yaml (#143) * Move to yaml Signed-off-by: Sayali Gaikawad <[email protected]> add ubuntu agent and cdn with lambda@edge resource for public access (#134) Signed-off-by: Rishabh Singh <[email protected]> Reload configuration as code via curl instead of cli (#145) * Reload configuration as code via curl instead of cli Signed-off-by: Sayali Gaikawad <[email protected]> Main/Agent Node new setups on tag and executors (#147) * Main/Agent Node new setups on tag and executors Signed-off-by: Peter Zhu <[email protected]> * Remove DCO check as it is replaced by dco app Signed-off-by: Peter Zhu <[email protected]> * Remove stack in node ts as it is not used anymore Signed-off-by: Peter Zhu <[email protected]> Fix Ubuntu agent init script bug (#148) Signed-off-by: Peter Zhu <[email protected]> Add Ubuntu Single Host for Gradle Check (#149) * Add Ubuntu Single Host for Gradle Check Signed-off-by: Peter Zhu <[email protected]> * Change default idle time to 60min Signed-off-by: Peter Zhu <[email protected]> Raise the gradle check runner to have c518xlarge as in Fork Jenkins (#150) Signed-off-by: Peter Zhu <[email protected]> New cmds reduce AL2 failure and increase gradle check to c524xlarge runner (#151) Signed-off-by: Peter Zhu <[email protected]> Add jdk14 as it is required by OS 1.x gradle check (#152) Signed-off-by: Peter Zhu <[email protected]> Migrate perf test agent setups to public jenkins (#153) Signed-off-by: Peter Zhu <[email protected]> Add Mac agent support Signed-off-by: pgodithi <[email protected]> Add Mac agent support: Added README Signed-off-by: pgodithi <[email protected]> Add Mac agent support: Added README Signed-off-by: pgodithi <[email protected]> Add Mac agent support: Added README Signed-off-by: pgodithi <[email protected]> Add Mac agent support: Added README Signed-off-by: pgodithi <[email protected]> Add Mac agent support: Added README Signed-off-by: pgodithi <[email protected]> Add Mac agent support: Added max-len, eqeqeq Signed-off-by: pgodithi <[email protected]> Add Mac agent support: Added tests Signed-off-by: pgodithi <[email protected]> Add Mac agent support: Added tests Signed-off-by: pgodithi <[email protected]> Add Mac agent support: Added tests Signed-off-by: pgodithi <[email protected]> * Add Mac agent support: fix conflicts Signed-off-by: pgodithi <[email protected]> * Add Mac agent support: fix conflicts Signed-off-by: pgodithi <[email protected]> * MacOS agents: packer Signed-off-by: pgodithi <[email protected]> * MacOS agents: packer, fix conflicts Signed-off-by: pgodithi <[email protected]> * MacOS agents: packer, fix conflicts Signed-off-by: pgodithi <[email protected]> * MacOS agents: packer fix suggestions Signed-off-by: pgodithi <[email protected]> * MacOS agents: packer fix suggestions Signed-off-by: pgodithi <[email protected]> * MacOS agents: packer fix suggestions Signed-off-by: pgodithi <[email protected]>
opensearch-project · Jul 22, 2022 · a249627 · a249627
1 parent fb47ac6
commit a249627
Show file tree

Hide file tree

Showing 5 changed files with 102 additions and 0 deletions.
diff --git a/lib/compute/agent-node-config.ts b/lib/compute/agent-node-config.ts
@@ -210,6 +210,7 @@ export class AgentNodeConfig {
        ebsOptimized: true,
        hostKeyVerificationStrategy: 'OFF',
        iamInstanceProfile: this.AgentNodeInstanceProfileArn,
+       idleTerminationMinutes: '720',
        labelString: 'Jenkins-Agent-MacOS12-X64-Mac1Metal-Multi-Host',
        maxTotalUses: -1,
        minimumNumberOfInstances: 1,

diff --git a/packer/README.md b/packer/README.md
@@ -10,6 +10,7 @@
 * jenkins-agent-win2016-x64.json: Windows 2016 Server.
 * jenkins-agent-win2019-x64.json: Windows 2019 Server (Recommended).
 * jenkins-agent-win2019-x64-alpine-wsl.json: Windows 2019 Server with WSL enabled running Alpine 3.
+* jenkins-agent-macos12-x64.json: MacOS 12 with x86_64_mac os_architecture.
 
 ### Usages
 

diff --git a/packer/jenkins-agent-macos12-x64.json b/packer/jenkins-agent-macos12-x64.json
@@ -0,0 +1,81 @@
+{
+  "variables":{
+    "ami_name":"Jenkins-Agent-MacOS-X64-Mac1Metal",
+    "os_version": "12.4",
+    "os_architecture": "x86_64_mac",
+    "build-region":"us-east-1",
+    "build-vpc":"vpc-05efb6d67e23254cf",
+    "build-subnet":"subnet-03e90b825a6634df8",
+    "build-secgrp":"sg-05ef1e255287ed3d2",
+    "build-time":"{{isotime \"2006-01-02T03-04-05Z\"}}",
+    "aws_ami_region":"us-east-1",
+    "root_volume_size_gb": "300"
+  },
+  "builders":[
+    {
+      "name":"{{user `ami_name`}}-{{user `os_version`}}-{{user `build-time`}}",
+      "ami_description":"{{user `ami_name`}}",
+      "type":"amazon-ebs",
+      "encrypt_boot":"false",
+      "region":"{{user `build-region`}}",
+      "ami_regions":"{{user `aws_ami_region`}}",
+      "instance_type":  "mac1.metal",
+      "ami_name":"{{user `ami_name`}}-{{user `os_version`}}-{{user `build-time`}}",
+      "vpc_id":"{{user `build-vpc`}}",
+      "subnet_id":"{{user `build-subnet`}}",
+      "security_group_ids":"{{user `build-secgrp`}}",
+      "ssh_interface": "",
+      "source_ami_filter":{
+        "filters":{
+          "virtualization-type":"hvm",
+          "name": "amzn-ec2-macos-{{user `os_version`}}-*",
+          "root-device-type":"ebs",
+          "architecture": "{{user `os_architecture`}}"
+        },
+        "owners":[
+          "amazon"
+        ],
+        "most_recent":true
+      },
+      "associate_public_ip_address":false,
+      "ssh_username":  "ec2-user", 
+      "ssh_timeout": "3h",
+      "tenancy": "host",
+      "ebs_optimized": "true",
+      "tags":{
+        "Name": "{{user `ami_name`}}-{{user `os_version`}}-{{user `build-time`}}",
+        "OS_Version":"{{user `os_version`}}",
+        "User":"Packer",
+        "Encrypted_AMI":"False",
+        "Created":"{{user `build-time`}}"
+      },
+      "launch_block_device_mappings":[
+        {
+          "device_name":"/dev/sda1",
+          "volume_size": "{{user `root_volume_size_gb`}}",
+          "delete_on_termination":true,
+          "volume_type":"gp3"
+        }
+      ]
+    }
+  ],
+  "provisioners":[
+    {
+      "type":"shell",
+      "scripts": ["scripts/macos/macos-disk-resize.sh"],
+      "max_retries": 3
+    },
+    {
+      "type":"shell",
+      "inline": [
+        "sudo /usr/local/bin/ec2-macos-init clean --all"
+      ],
+      "max_retries": 3
+    },
+    {
+      "type":"shell",
+      "scripts": ["scripts/macos/macos-agentsetup.sh"],
+      "max_retries": 3
+    }
+  ]
+}
diff --git a/packer/scripts/macos/macos-agentsetup.sh b/packer/scripts/macos/macos-agentsetup.sh
@@ -0,0 +1,14 @@
+#!/usr/bin/env bash
+sudo mkdir -p /var/jenkins/
+sudo chown -R ec2-user:staff /var/jenkins
+/usr/local/bin/brew update --preinstall
+/usr/local/bin/brew update
+/usr/local/bin/brew upgrade
+/usr/local/bin/brew install openjdk@8
+sudo ln -sfn /usr/local/opt/openjdk@8/libexec/openjdk.jdk /Library/Java/JavaVirtualMachines/openjdk-8.jdk
+/usr/local/bin/brew install [email protected]
+sudo cp /usr/local/opt/[email protected]/bin/pip3 /usr/local/bin/pip
+/usr/local/opt/[email protected]/bin/python3.7 -m pip install pipenv
+/usr/local/opt/[email protected]/bin/python3.7 -m pip install awscli
+/usr/local/bin/brew install curl 
+/usr/local/bin/brew install wget 
diff --git a/packer/scripts/macos/macos-disk-resize.sh b/packer/scripts/macos/macos-disk-resize.sh
@@ -0,0 +1,5 @@
+#!/usr/bin/env bash
+PDISK=$(diskutil list physical external | head -n1 | cut -d' ' -f1)
+APFSCONT=$(diskutil list physical external | grep Apple_APFS | tr -s ' ' | cut -d' ' -f8)
+yes | sudo diskutil repairDisk $PDISK
+sudo diskutil apfs resizeContainer $APFSCONT 0