Amazon Linux 2023 proof-of-concept (Take 2) #1340

dims · 2023-07-06T18:23:17Z

Makefile has a converter that generates al2023 configs from existing al2 ones

Very minimal changes to other files to ensure al2023 works and does not get in the way of AL2 ongoing development.

Based on earlier work by @cartermckinnon in:
#1212

cartermckinnon

Love it! I'll close #1212 in favor of this PR.

scripts/upgrade_kernel.sh

Makefile

scripts/install-worker.sh

scripts/validate.sh

Makefile

scripts/install-worker.sh

Makefile

Signed-off-by: Davanum Srinivas <[email protected]>

dims · 2023-07-09T13:33:23Z

@cartermckinnon if this PR is good enough, let's merge and iterate, that way i can throw a few CI jobs at the generated image to see how it fares.

hack/transform-al2-to-al2023.sh

Co-authored-by: Carter <[email protected]>

dims · 2023-07-10T17:24:54Z

@cartermckinnon thanks! applied the suggestion

cartermckinnon · 2023-07-10T17:41:36Z

I think this is fine to get us going. Some things I'd like to see in followup PR's:

A user guide entry on distro support (AL2023 being called out as experimental/in-progress).
A make var to select the distro, something like make 1.27 distro=al2023
Static variable file for al2023 instead of modifying the one for al2 (once the wrinkles are ironed out).

I'm not sure what the right approach will be for the template file. We need to eventually move to the HCL template format, and AL2023 is an opportunity to do that.

* Update CHANGELOG.md for v20230703 AMI release (awslabs#1337) * Update CHANGELOG.md for v20230703 AMI release * Update CHANGELOG.md Co-authored-by: Carter <[email protected]> * Update CHANGELOG.md --------- Co-authored-by: Carter <[email protected]> * Update CHANGELOG.md (awslabs#1338) * Add logging for aws managed csi drivers (awslabs#1336) * Update CHANGELOG.md latest AMI release notes to highlight this was last 1.22 AMI (awslabs#1342) * Removing 1.22 from Makefile (awslabs#1343) * Generate version info for cached images only when is active (awslabs#1341) * Remove region names from us-iso/us-isob credential provider config (awslabs#1344) * Amazon Linux 2023 proof-of-concept (awslabs#1340) * Remove hardcoded pull_cni_from_github var (awslabs#1346) * Remove sonobuoy_e2e_registry (awslabs#1249) * Revert "avoid hard coding provisioner index array" (awslabs#1347) This reverts commit 6c16765. Signed-off-by: Davanum Srinivas <[email protected]> * Update sync-eni-max-pods.yaml role ARN (awslabs#1350) * Add CodeCommit sync action (awslabs#1351) * update core CNI plugins version (awslabs#1308) * Update internal build config (awslabs#1353) * Update binary references (awslabs#1355) * Update CHANGELOG.md for 20230711 AMI release (awslabs#1357) * Enable discard_unpacked_layers by default (awslabs#1360) * Mount bpffs on all supported Kubernetes versions (awslabs#1349) * Cleanup /var/log/audit (awslabs#1363) * Use GitHub bot user as committer/author (awslabs#1366) * Update eni-max-pods.txt (awslabs#1365) * Update CHANGELOG.md for 20230728 AMI release (awslabs#1371) * Update eni-max-pods.txt (awslabs#1373) Co-authored-by: GitHub <[email protected]> * Install latest amazon-ssm-agent from S3 (awslabs#1370) * Do not set KubeletCredentialProviders feature flag for 1.28+ (awslabs#1375) * Fix bug in var doc gen (awslabs#1378) * Generate docs for GitHub Pages (awslabs#1379) * Add write permissions to deploy-docs workflow (awslabs#1381) * Force-push docs to gh-pages (awslabs#1382) * Cache IMDS tokens per-user (awslabs#1386) * Install latest runc 1.1.* (awslabs#1384) * Update eni-max-pods.txt (awslabs#1388) * Update binary build dates (awslabs#1390) * Fetch new IMDS token for every request (awslabs#1395) * Update CHANGELOG for v20230816 (awslabs#1396) * Update eni-max-pods.txt (awslabs#1397) * Update Makefile with latest binaries (awslabs#1403) * Add CI bot (awslabs#1402) * Disable janitor in forks (awslabs#1407) * Add note about bot authorization (awslabs#1406) * noproxy for direct communication to apiserver and timeouts of 3 seconds (awslabs#1393) * Update CHANGELOG.md for 20230825 AMI release (awslabs#1408) * Update CHANGELOG.md for 20230825 AMI release --------- Co-authored-by: Vela WU <[email protected]> * Allow --reserved-cpus kubelet arg to be used (awslabs#1405) * Install kernel-headers, kernel-devel (awslabs#1302) * Handle eventually-consistent PrivateDnsName (awslabs#1383) * Add .git-commit to archivebuild (awslabs#1411) * Use archivebuild-wrapper system (awslabs#1413) * Discover .git-commit from environment (awslabs#1418) * Update eni-max-pods.txt (awslabs#1423) Co-authored-by: GitHub <[email protected]> * Update eni-max-pods.txt (awslabs#1424) Co-authored-by: GitHub <[email protected]> * Require builder instance to use IMDSv2 (awslabs#1422) * Add release note config (awslabs#1426) * Update eni-max-pods.txt (awslabs#1429) Co-authored-by: GitHub <[email protected]> * Use 2023-09-14 binaries, add 1.28 target (awslabs#1431) * Update eni-max-pods.txt (awslabs#1432) Co-authored-by: GitHub <[email protected]> * Set pid_max to 4194304 (awslabs#1434) * Install nerdctl (awslabs#1321) * Update CHANGELOG.md for 20230919 AMI release (awslabs#1439) * Update CHANGELOG.md for 20230919 AMI release Co-authored-by: Carter <[email protected]> --------- Co-authored-by: Carter <[email protected]> * bump latest Kubernetes build target version (awslabs#1440) * fix: Tag cached image with the ECR URI for the target region (awslabs#1442) * Add H100 into gpu clock (awslabs#1447) * bug: incorrect region variable name (awslabs#1449) Co-authored-by: ljosyula <[email protected]> * Update eni-max-pods.txt (awslabs#1452) Co-authored-by: GitHub <[email protected]> * Update CHANGELOG.md for 20231002 AMI release (awslabs#1456) Co-authored-by: ljosyula <[email protected]> * Build with latest binaries by default (awslabs#1391) * Fix region in cached image names (awslabs#1461) * Add 1.28 to CI (awslabs#1464) * Add optional FIPS support (awslabs#1458) * Set remote_folder on all shell provisioners (awslabs#1462) * Pull eksctl supported versions for CI (awslabs#1465) * remove kubernetes versions file and use eksctl supported version list * recognize compression Co-authored-by: Carter <[email protected]> --------- Co-authored-by: Carter <[email protected]> * Add CHANGELOG entry placeholder (awslabs#1466) * Add named arguments to bot commands (awslabs#1463) * get-ecr-uri.sh falls back to use another region in partition if region unconfigured (awslabs#1468) * Force delete CI clusters, don't wait for pod eviction (awslabs#1472) * Add CHANGELOG workflow for new releases (awslabs#1467) * Allow more flexible kernel_version (awslabs#1469) * Add r7i to eni-max-pods.txt (awslabs#1473) Co-authored-by: GitHub <[email protected]> * Fix containerd slice configuration (awslabs#1437) * Correctly tag cached images for us-gov-west-1 FIPS endpoint (awslabs#1476) * Lint space errors (awslabs#1121) * Ignore commit to address space errors (awslabs#1478) * Collect more info about Amazon VPC CNI (awslabs#1245) * Update eni-max-pods.txt (awslabs#1485) Co-authored-by: GitHub <[email protected]> * Fail fast if we cannot determine kubelet version (awslabs#1484) kubelet is likely to fail when there is a mismatch with GLIBC that is in the image vs the one golang uses to build the kubelet. So fail the image right away when this happens as this specific kubelet binary will NOT work in any instance started with this image. ``` 2023-10-25T10:11:38-04:00: amazon-ebs: kubelet: /lib64/libc.so.6: version `GLIBC_2.32' not found (required by kubelet) 2023-10-25T10:11:38-04:00: amazon-ebs: kubelet: /lib64/libc.so.6: version `GLIBC_2.34' not found (required by kubelet) ``` Signed-off-by: Davanum Srinivas <[email protected]> * Persist CI version-info.json as artifact (awslabs#1493) * Add new i4i sizes to eni-max-pods.txt (awslabs#1495) Co-authored-by: GitHub <[email protected]> * Update eni-max-pods.txt (awslabs#1497) Co-authored-by: GitHub <[email protected]> * Drop the FIPS related provisioners for al2023 (awslabs#1499) Signed-off-by: Davanum Srinivas <[email protected]> * Set nerdctl default namespace to k8s.io (awslabs#1488) * Update CHANGELOG.md for release v20231027 (awslabs#1502) Co-authored-by: GitHub <[email protected]> * Skip installing amazon-ssm-agent if already present (awslabs#1501) * Exclude automated eni-max-pods.txt PR's from release notes (awslabs#1498) * Remove extraneous space character (awslabs#1505) * Update CHANGELOG.md (awslabs#1507) * Update CHANGELOG.md to fix docker version (awslabs#1511) * Update docker to the latest 20.10 version (awslabs#1510) * Changelog entry format tweaks (awslabs#1508) * Document how to collect UserData (awslabs#1504) * Update Fluence changelog * Update what kubernetes ami will be build --------- Signed-off-by: Davanum Srinivas <[email protected]> Co-authored-by: Xavier Ryan <[email protected]> Co-authored-by: Carter <[email protected]> Co-authored-by: jacobwolfaws <[email protected]> Co-authored-by: Prasad Shende <[email protected]> Co-authored-by: camrakin <[email protected]> Co-authored-by: Davanum Srinivas <[email protected]> Co-authored-by: Jeffrey Nelson <[email protected]> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Sichaow <[email protected]> Co-authored-by: GitHub <[email protected]> Co-authored-by: Vincent Marguerie <[email protected]> Co-authored-by: Andrew Johnstone <[email protected]> Co-authored-by: Vela WU <[email protected]> Co-authored-by: Vela WU <[email protected]> Co-authored-by: Raghvendra Singh <[email protected]> Co-authored-by: Matthew Wong <[email protected]> Co-authored-by: Nick Baker <[email protected]> Co-authored-by: ddl-retornam <[email protected]> Co-authored-by: Carter <[email protected]> Co-authored-by: Bryant Biggs <[email protected]> Co-authored-by: Laxmi Soumya Josyula <[email protected]> Co-authored-by: ljosyula <[email protected]> Co-authored-by: Alex Schultz <[email protected]> Co-authored-by: Julien Baladier <[email protected]> Co-authored-by: Matt <[email protected]> Co-authored-by: Zoltán Reegn <[email protected]> Co-authored-by: donovanrost <[email protected]> Co-authored-by: guessi <[email protected]> Co-authored-by: pjaudiomv <[email protected]> Co-authored-by: Edmond Ceausu <[email protected]>

* Update CHANGELOG.md for v20230703 AMI release (awslabs#1337) * Update CHANGELOG.md for v20230703 AMI release * Update CHANGELOG.md Co-authored-by: Carter <[email protected]> * Update CHANGELOG.md --------- Co-authored-by: Carter <[email protected]> * Update CHANGELOG.md (awslabs#1338) * Add logging for aws managed csi drivers (awslabs#1336) * Update CHANGELOG.md latest AMI release notes to highlight this was last 1.22 AMI (awslabs#1342) * Removing 1.22 from Makefile (awslabs#1343) * Generate version info for cached images only when is active (awslabs#1341) * Remove region names from us-iso/us-isob credential provider config (awslabs#1344) * Amazon Linux 2023 proof-of-concept (awslabs#1340) * Remove hardcoded pull_cni_from_github var (awslabs#1346) * Remove sonobuoy_e2e_registry (awslabs#1249) * Revert "avoid hard coding provisioner index array" (awslabs#1347) This reverts commit 6c16765. Signed-off-by: Davanum Srinivas <[email protected]> * Update sync-eni-max-pods.yaml role ARN (awslabs#1350) * Add CodeCommit sync action (awslabs#1351) * update core CNI plugins version (awslabs#1308) * Update internal build config (awslabs#1353) * Update binary references (awslabs#1355) * Update CHANGELOG.md for 20230711 AMI release (awslabs#1357) * Enable discard_unpacked_layers by default (awslabs#1360) * Mount bpffs on all supported Kubernetes versions (awslabs#1349) * Cleanup /var/log/audit (awslabs#1363) * Use GitHub bot user as committer/author (awslabs#1366) * Update eni-max-pods.txt (awslabs#1365) * Update CHANGELOG.md for 20230728 AMI release (awslabs#1371) * Update eni-max-pods.txt (awslabs#1373) Co-authored-by: GitHub <[email protected]> * Install latest amazon-ssm-agent from S3 (awslabs#1370) * Do not set KubeletCredentialProviders feature flag for 1.28+ (awslabs#1375) * Fix bug in var doc gen (awslabs#1378) * Generate docs for GitHub Pages (awslabs#1379) * Add write permissions to deploy-docs workflow (awslabs#1381) * Force-push docs to gh-pages (awslabs#1382) * Cache IMDS tokens per-user (awslabs#1386) * Install latest runc 1.1.* (awslabs#1384) * Update eni-max-pods.txt (awslabs#1388) * Update binary build dates (awslabs#1390) * Fetch new IMDS token for every request (awslabs#1395) * Update CHANGELOG for v20230816 (awslabs#1396) * Update eni-max-pods.txt (awslabs#1397) * Update Makefile with latest binaries (awslabs#1403) * Add CI bot (awslabs#1402) * Disable janitor in forks (awslabs#1407) * Add note about bot authorization (awslabs#1406) * noproxy for direct communication to apiserver and timeouts of 3 seconds (awslabs#1393) * Update CHANGELOG.md for 20230825 AMI release (awslabs#1408) * Update CHANGELOG.md for 20230825 AMI release --------- Co-authored-by: Vela WU <[email protected]> * Allow --reserved-cpus kubelet arg to be used (awslabs#1405) * Install kernel-headers, kernel-devel (awslabs#1302) * Handle eventually-consistent PrivateDnsName (awslabs#1383) * Add .git-commit to archivebuild (awslabs#1411) * Use archivebuild-wrapper system (awslabs#1413) * Discover .git-commit from environment (awslabs#1418) * Update eni-max-pods.txt (awslabs#1423) Co-authored-by: GitHub <[email protected]> * Update eni-max-pods.txt (awslabs#1424) Co-authored-by: GitHub <[email protected]> * Require builder instance to use IMDSv2 (awslabs#1422) * Add release note config (awslabs#1426) * Update eni-max-pods.txt (awslabs#1429) Co-authored-by: GitHub <[email protected]> * Use 2023-09-14 binaries, add 1.28 target (awslabs#1431) * Update eni-max-pods.txt (awslabs#1432) Co-authored-by: GitHub <[email protected]> * Set pid_max to 4194304 (awslabs#1434) * Install nerdctl (awslabs#1321) * Update CHANGELOG.md for 20230919 AMI release (awslabs#1439) * Update CHANGELOG.md for 20230919 AMI release Co-authored-by: Carter <[email protected]> --------- Co-authored-by: Carter <[email protected]> * bump latest Kubernetes build target version (awslabs#1440) * fix: Tag cached image with the ECR URI for the target region (awslabs#1442) * Add H100 into gpu clock (awslabs#1447) * bug: incorrect region variable name (awslabs#1449) Co-authored-by: ljosyula <[email protected]> * Update eni-max-pods.txt (awslabs#1452) Co-authored-by: GitHub <[email protected]> * Update CHANGELOG.md for 20231002 AMI release (awslabs#1456) Co-authored-by: ljosyula <[email protected]> * Build with latest binaries by default (awslabs#1391) * Fix region in cached image names (awslabs#1461) * Add 1.28 to CI (awslabs#1464) * Add optional FIPS support (awslabs#1458) * Set remote_folder on all shell provisioners (awslabs#1462) * Pull eksctl supported versions for CI (awslabs#1465) * remove kubernetes versions file and use eksctl supported version list * recognize compression Co-authored-by: Carter <[email protected]> --------- Co-authored-by: Carter <[email protected]> * Add CHANGELOG entry placeholder (awslabs#1466) * Add named arguments to bot commands (awslabs#1463) * get-ecr-uri.sh falls back to use another region in partition if region unconfigured (awslabs#1468) * Force delete CI clusters, don't wait for pod eviction (awslabs#1472) * Add CHANGELOG workflow for new releases (awslabs#1467) * Allow more flexible kernel_version (awslabs#1469) * Add r7i to eni-max-pods.txt (awslabs#1473) Co-authored-by: GitHub <[email protected]> * Fix containerd slice configuration (awslabs#1437) * Correctly tag cached images for us-gov-west-1 FIPS endpoint (awslabs#1476) * Lint space errors (awslabs#1121) * Ignore commit to address space errors (awslabs#1478) * Collect more info about Amazon VPC CNI (awslabs#1245) * Update eni-max-pods.txt (awslabs#1485) Co-authored-by: GitHub <[email protected]> * Fail fast if we cannot determine kubelet version (awslabs#1484) kubelet is likely to fail when there is a mismatch with GLIBC that is in the image vs the one golang uses to build the kubelet. So fail the image right away when this happens as this specific kubelet binary will NOT work in any instance started with this image. ``` 2023-10-25T10:11:38-04:00: amazon-ebs: kubelet: /lib64/libc.so.6: version `GLIBC_2.32' not found (required by kubelet) 2023-10-25T10:11:38-04:00: amazon-ebs: kubelet: /lib64/libc.so.6: version `GLIBC_2.34' not found (required by kubelet) ``` Signed-off-by: Davanum Srinivas <[email protected]> * Persist CI version-info.json as artifact (awslabs#1493) * Add new i4i sizes to eni-max-pods.txt (awslabs#1495) Co-authored-by: GitHub <[email protected]> * Update eni-max-pods.txt (awslabs#1497) Co-authored-by: GitHub <[email protected]> * Drop the FIPS related provisioners for al2023 (awslabs#1499) Signed-off-by: Davanum Srinivas <[email protected]> * Set nerdctl default namespace to k8s.io (awslabs#1488) * Update CHANGELOG.md for release v20231027 (awslabs#1502) Co-authored-by: GitHub <[email protected]> * Skip installing amazon-ssm-agent if already present (awslabs#1501) * Exclude automated eni-max-pods.txt PR's from release notes (awslabs#1498) * Remove extraneous space character (awslabs#1505) * Update CHANGELOG.md (awslabs#1507) * Update CHANGELOG.md to fix docker version (awslabs#1511) * Update docker to the latest 20.10 version (awslabs#1510) * Changelog entry format tweaks (awslabs#1508) * Document how to collect UserData (awslabs#1504) * Update eni-max-pods.txt (awslabs#1518) Co-authored-by: GitHub <[email protected]> * Update CHANGELOG.md for release v20231116 (awslabs#1521) Co-authored-by: GitHub <[email protected]> * Add check for ecr-fips endpoint availability (awslabs#1524) * Miscellaneous fixes from AL2023 testing (awslabs#1528) Signed-off-by: Davanum Srinivas <[email protected]> * fix Permission denied for 99-default.link (awslabs#1529) Signed-off-by: Davanum Srinivas <[email protected]> * Install SSM agent from AL core repo by default (awslabs#1531) * Update to `containerd` 1.7 (awslabs#1516) * Capture logs for EKS Pod Identity Agent (awslabs#1533) * change how aws cli is installed * Update CHANGELOG.md for release v20231201 (awslabs#1538) Co-authored-by: GitHub <[email protected]> * AL2023 networking changes for VPC CNI compatibility (awslabs#1539) * Set containerd LimitNOFILE to recommended value (awslabs#1535) * fix networkd settings (awslabs#1540) * Update get-ecr-uri.sh with ca-west-1 account (awslabs#1542) * Install amazon packer plugin for CI (awslabs#1545) * Fix flag typo in logging (awslabs#1547) * Update CHANGELOG.md for release v20231220 (awslabs#1550) Co-authored-by: GitHub <[email protected]> * Revert "Set containerd LimitNOFILE to recommended value (awslabs#1535)" (awslabs#1552) This reverts commit e098953. * set ssm_agent_version after updating from upstream * Uncomment filtering for circle ci config --------- Signed-off-by: Davanum Srinivas <[email protected]> Co-authored-by: Xavier Ryan <[email protected]> Co-authored-by: Carter <[email protected]> Co-authored-by: jacobwolfaws <[email protected]> Co-authored-by: Prasad Shende <[email protected]> Co-authored-by: camrakin <[email protected]> Co-authored-by: Davanum Srinivas <[email protected]> Co-authored-by: Jeffrey Nelson <[email protected]> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Sichaow <[email protected]> Co-authored-by: GitHub <[email protected]> Co-authored-by: Vincent Marguerie <[email protected]> Co-authored-by: Andrew Johnstone <[email protected]> Co-authored-by: Vela WU <[email protected]> Co-authored-by: Vela WU <[email protected]> Co-authored-by: Raghvendra Singh <[email protected]> Co-authored-by: Matthew Wong <[email protected]> Co-authored-by: Nick Baker <[email protected]> Co-authored-by: ddl-retornam <[email protected]> Co-authored-by: Carter <[email protected]> Co-authored-by: Bryant Biggs <[email protected]> Co-authored-by: Laxmi Soumya Josyula <[email protected]> Co-authored-by: ljosyula <[email protected]> Co-authored-by: Alex Schultz <[email protected]> Co-authored-by: Julien Baladier <[email protected]> Co-authored-by: Matt <[email protected]> Co-authored-by: Zoltán Reegn <[email protected]> Co-authored-by: donovanrost <[email protected]> Co-authored-by: guessi <[email protected]> Co-authored-by: pjaudiomv <[email protected]> Co-authored-by: Edmond Ceausu <[email protected]> Co-authored-by: Joe North <[email protected]> Co-authored-by: Keto D. Zhang <[email protected]>

* Merge with upstream v20231116 (#30) * Update CHANGELOG.md for v20230703 AMI release (awslabs#1337) * Update CHANGELOG.md for v20230703 AMI release * Update CHANGELOG.md Co-authored-by: Carter <[email protected]> * Update CHANGELOG.md --------- Co-authored-by: Carter <[email protected]> * Update CHANGELOG.md (awslabs#1338) * Add logging for aws managed csi drivers (awslabs#1336) * Update CHANGELOG.md latest AMI release notes to highlight this was last 1.22 AMI (awslabs#1342) * Removing 1.22 from Makefile (awslabs#1343) * Generate version info for cached images only when is active (awslabs#1341) * Remove region names from us-iso/us-isob credential provider config (awslabs#1344) * Amazon Linux 2023 proof-of-concept (awslabs#1340) * Remove hardcoded pull_cni_from_github var (awslabs#1346) * Remove sonobuoy_e2e_registry (awslabs#1249) * Revert "avoid hard coding provisioner index array" (awslabs#1347) This reverts commit 6c16765. Signed-off-by: Davanum Srinivas <[email protected]> * Update sync-eni-max-pods.yaml role ARN (awslabs#1350) * Add CodeCommit sync action (awslabs#1351) * update core CNI plugins version (awslabs#1308) * Update internal build config (awslabs#1353) * Update binary references (awslabs#1355) * Update CHANGELOG.md for 20230711 AMI release (awslabs#1357) * Enable discard_unpacked_layers by default (awslabs#1360) * Mount bpffs on all supported Kubernetes versions (awslabs#1349) * Cleanup /var/log/audit (awslabs#1363) * Use GitHub bot user as committer/author (awslabs#1366) * Update eni-max-pods.txt (awslabs#1365) * Update CHANGELOG.md for 20230728 AMI release (awslabs#1371) * Update eni-max-pods.txt (awslabs#1373) Co-authored-by: GitHub <[email protected]> * Install latest amazon-ssm-agent from S3 (awslabs#1370) * Do not set KubeletCredentialProviders feature flag for 1.28+ (awslabs#1375) * Fix bug in var doc gen (awslabs#1378) * Generate docs for GitHub Pages (awslabs#1379) * Add write permissions to deploy-docs workflow (awslabs#1381) * Force-push docs to gh-pages (awslabs#1382) * Cache IMDS tokens per-user (awslabs#1386) * Install latest runc 1.1.* (awslabs#1384) * Update eni-max-pods.txt (awslabs#1388) * Update binary build dates (awslabs#1390) * Fetch new IMDS token for every request (awslabs#1395) * Update CHANGELOG for v20230816 (awslabs#1396) * Update eni-max-pods.txt (awslabs#1397) * Update Makefile with latest binaries (awslabs#1403) * Add CI bot (awslabs#1402) * Disable janitor in forks (awslabs#1407) * Add note about bot authorization (awslabs#1406) * noproxy for direct communication to apiserver and timeouts of 3 seconds (awslabs#1393) * Update CHANGELOG.md for 20230825 AMI release (awslabs#1408) * Update CHANGELOG.md for 20230825 AMI release --------- Co-authored-by: Vela WU <[email protected]> * Allow --reserved-cpus kubelet arg to be used (awslabs#1405) * Install kernel-headers, kernel-devel (awslabs#1302) * Handle eventually-consistent PrivateDnsName (awslabs#1383) * Add .git-commit to archivebuild (awslabs#1411) * Use archivebuild-wrapper system (awslabs#1413) * Discover .git-commit from environment (awslabs#1418) * Update eni-max-pods.txt (awslabs#1423) Co-authored-by: GitHub <[email protected]> * Update eni-max-pods.txt (awslabs#1424) Co-authored-by: GitHub <[email protected]> * Require builder instance to use IMDSv2 (awslabs#1422) * Add release note config (awslabs#1426) * Update eni-max-pods.txt (awslabs#1429) Co-authored-by: GitHub <[email protected]> * Use 2023-09-14 binaries, add 1.28 target (awslabs#1431) * Update eni-max-pods.txt (awslabs#1432) Co-authored-by: GitHub <[email protected]> * Set pid_max to 4194304 (awslabs#1434) * Install nerdctl (awslabs#1321) * Update CHANGELOG.md for 20230919 AMI release (awslabs#1439) * Update CHANGELOG.md for 20230919 AMI release Co-authored-by: Carter <[email protected]> --------- Co-authored-by: Carter <[email protected]> * bump latest Kubernetes build target version (awslabs#1440) * fix: Tag cached image with the ECR URI for the target region (awslabs#1442) * Add H100 into gpu clock (awslabs#1447) * bug: incorrect region variable name (awslabs#1449) Co-authored-by: ljosyula <[email protected]> * Update eni-max-pods.txt (awslabs#1452) Co-authored-by: GitHub <[email protected]> * Update CHANGELOG.md for 20231002 AMI release (awslabs#1456) Co-authored-by: ljosyula <[email protected]> * Build with latest binaries by default (awslabs#1391) * Fix region in cached image names (awslabs#1461) * Add 1.28 to CI (awslabs#1464) * Add optional FIPS support (awslabs#1458) * Set remote_folder on all shell provisioners (awslabs#1462) * Pull eksctl supported versions for CI (awslabs#1465) * remove kubernetes versions file and use eksctl supported version list * recognize compression Co-authored-by: Carter <[email protected]> --------- Co-authored-by: Carter <[email protected]> * Add CHANGELOG entry placeholder (awslabs#1466) * Add named arguments to bot commands (awslabs#1463) * get-ecr-uri.sh falls back to use another region in partition if region unconfigured (awslabs#1468) * Force delete CI clusters, don't wait for pod eviction (awslabs#1472) * Add CHANGELOG workflow for new releases (awslabs#1467) * Allow more flexible kernel_version (awslabs#1469) * Add r7i to eni-max-pods.txt (awslabs#1473) Co-authored-by: GitHub <[email protected]> * Fix containerd slice configuration (awslabs#1437) * Correctly tag cached images for us-gov-west-1 FIPS endpoint (awslabs#1476) * Lint space errors (awslabs#1121) * Ignore commit to address space errors (awslabs#1478) * Collect more info about Amazon VPC CNI (awslabs#1245) * Update eni-max-pods.txt (awslabs#1485) Co-authored-by: GitHub <[email protected]> * Fail fast if we cannot determine kubelet version (awslabs#1484) kubelet is likely to fail when there is a mismatch with GLIBC that is in the image vs the one golang uses to build the kubelet. So fail the image right away when this happens as this specific kubelet binary will NOT work in any instance started with this image. ``` 2023-10-25T10:11:38-04:00: amazon-ebs: kubelet: /lib64/libc.so.6: version `GLIBC_2.32' not found (required by kubelet) 2023-10-25T10:11:38-04:00: amazon-ebs: kubelet: /lib64/libc.so.6: version `GLIBC_2.34' not found (required by kubelet) ``` Signed-off-by: Davanum Srinivas <[email protected]> * Persist CI version-info.json as artifact (awslabs#1493) * Add new i4i sizes to eni-max-pods.txt (awslabs#1495) Co-authored-by: GitHub <[email protected]> * Update eni-max-pods.txt (awslabs#1497) Co-authored-by: GitHub <[email protected]> * Drop the FIPS related provisioners for al2023 (awslabs#1499) Signed-off-by: Davanum Srinivas <[email protected]> * Set nerdctl default namespace to k8s.io (awslabs#1488) * Update CHANGELOG.md for release v20231027 (awslabs#1502) Co-authored-by: GitHub <[email protected]> * Skip installing amazon-ssm-agent if already present (awslabs#1501) * Exclude automated eni-max-pods.txt PR's from release notes (awslabs#1498) * Remove extraneous space character (awslabs#1505) * Update CHANGELOG.md (awslabs#1507) * Update CHANGELOG.md to fix docker version (awslabs#1511) * Update docker to the latest 20.10 version (awslabs#1510) * Changelog entry format tweaks (awslabs#1508) * Document how to collect UserData (awslabs#1504) * Update Fluence changelog * Update what kubernetes ami will be build --------- Signed-off-by: Davanum Srinivas <[email protected]> Co-authored-by: Xavier Ryan <[email protected]> Co-authored-by: Carter <[email protected]> Co-authored-by: jacobwolfaws <[email protected]> Co-authored-by: Prasad Shende <[email protected]> Co-authored-by: camrakin <[email protected]> Co-authored-by: Davanum Srinivas <[email protected]> Co-authored-by: Jeffrey Nelson <[email protected]> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Sichaow <[email protected]> Co-authored-by: GitHub <[email protected]> Co-authored-by: Vincent Marguerie <[email protected]> Co-authored-by: Andrew Johnstone <[email protected]> Co-authored-by: Vela WU <[email protected]> Co-authored-by: Vela WU <[email protected]> Co-authored-by: Raghvendra Singh <[email protected]> Co-authored-by: Matthew Wong <[email protected]> Co-authored-by: Nick Baker <[email protected]> Co-authored-by: ddl-retornam <[email protected]> Co-authored-by: Carter <[email protected]> Co-authored-by: Bryant Biggs <[email protected]> Co-authored-by: Laxmi Soumya Josyula <[email protected]> Co-authored-by: ljosyula <[email protected]> Co-authored-by: Alex Schultz <[email protected]> Co-authored-by: Julien Baladier <[email protected]> Co-authored-by: Matt <[email protected]> Co-authored-by: Zoltán Reegn <[email protected]> Co-authored-by: donovanrost <[email protected]> Co-authored-by: guessi <[email protected]> Co-authored-by: pjaudiomv <[email protected]> Co-authored-by: Edmond Ceausu <[email protected]> * Add awscli to build step (#31) * Update CHANGELOG.md for v20230703 AMI release (awslabs#1337) * Update CHANGELOG.md for v20230703 AMI release * Update CHANGELOG.md Co-authored-by: Carter <[email protected]> * Update CHANGELOG.md --------- Co-authored-by: Carter <[email protected]> * Update CHANGELOG.md (awslabs#1338) * Add logging for aws managed csi drivers (awslabs#1336) * Update CHANGELOG.md latest AMI release notes to highlight this was last 1.22 AMI (awslabs#1342) * Removing 1.22 from Makefile (awslabs#1343) * Generate version info for cached images only when is active (awslabs#1341) * Remove region names from us-iso/us-isob credential provider config (awslabs#1344) * Amazon Linux 2023 proof-of-concept (awslabs#1340) * Remove hardcoded pull_cni_from_github var (awslabs#1346) * Remove sonobuoy_e2e_registry (awslabs#1249) * Revert "avoid hard coding provisioner index array" (awslabs#1347) This reverts commit 6c16765. Signed-off-by: Davanum Srinivas <[email protected]> * Update sync-eni-max-pods.yaml role ARN (awslabs#1350) * Add CodeCommit sync action (awslabs#1351) * update core CNI plugins version (awslabs#1308) * Update internal build config (awslabs#1353) * Update binary references (awslabs#1355) * Update CHANGELOG.md for 20230711 AMI release (awslabs#1357) * Enable discard_unpacked_layers by default (awslabs#1360) * Mount bpffs on all supported Kubernetes versions (awslabs#1349) * Cleanup /var/log/audit (awslabs#1363) * Use GitHub bot user as committer/author (awslabs#1366) * Update eni-max-pods.txt (awslabs#1365) * Update CHANGELOG.md for 20230728 AMI release (awslabs#1371) * Update eni-max-pods.txt (awslabs#1373) Co-authored-by: GitHub <[email protected]> * Install latest amazon-ssm-agent from S3 (awslabs#1370) * Do not set KubeletCredentialProviders feature flag for 1.28+ (awslabs#1375) * Fix bug in var doc gen (awslabs#1378) * Generate docs for GitHub Pages (awslabs#1379) * Add write permissions to deploy-docs workflow (awslabs#1381) * Force-push docs to gh-pages (awslabs#1382) * Cache IMDS tokens per-user (awslabs#1386) * Install latest runc 1.1.* (awslabs#1384) * Update eni-max-pods.txt (awslabs#1388) * Update binary build dates (awslabs#1390) * Fetch new IMDS token for every request (awslabs#1395) * Update CHANGELOG for v20230816 (awslabs#1396) * Update eni-max-pods.txt (awslabs#1397) * Update Makefile with latest binaries (awslabs#1403) * Add CI bot (awslabs#1402) * Disable janitor in forks (awslabs#1407) * Add note about bot authorization (awslabs#1406) * noproxy for direct communication to apiserver and timeouts of 3 seconds (awslabs#1393) * Update CHANGELOG.md for 20230825 AMI release (awslabs#1408) * Update CHANGELOG.md for 20230825 AMI release --------- Co-authored-by: Vela WU <[email protected]> * Allow --reserved-cpus kubelet arg to be used (awslabs#1405) * Install kernel-headers, kernel-devel (awslabs#1302) * Handle eventually-consistent PrivateDnsName (awslabs#1383) * Add .git-commit to archivebuild (awslabs#1411) * Use archivebuild-wrapper system (awslabs#1413) * Discover .git-commit from environment (awslabs#1418) * Update eni-max-pods.txt (awslabs#1423) Co-authored-by: GitHub <[email protected]> * Update eni-max-pods.txt (awslabs#1424) Co-authored-by: GitHub <[email protected]> * Require builder instance to use IMDSv2 (awslabs#1422) * Add release note config (awslabs#1426) * Update eni-max-pods.txt (awslabs#1429) Co-authored-by: GitHub <[email protected]> * Use 2023-09-14 binaries, add 1.28 target (awslabs#1431) * Update eni-max-pods.txt (awslabs#1432) Co-authored-by: GitHub <[email protected]> * Set pid_max to 4194304 (awslabs#1434) * Install nerdctl (awslabs#1321) * Update CHANGELOG.md for 20230919 AMI release (awslabs#1439) * Update CHANGELOG.md for 20230919 AMI release Co-authored-by: Carter <[email protected]> --------- Co-authored-by: Carter <[email protected]> * bump latest Kubernetes build target version (awslabs#1440) * fix: Tag cached image with the ECR URI for the target region (awslabs#1442) * Add H100 into gpu clock (awslabs#1447) * bug: incorrect region variable name (awslabs#1449) Co-authored-by: ljosyula <[email protected]> * Update eni-max-pods.txt (awslabs#1452) Co-authored-by: GitHub <[email protected]> * Update CHANGELOG.md for 20231002 AMI release (awslabs#1456) Co-authored-by: ljosyula <[email protected]> * Build with latest binaries by default (awslabs#1391) * Fix region in cached image names (awslabs#1461) * Add 1.28 to CI (awslabs#1464) * Add optional FIPS support (awslabs#1458) * Set remote_folder on all shell provisioners (awslabs#1462) * Pull eksctl supported versions for CI (awslabs#1465) * remove kubernetes versions file and use eksctl supported version list * recognize compression Co-authored-by: Carter <[email protected]> --------- Co-authored-by: Carter <[email protected]> * Add CHANGELOG entry placeholder (awslabs#1466) * Add named arguments to bot commands (awslabs#1463) * get-ecr-uri.sh falls back to use another region in partition if region unconfigured (awslabs#1468) * Force delete CI clusters, don't wait for pod eviction (awslabs#1472) * Add CHANGELOG workflow for new releases (awslabs#1467) * Allow more flexible kernel_version (awslabs#1469) * Add r7i to eni-max-pods.txt (awslabs#1473) Co-authored-by: GitHub <[email protected]> * Fix containerd slice configuration (awslabs#1437) * Correctly tag cached images for us-gov-west-1 FIPS endpoint (awslabs#1476) * Lint space errors (awslabs#1121) * Ignore commit to address space errors (awslabs#1478) * Collect more info about Amazon VPC CNI (awslabs#1245) * Update eni-max-pods.txt (awslabs#1485) Co-authored-by: GitHub <[email protected]> * Fail fast if we cannot determine kubelet version (awslabs#1484) kubelet is likely to fail when there is a mismatch with GLIBC that is in the image vs the one golang uses to build the kubelet. So fail the image right away when this happens as this specific kubelet binary will NOT work in any instance started with this image. ``` 2023-10-25T10:11:38-04:00: amazon-ebs: kubelet: /lib64/libc.so.6: version `GLIBC_2.32' not found (required by kubelet) 2023-10-25T10:11:38-04:00: amazon-ebs: kubelet: /lib64/libc.so.6: version `GLIBC_2.34' not found (required by kubelet) ``` Signed-off-by: Davanum Srinivas <[email protected]> * Persist CI version-info.json as artifact (awslabs#1493) * Add new i4i sizes to eni-max-pods.txt (awslabs#1495) Co-authored-by: GitHub <[email protected]> * Update eni-max-pods.txt (awslabs#1497) Co-authored-by: GitHub <[email protected]> * Drop the FIPS related provisioners for al2023 (awslabs#1499) Signed-off-by: Davanum Srinivas <[email protected]> * Set nerdctl default namespace to k8s.io (awslabs#1488) * Update CHANGELOG.md for release v20231027 (awslabs#1502) Co-authored-by: GitHub <[email protected]> * Skip installing amazon-ssm-agent if already present (awslabs#1501) * Exclude automated eni-max-pods.txt PR's from release notes (awslabs#1498) * Remove extraneous space character (awslabs#1505) * Update CHANGELOG.md (awslabs#1507) * Update CHANGELOG.md to fix docker version (awslabs#1511) * Update docker to the latest 20.10 version (awslabs#1510) * Changelog entry format tweaks (awslabs#1508) * Document how to collect UserData (awslabs#1504) * Update eni-max-pods.txt (awslabs#1518) Co-authored-by: GitHub <[email protected]> * Update CHANGELOG.md for release v20231116 (awslabs#1521) Co-authored-by: GitHub <[email protected]> * Add check for ecr-fips endpoint availability (awslabs#1524) * Miscellaneous fixes from AL2023 testing (awslabs#1528) Signed-off-by: Davanum Srinivas <[email protected]> * fix Permission denied for 99-default.link (awslabs#1529) Signed-off-by: Davanum Srinivas <[email protected]> * Install SSM agent from AL core repo by default (awslabs#1531) * Update to `containerd` 1.7 (awslabs#1516) * Capture logs for EKS Pod Identity Agent (awslabs#1533) * change how aws cli is installed * Update CHANGELOG.md for release v20231201 (awslabs#1538) Co-authored-by: GitHub <[email protected]> * AL2023 networking changes for VPC CNI compatibility (awslabs#1539) * Set containerd LimitNOFILE to recommended value (awslabs#1535) * fix networkd settings (awslabs#1540) * Update get-ecr-uri.sh with ca-west-1 account (awslabs#1542) * Install amazon packer plugin for CI (awslabs#1545) * Fix flag typo in logging (awslabs#1547) * Update CHANGELOG.md for release v20231220 (awslabs#1550) Co-authored-by: GitHub <[email protected]> * Revert "Set containerd LimitNOFILE to recommended value (awslabs#1535)" (awslabs#1552) This reverts commit e098953. * set ssm_agent_version after updating from upstream * Uncomment filtering for circle ci config --------- Signed-off-by: Davanum Srinivas <[email protected]> Co-authored-by: Xavier Ryan <[email protected]> Co-authored-by: Carter <[email protected]> Co-authored-by: jacobwolfaws <[email protected]> Co-authored-by: Prasad Shende <[email protected]> Co-authored-by: camrakin <[email protected]> Co-authored-by: Davanum Srinivas <[email protected]> Co-authored-by: Jeffrey Nelson <[email protected]> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Sichaow <[email protected]> Co-authored-by: GitHub <[email protected]> Co-authored-by: Vincent Marguerie <[email protected]> Co-authored-by: Andrew Johnstone <[email protected]> Co-authored-by: Vela WU <[email protected]> Co-authored-by: Vela WU <[email protected]> Co-authored-by: Raghvendra Singh <[email protected]> Co-authored-by: Matthew Wong <[email protected]> Co-authored-by: Nick Baker <[email protected]> Co-authored-by: ddl-retornam <[email protected]> Co-authored-by: Carter <[email protected]> Co-authored-by: Bryant Biggs <[email protected]> Co-authored-by: Laxmi Soumya Josyula <[email protected]> Co-authored-by: ljosyula <[email protected]> Co-authored-by: Alex Schultz <[email protected]> Co-authored-by: Julien Baladier <[email protected]> Co-authored-by: Matt <[email protected]> Co-authored-by: Zoltán Reegn <[email protected]> Co-authored-by: donovanrost <[email protected]> Co-authored-by: guessi <[email protected]> Co-authored-by: pjaudiomv <[email protected]> Co-authored-by: Edmond Ceausu <[email protected]> Co-authored-by: Joe North <[email protected]> Co-authored-by: Keto D. Zhang <[email protected]> --------- Signed-off-by: Davanum Srinivas <[email protected]> Co-authored-by: Xavier Ryan <[email protected]> Co-authored-by: Carter <[email protected]> Co-authored-by: jacobwolfaws <[email protected]> Co-authored-by: Prasad Shende <[email protected]> Co-authored-by: camrakin <[email protected]> Co-authored-by: Davanum Srinivas <[email protected]> Co-authored-by: Jeffrey Nelson <[email protected]> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Sichaow <[email protected]> Co-authored-by: GitHub <[email protected]> Co-authored-by: Vincent Marguerie <[email protected]> Co-authored-by: Andrew Johnstone <[email protected]> Co-authored-by: Vela WU <[email protected]> Co-authored-by: Vela WU <[email protected]> Co-authored-by: Raghvendra Singh <[email protected]> Co-authored-by: Matthew Wong <[email protected]> Co-authored-by: Nick Baker <[email protected]> Co-authored-by: ddl-retornam <[email protected]> Co-authored-by: Carter <[email protected]> Co-authored-by: Bryant Biggs <[email protected]> Co-authored-by: Laxmi Soumya Josyula <[email protected]> Co-authored-by: ljosyula <[email protected]> Co-authored-by: Alex Schultz <[email protected]> Co-authored-by: Julien Baladier <[email protected]> Co-authored-by: Matt <[email protected]> Co-authored-by: Zoltán Reegn <[email protected]> Co-authored-by: donovanrost <[email protected]> Co-authored-by: guessi <[email protected]> Co-authored-by: pjaudiomv <[email protected]> Co-authored-by: Edmond Ceausu <[email protected]> Co-authored-by: Joe North <[email protected]> Co-authored-by: Keto D. Zhang <[email protected]>

dims force-pushed the tolerate-al-2023-changes branch from 0b5da24 to db1108e Compare July 6, 2023 18:26

cartermckinnon requested changes Jul 6, 2023

View reviewed changes

scripts/upgrade_kernel.sh Outdated Show resolved Hide resolved

Makefile Show resolved Hide resolved

Makefile Outdated Show resolved Hide resolved

scripts/install-worker.sh Show resolved Hide resolved

scripts/validate.sh Outdated Show resolved Hide resolved

cartermckinnon mentioned this pull request Jul 6, 2023

[WIP] Amazon Linux 2023 proof-of-concept #1212

Closed