-
Notifications
You must be signed in to change notification settings - Fork 1.4k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
* EMR on EKS example update with multiple node groups * EMR on EKS example update with multiple node groups * EMR on EKS example with updated config * EMR on EKS example with updated config * removed redundant cluster version from locals * New add-on for cluster proportional autoscaler * Added condition for coredns helm config
- Loading branch information
1 parent
fdf0da7
commit e80009e
Showing
45 changed files
with
101,072 additions
and
617 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,81 @@ | ||
# Horizontal cluster-proportional-autoscaler container | ||
|
||
Horizontal cluster-proportional-autoscaler watches over the number of schedulable nodes and cores of the cluster and resizes the number of replicas for the required resource. This functionality may be desirable for applications that need to be autoscaled with the size of the cluster, such as CoreDNS and other services that scale with the number of nodes/pods in the cluster. | ||
|
||
The [cluster-proportional-autoscaler](https://github.com/kubernetes-sigs/cluster-proportional-autoscaler) helps to scale the applications using deployment or replicationcontroller or replicaset. This is an alternative solution to Horizontal Pod Autoscaling. | ||
It is typically installed as a **Deployment** in your cluster. | ||
|
||
## Usage | ||
|
||
This add-on requires both `enable_coredns_autoscaler` and `coredns_autoscaler_helm_config` as mandatory fields. | ||
|
||
[cluster-proportional-autoscaler](https://github.com/aws-ia/terraform-aws-eks-blueprints/tree/main/modules/kubernetes-addons/cluster-proportional-autoscaler) can be deployed by enabling the add-on via the following. | ||
|
||
The example shows how to enable `cluster-proportional-autoscaler` for `CoreDNS Deployment`. CoreDNS deployment is not configured with HPA. So, this add-on helps to scale CoreDNS Add-on according to the size of the nodes and cores. | ||
|
||
This Add-on can be used to scale any application with Deployment objects. | ||
|
||
```hcl | ||
enable_coredns_autoscaler = true | ||
coredns_autoscaler_helm_config = { | ||
name = "cluster-proportional-autoscaler" | ||
chart = "cluster-proportional-autoscaler" | ||
repository = "https://kubernetes-sigs.github.io/cluster-proportional-autoscaler" | ||
version = "1.0.0" | ||
namespace = "kube-system" | ||
timeout = "300" | ||
values = [ | ||
<<-EOT | ||
nameOverride: kube-dns-autoscaler | ||
# Formula for controlling the replicas. Adjust according to your needs | ||
# replicas = max( ceil( cores * 1/coresPerReplica ) , ceil( nodes * 1/nodesPerReplica ) ) | ||
config: | ||
linear: | ||
coresPerReplica: 256 | ||
nodesPerReplica: 16 | ||
min: 1 | ||
max: 100 | ||
preventSinglePointFailure: true | ||
includeUnschedulableNodes: true | ||
# Target to scale. In format: deployment/*, replicationcontroller/* or replicaset/* (not case sensitive). | ||
options: | ||
target: deployment/coredns # Notice the target as `deployment/coredns` | ||
serviceAccount: | ||
create: true | ||
name: kube-dns-autoscaler | ||
podSecurityContext: | ||
seccompProfile: | ||
type: RuntimeDefault | ||
supplementalGroups: [ 65534 ] | ||
fsGroup: 65534 | ||
resources: | ||
limits: | ||
cpu: 100m | ||
memory: 128Mi | ||
requests: | ||
cpu: 100m | ||
memory: 128Mi | ||
tolerations: | ||
- key: "CriticalAddonsOnly" | ||
operator: "Exists" | ||
description = "Cluster Proportional Autoscaler for CoreDNS Service" | ||
EOT | ||
] | ||
} | ||
``` | ||
|
||
### GitOps Configuration | ||
|
||
The following properties are made available for use when managing the add-on via GitOps. | ||
|
||
``` | ||
corednsAutoscaler = { | ||
enable = true | ||
} | ||
``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
30 changes: 0 additions & 30 deletions
30
examples/analytics/emr-on-eks/examples/create_emr_virtual_cluster_for_eks.sh
This file was deleted.
Oops, something went wrong.
17 changes: 0 additions & 17 deletions
17
examples/analytics/emr-on-eks/examples/delete_emr_virtual_cluster_for_eks.sh
This file was deleted.
Oops, something went wrong.
31 changes: 0 additions & 31 deletions
31
examples/analytics/emr-on-eks/examples/spark-execute/1-basic-spark-job.sh
This file was deleted.
Oops, something went wrong.
49 changes: 0 additions & 49 deletions
49
examples/analytics/emr-on-eks/examples/spark-execute/2-spark-job-with-logs.sh
This file was deleted.
Oops, something went wrong.
Oops, something went wrong.