Skip to content

Commit

Permalink
Adding option to enable autoscaling nodepools and regional clusters.
Browse files Browse the repository at this point in the history
  • Loading branch information
chiayi committed Nov 17, 2022
1 parent b8eb48c commit 6362582
Show file tree
Hide file tree
Showing 126 changed files with 6,414 additions and 2,130 deletions.
17 changes: 15 additions & 2 deletions .golangci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ run:
# from this option's value:
# vendor$, third_party$, testdata$, examples$, Godeps$, builtin$
skip-dirs:
- test/sdk/restapi

# which files to skip: they will be analyzed, but issues from them
# won't be reported. Default value is empty list, but there is
Expand All @@ -47,7 +48,6 @@ output:
linters:
enable:
- bodyclose
- deadcode
- dupl
- exportloopref
- goconst
Expand All @@ -60,11 +60,16 @@ linters:
- nakedret
- revive
- staticcheck
- structcheck
- unconvert
- unparam

linters-settings:
govet:
disable-all: false
enable-all: true
disable:
# extremely noisy, also against common Go style in most cases
- shadow
gocritic:
enabled-tags:
- performance
Expand All @@ -89,3 +94,11 @@ issues:
exclude-use-default: false
exclude:
- Using the variable on range scope .* in function literal
exclude-rules:
# Skip fieldalignment checks on:
# - tests: memory footprint doesn't matter
# - cmd/: we assume these are one-off config entries
# - pkg/apis: Keep strong convention on TypeMeta/ObjectMeta/Spec/Status
- path: '(.+)_test\.go|^test/|^cmd/.*|^pkg/apis/.*'
# fieldalignment is in the `govet` linter, so exclude based on text instead of all of govet
text: '^fieldalignment: .*'
42 changes: 28 additions & 14 deletions CHANGELOG.md

Large diffs are not rendered by default.

3 changes: 2 additions & 1 deletion build/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@
#

# base version target. This is usually the next release.
base_version = 1.27.0
base_version = 1.28.0

# agones image release registry
release_registry = gcr.io/agones-images
Expand All @@ -43,6 +43,7 @@ KUBECONFIG ?= ~/.kube/config
# The (gcloud) test cluster that is being worked against
GCP_CLUSTER_NAME ?= test-cluster
GCP_CLUSTER_ZONE ?= us-west1-c
GCP_CLUSTER_LOCATION ?= $(GCP_CLUSTER_ZONE)
GCP_BUCKET_CHARTS ?= agones-chart
# the profile to use when developing on minikube
MINIKUBE_PROFILE ?= agones
Expand Down
10 changes: 9 additions & 1 deletion build/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,7 @@ Table of Contents
* [make build-images](#make-build-images)
* [make build-sdks](#make-build-sdks)
* [make build-sdk](#make-build-sdk)
* [make run-sdk-command](#make-run-sdk-command)
* [make run-sdk-conformance-tests](#make-run-sdk-conformance-tests)
* [make clean-sdk-conformance-tests](#make-clean-sdk-conformance-tests)
* [make test](#make-test)
Expand Down Expand Up @@ -198,7 +199,10 @@ See the table below for available customizations :
| Parameter | Description | Default |
|------------------------------------------------|---------------------------------------------------------------------------------------|-----------------|
| `GCP_CLUSTER_NAME` | The name of the cluster | `test-cluster` |
| `GCP_CLUSTER_ZONE` | The name of the Google Compute Engine zone in which the cluster will resides. | `us-west1-c` |
| `GCP_CLUSTER_ZONE` or `GCP_CLUSTER_LOCATION` | The name of the Google Compute Engine zone/location in which the cluster will resides | `us-west1-c` |
| `GCP_CLUSTER_NODEPOOL_AUTOSCALE` | Whether or not to enable autoscaling on game server nodepool | `false` |
| `GCP_CLUSTER_NODEPOOL_MIN_NODECOUNT` | The number of minimum nodes if autoscale is enabled | `1` |
| `GCP_CLUSTER_NODEPOOL_MAX_NODECOUNT` | The number of maximum nodes if autoscale is enabled | `5` |
| `GCP_CLUSTER_NODEPOOL_INITIALNODECOUNT` | The number of nodes to create in this cluster. | `4` |
| `GCP_CLUSTER_NODEPOOL_MACHINETYPE` | The name of a Google Compute Engine machine type. | `e2-standard-4` |
| `GCP_CLUSTER_NODEPOOL_ENABLEIMAGESTREAMING` | Whether or not to enable image streaming for the `"default"` node pool in the cluster | `true` |
Expand Down Expand Up @@ -480,6 +484,10 @@ Build all the sdks required for Agones
Next command `make build-sdk SDK_FOLDER=[SDK_TYPE]` will build SDK of `SDK_TYPE`.
For instance, in order to build the cpp sdk static and dynamic libraries (linux libraries only) use `SDK_FOLDER=cpp`

#### `make run-sdk-command`
Next command `make run-sdk-command COMMAND=[COMMAND] SDK_FOLDER=[SDK_TYPE]` will execute command for `SDK_TYPE`.
For instance, in order to generate swagger codes when you change swagger.json definition, use `make run-sdk-command COMMAND=gen SDK_FOLDER=restapi`

#### `make run-sdk-conformance-local`
Run Agones sidecar which would wait for all requests from the SDK client.
Note that annotation should contain UID and label should contain CreationTimestamp values to pass the test.
Expand Down
2 changes: 1 addition & 1 deletion build/build-image/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -88,7 +88,7 @@ RUN rm -f /etc/apt/sources.list.d/chris-lea-node_js-*.list
RUN rm -f /etc/apt/sources.list.d/chris-lea-node_js-*.list.save

ARG KEYRING=/usr/share/keyrings/nodesource.gpg
ARG VERSION=node_16.x
ARG VERSION=node_18.x

RUN curl -fsSL https://deb.nodesource.com/gpgkey/nodesource.gpg.key | gpg --dearmor | tee "$KEYRING" >/dev/null
RUN gpg --no-default-keyring --keyring "$KEYRING" --list-keys
Expand Down
17 changes: 6 additions & 11 deletions test/load/Dockerfile → build/build-sdk-images/cpp/shell.sh
Original file line number Diff line number Diff line change
@@ -1,4 +1,6 @@
# Copyright 2018 Google LLC All Rights Reserved.
#!/usr/bin/env bash

# Copyright 2022 Google LLC All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
Expand All @@ -12,13 +14,6 @@
# See the License for the specific language governing permissions and
# limitations under the License.

FROM hopsoft/graphite-statsd

# Locust
RUN pip install locustio
EXPOSE 8089 5557 5558

RUN mkdir /etc/service/locust
COPY /locust-files ./
COPY /run.sh /etc/service/locust/run
RUN chmod +x /etc/service/locust/run
set -ex
cd ./sdks/cpp
bash
2 changes: 1 addition & 1 deletion build/build-sdk-images/node/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ RUN rm -f /etc/apt/sources.list.d/chris-lea-node_js-*.list
RUN rm -f /etc/apt/sources.list.d/chris-lea-node_js-*.list.save

ARG KEYRING=/usr/share/keyrings/nodesource.gpg
ARG VERSION=node_16.x
ARG VERSION=node_18.x

RUN curl -fsSL https://deb.nodesource.com/gpgkey/nodesource.gpg.key | gpg --dearmor | tee "$KEYRING" >/dev/null
RUN gpg --no-default-keyring --keyring "$KEYRING" --list-keys
Expand Down
2 changes: 1 addition & 1 deletion build/build-sdk-images/restapi/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ RUN rm -f /etc/apt/sources.list.d/chris-lea-node_js-*.list
RUN rm -f /etc/apt/sources.list.d/chris-lea-node_js-*.list.save

ARG KEYRING=/usr/share/keyrings/nodesource.gpg
ARG VERSION=node_16.x
ARG VERSION=node_18.x

RUN curl -fsSL https://deb.nodesource.com/gpgkey/nodesource.gpg.key | gpg --dearmor | tee "$KEYRING" >/dev/null
RUN gpg --no-default-keyring --keyring "$KEYRING" --list-keys
Expand Down
20 changes: 0 additions & 20 deletions build/build-sdk-images/restapi/build-sdk-test.sh

This file was deleted.

2 changes: 0 additions & 2 deletions build/build-sdk-images/restapi/clean.sh
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,4 @@
# limitations under the License.

set -ex
rm -rf /go/src/agones.dev/agones/test/sdk/restapi/swagger
rm -rf /go/src/agones.dev/agones/test/sdk/restapi/alpha/swagger
rm /go/src/agones.dev/agones/test/sdk/restapi/http-api-test.go || true
45 changes: 45 additions & 0 deletions build/build-sdk-images/restapi/gen.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
#!/usr/bin/env bash

# Copyright 2022 Google LLC All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

header() {
cat /go/src/agones.dev/agones/build/boilerplate.go.txt "$1" | sponge "$1"
}

wget -q https://repo1.maven.org/maven2/io/swagger/codegen/v3/swagger-codegen-cli/3.0.35/swagger-codegen-cli-3.0.35.jar -O /tmp/swagger-codegen-cli.jar
java -jar /tmp/swagger-codegen-cli.jar generate -i /go/src/agones.dev/agones/sdks/swagger/sdk.swagger.json -l go -o /go/src/agones.dev/agones/test/sdk/restapi/swagger
java -jar /tmp/swagger-codegen-cli.jar generate -i /go/src/agones.dev/agones/sdks/swagger/alpha.swagger.json -l go -o /go/src/agones.dev/agones/test/sdk/restapi/alpha/swagger

# remove un-used files
rm -rf /go/src/agones.dev/agones/test/sdk/restapi/swagger/.*
rm -rf /go/src/agones.dev/agones/test/sdk/restapi/swagger/*.md
rm -rf /go/src/agones.dev/agones/test/sdk/restapi/swagger/*.sh
rm -rf /go/src/agones.dev/agones/test/sdk/restapi/swagger/docs
rm -rf /go/src/agones.dev/agones/test/sdk/restapi/swagger/api
rm -rf /go/src/agones.dev/agones/test/sdk/restapi/alpha/swagger/.*
rm -rf /go/src/agones.dev/agones/test/sdk/restapi/alpha/swagger/*.md
rm -rf /go/src/agones.dev/agones/test/sdk/restapi/alpha/swagger/*.sh
rm -rf /go/src/agones.dev/agones/test/sdk/restapi/alpha/swagger/docs
rm -rf /go/src/agones.dev/agones/test/sdk/restapi/alpha/swagger/api

for file in `ls /go/src/agones.dev/agones/test/sdk/restapi/swagger`
do
header /go/src/agones.dev/agones/test/sdk/restapi/swagger/${file}
done

for alpha in `ls /go/src/agones.dev/agones/test/sdk/restapi/alpha/swagger`
do
header /go/src/agones.dev/agones/test/sdk/restapi/alpha/swagger/${alpha}
done
4 changes: 2 additions & 2 deletions build/build-sdk-images/tool/base/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@
# Base images for SDKs Bump: 2
#

FROM debian:buster
FROM debian:bullseye

RUN apt-get update && apt-get install -y \
build-essential autoconf libtool git pkg-config curl \
Expand All @@ -31,7 +31,7 @@ RUN git clone --recurse-submodules -b $GRPC_RELEASE_TAG --depth 1 --shallow-subm
mkdir -p cmake/build && \
cd cmake/build && \
cmake -DgRPC_INSTALL=ON -DgRPC_BUILD_TESTS=OFF ../.. && \
make -j && make install
make -j$(nproc) && make install

WORKDIR /go/src/agones.dev/agones

Expand Down
13 changes: 9 additions & 4 deletions build/includes/google-cloud.mk
Original file line number Diff line number Diff line change
Expand Up @@ -27,10 +27,17 @@ gcloud-init: ensure-build-config
gcloud-test-cluster: GCP_CLUSTER_NODEPOOL_INITIALNODECOUNT ?= 4
gcloud-test-cluster: GCP_CLUSTER_NODEPOOL_MACHINETYPE ?= e2-standard-4
gcloud-test-cluster: GCP_CLUSTER_NODEPOOL_ENABLEIMAGESTREAMING ?= true
gcloud-test-cluster: GCP_CLUSTER_NODEPOOL_AUTOSCALE ?= false
gcloud-test-cluster: GCP_CLUSTER_NODEPOOL_MIN_NODECOUNT ?= 1
gcloud-test-cluster: GCP_CLUSTER_NODEPOOL_MAX_NODECOUNT ?= 5
gcloud-test-cluster: GCP_CLUSTER_NODEPOOL_WINDOWSINITIALNODECOUNT ?= 0
gcloud-test-cluster: GCP_CLUSTER_NODEPOOL_WINDOWSMACHINETYPE ?= e2-standard-4
gcloud-test-cluster: $(ensure-build-image)
$(MAKE) gcloud-terraform-cluster GCP_TF_CLUSTER_NAME="$(GCP_CLUSTER_NAME)" GCP_CLUSTER_ZONE="$(GCP_CLUSTER_ZONE)" \
$(MAKE) gcloud-terraform-cluster GCP_TF_CLUSTER_NAME="$(GCP_CLUSTER_NAME)" \
GCP_CLUSTER_LOCATION="$(GCP_CLUSTER_LOCATION)" \
GCP_CLUSTER_AUTOSCALE="$(GCP_CLUSTER_NODEPOOL_AUTOSCALE)" \
GCP_CLUSTER_MIN_NODECOUNT="$(GCP_CLUSTER_NODEPOOL_MIN_NODECOUNT)" \
GCP_CLUSTER_MAX_NODECOUNT="$(GCP_CLUSTER_NODEPOOL_MAX_NODECOUNT)" \
GCP_CLUSTER_NODEPOOL_INITIALNODECOUNT="$(GCP_CLUSTER_NODEPOOL_INITIALNODECOUNT)" \
GCP_CLUSTER_NODEPOOL_MACHINETYPE="$(GCP_CLUSTER_NODEPOOL_MACHINETYPE)" \
GCP_CLUSTER_NODEPOOL_ENABLEIMAGESTREAMING="$(GCP_CLUSTER_NODEPOOL_ENABLEIMAGESTREAMING)" \
Expand Down Expand Up @@ -73,9 +80,7 @@ clean-gcloud-prow-build-cluster: $(ensure-build-image)
# Pulls down authentication information for kubectl against a cluster, name can be specified through GCP_CLUSTER_NAME
# (defaults to 'test-cluster')
gcloud-auth-cluster: $(ensure-build-image)
docker run --rm $(common_mounts) $(build_tag) gcloud config set container/cluster $(GCP_CLUSTER_NAME)
docker run --rm $(common_mounts) $(build_tag) gcloud config set compute/zone $(GCP_CLUSTER_ZONE)
docker run --rm $(common_mounts) $(build_tag) gcloud container clusters get-credentials $(GCP_CLUSTER_NAME)
docker run --rm $(common_mounts) $(build_tag) gcloud container clusters get-credentials $(GCP_CLUSTER_NAME) --zone $(GCP_CLUSTER_LOCATION)

# authenticate our docker configuration so that you can do a docker push directly
# to the gcr.io repository
Expand Down
18 changes: 16 additions & 2 deletions build/includes/terraform.mk
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,9 @@ terraform-clean:
gcloud-terraform-cluster: GCP_CLUSTER_NODEPOOL_INITIALNODECOUNT ?= 4
gcloud-terraform-cluster: GCP_CLUSTER_NODEPOOL_MACHINETYPE ?= e2-standard-4
gcloud-terraform-cluster: GCP_CLUSTER_NODEPOOL_ENABLEIMAGESTREAMING ?= true
gcloud-terraform-cluster: GCP_CLUSTER_NODEPOOL_AUTOSCALE ?= false
gcloud-terraform-cluster: GCP_CLUSTER_NODEPOOL_MIN_NODECOUNT ?= 1
gcloud-terraform-cluster: GCP_CLUSTER_NODEPOOL_MAX_NODECOUNT ?= 5
gcloud-terraform-cluster: GCP_CLUSTER_NODEPOOL_WINDOWSINITIALNODECOUNT ?= 0
gcloud-terraform-cluster: GCP_CLUSTER_NODEPOOL_WINDOWSMACHINETYPE ?= e2-standard-4
gcloud-terraform-cluster: AGONES_VERSION ?= ''
Expand All @@ -53,8 +56,12 @@ gcloud-terraform-cluster:
-var name=$(GCP_TF_CLUSTER_NAME) -var machine_type="$(GCP_CLUSTER_NODEPOOL_MACHINETYPE)" \
-var values_file="" \
-var feature_gates=$(FEATURE_GATES) \
-var zone="$(GCP_CLUSTER_ZONE)" -var project="$(GCP_PROJECT)" \
-var project="$(GCP_PROJECT)" \
-var location="$(GCP_CLUSTER_LOCATION)" \
-var log_level="$(LOG_LEVEL)" \
-var autoscale=$(GCP_CLUSTER_NODEPOOL_AUTOSCALE) \
-var min_node_count=$(GCP_CLUSTER_NODEPOOL_MIN_NODECOUNT) \
-var max_node_count=$(GCP_CLUSTER_NODEPOOL_MAX_NODECOUNT) \
-var node_count=$(GCP_CLUSTER_NODEPOOL_INITIALNODECOUNT) \
-var enable_image_streaming=$(GCP_CLUSTER_NODEPOOL_ENABLEIMAGESTREAMING) \
-var windows_node_count=$(GCP_CLUSTER_NODEPOOL_WINDOWSINITIALNODECOUNT) \
Expand All @@ -66,6 +73,9 @@ gcloud-terraform-cluster:
# Unifies previous `make gcloud-test-cluster` and `make install` targets
gcloud-terraform-install: GCP_CLUSTER_NODEPOOL_INITIALNODECOUNT ?= 4
gcloud-terraform-install: GCP_CLUSTER_NODEPOOL_MACHINETYPE ?= e2-standard-4
gcloud-terraform-install: GCP_CLUSTER_NODEPOOL_AUTOSCALE ?= false
gcloud-terraform-install: GCP_CLUSTER_NODEPOOL_MIN_NODECOUNT ?= 1
gcloud-terraform-install: GCP_CLUSTER_NODEPOOL_MAX_NODECOUNT ?= 5
gcloud-terraform-install: GCP_CLUSTER_NODEPOOL_WINDOWSINITIALNODECOUNT ?= 0
gcloud-terraform-install: GCP_CLUSTER_NODEPOOL_WINDOWSMACHINETYPE ?= e2-standard-4
gcloud-terraform-install: ALWAYS_PULL_SIDECAR := true
Expand All @@ -87,8 +97,12 @@ gcloud-terraform-install:
-var crd_cleanup="$(CRD_CLEANUP)" \
-var chart="../../../install/helm/agones/" \
-var name=$(GCP_TF_CLUSTER_NAME) -var machine_type="$(GCP_CLUSTER_NODEPOOL_MACHINETYPE)" \
-var zone=$(GCP_CLUSTER_ZONE) -var project=$(GCP_PROJECT) \
-var project=$(GCP_PROJECT) \
-var location=$(GCP_CLUSTER_LOCATION) \
-var log_level=$(LOG_LEVEL) \
-var autoscale=$(GCP_CLUSTER_NODEPOOL_AUTOSCALE) \
-var min_node_count=$(GCP_CLUSTER_NODEPOOL_MIN_NODECOUNT) \
-var max_node_count=$(GCP_CLUSTER_NODEPOOL_MAX_NODECOUNT) \
-var feature_gates=$(FEATURE_GATES) \
-var node_count=$(GCP_CLUSTER_NODEPOOL_INITIALNODECOUNT) \
-var windows_node_count=$(GCP_CLUSTER_NODEPOOL_WINDOWSINITIALNODECOUNT) \
Expand Down
25 changes: 23 additions & 2 deletions build/terraform/gke/module.tf
Original file line number Diff line number Diff line change
Expand Up @@ -63,9 +63,14 @@ variable "ping_service_type" {
default = "LoadBalancer"
}

variable "zone" {
variable "location" {
default = "us-west1-c"
description = "The GCP zone to create the cluster in"
description = "The GCP location to create the cluster in"
}

variable "zone" {
default = ""
description = "The GCP zone to create the cluster in (deprecated, use `location`)"
}

variable "pull_policy" {
Expand All @@ -88,6 +93,18 @@ variable "log_level" {
default = "info"
}

variable "autoscale" {
default = "false"
}

variable "min_node_count" {
default = "1"
}

variable "max_node_count" {
default = "5"
}

// Note: This is the number of gameserver nodes. The Agones module will automatically create an additional
// two node pools with 1 node each for "agones-system" and "agones-metrics".
variable "node_count" {
Expand Down Expand Up @@ -116,6 +133,7 @@ module "gke_cluster" {

cluster = {
"name" = var.name
"location" = var.location
"zone" = var.zone
"machineType" = var.machine_type
"initialNodeCount" = var.node_count
Expand All @@ -124,6 +142,9 @@ module "gke_cluster" {
"windowsInitialNodeCount" = var.windows_node_count
"project" = var.project
"network" = var.network
"autoscale" = var.autoscale
"minNodeCount" = var.min_node_count
"maxNodeCount" = var.max_node_count
}
}

Expand Down
Loading

0 comments on commit 6362582

Please sign in to comment.