Skip to content

Commit

Permalink
[Breaking Change] Move GameServerAllocation to an API Extension Server.
Browse files Browse the repository at this point in the history
This moves the implementation of GameServerAllocation (GSA) to a
[Kubernetes API Extension](https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/apiserver-aggregation/)
instead of using CRDs. This was essentially done for performance reasons, but to break it down:

1. GSA is now create only. Since we had no need for GSA storage, and don't want the performance hit.
1. This removes the mutation and validation webhooks, which have 30s timeout and are run in serial
    1. API Server still does cut off a response after 60s, but the api can
       continue processing (60s gives us enough time, I think for a SDK.Ack() on Allocate, which I don't think
       we had before)
    1. Validation now happens in the request.
1. We can now do batching of requests for higher throughput (googleforgames#536), since we control the entire http request.
1. Sets us up if we decide we also want to have an alternative (http and/or gRPC) endpoint for allocation, based
   on feedback from this implementation.

The breaking changes are:
1. GameServerAllocation's group is now `allocation.agones.dev` rather than `stable.agones.dev`,
because a CRD group can't overlap with a api server.
1. Since there is only the `create` verb for GSA, there is no get/list/watch options for GameServerAllocations - so no
   informers/listers either. But this could be added at a later date, if needed.

This also includes some libraries for building further api server extension points.
  • Loading branch information
markmandel committed Feb 21, 2019
1 parent 861771b commit 287c225
Show file tree
Hide file tree
Showing 299 changed files with 115,678 additions and 1,549 deletions.
72 changes: 72 additions & 0 deletions Gopkg.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

8 changes: 8 additions & 0 deletions Gopkg.toml
Original file line number Diff line number Diff line change
Expand Up @@ -100,6 +100,14 @@
name = "contrib.go.opencensus.io/exporter/stackdriver"
version = "v0.8.0"

[[constraint]]
name = "github.com/go-openapi/spec"
version = "0.18.0"

[[constraint]]
branch = "master"
name = "github.com/munnerz/goautoneg"

[[constraint]]
name = "fortio.org/fortio"
version = "1.3.1"
2 changes: 1 addition & 1 deletion build/build-image/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@
# See the License for the specific language governing permissions and
# limitations under the License.

# ForceUpdate 6 -- change here if you need to force a rebuild
# ForceUpdate 7 -- change here if you need to force a rebuild

# compiling proto + grpc takes an exceptionally long time
# so we'll use a base from `base` - which is manually built using the below tag.
Expand Down
5 changes: 4 additions & 1 deletion build/build-image/gen-crd-client.sh
Original file line number Diff line number Diff line change
Expand Up @@ -14,9 +14,12 @@
# See the License for the specific language governing permissions and
# limitations under the License.

set -x

rm -r $GOPATH/src/agones.dev/agones/pkg/client
rsync -r /go/src/agones.dev/agones/vendor/k8s.io/ /go/src/k8s.io/
cd /go/src/k8s.io/code-generator
./generate-groups.sh "all" \
agones.dev/agones/pkg/client \
agones.dev/agones/pkg/apis stable:v1alpha1 \
agones.dev/agones/pkg/apis "allocation:v1alpha1 stable:v1alpha1" \
--go-header-file=/go/src/agones.dev/agones/build/boilerplate.go.txt
14 changes: 10 additions & 4 deletions cmd/controller/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,8 @@ import (
"agones.dev/agones/pkg/gameservers"
"agones.dev/agones/pkg/gameserversets"
"agones.dev/agones/pkg/metrics"
"agones.dev/agones/pkg/util/apiserver"
"agones.dev/agones/pkg/util/https"
"agones.dev/agones/pkg/util/runtime"
"agones.dev/agones/pkg/util/signals"
"agones.dev/agones/pkg/util/webhooks"
Expand Down Expand Up @@ -105,7 +107,11 @@ func main() {
logger.WithError(err).Fatal("Could not create the agones api clientset")
}

wh := webhooks.NewWebHook(ctlConf.CertFile, ctlConf.KeyFile)
// https server and the items that share the Mux for routing
httpsServer := https.NewServer(ctlConf.CertFile, ctlConf.KeyFile)
wh := webhooks.NewWebHook(httpsServer.Mux)
api := apiserver.NewAPIServer(httpsServer.Mux)

agonesInformerFactory := externalversions.NewSharedInformerFactory(agonesClient, defaultResync)
kubeInformerFactory := informers.NewSharedInformerFactory(kubeClient, defaultResync)

Expand Down Expand Up @@ -159,13 +165,13 @@ func main() {
fleetController := fleets.NewController(wh, health, kubeClient, extClient, agonesClient, agonesInformerFactory)
faController := fleetallocation.NewController(wh, allocationMutex,
kubeClient, extClient, agonesClient, agonesInformerFactory)
gasController := gameserverallocations.NewController(wh, health, allocationMutex, kubeClient,
kubeInformerFactory, extClient, agonesClient, agonesInformerFactory)
gasController := gameserverallocations.NewController(api, allocationMutex, kubeClient,
kubeInformerFactory, agonesClient, agonesInformerFactory)
fasController := fleetautoscalers.NewController(wh, health,
kubeClient, extClient, agonesClient, agonesInformerFactory)

rs = append(rs,
wh, gsController, gsSetController, fleetController, faController, fasController, gasController, server)
httpsServer, gsController, gsSetController, fleetController, faController, fasController, gasController, server)

stop := signals.NewStopChannel()

Expand Down
2 changes: 1 addition & 1 deletion examples/gameserverallocation.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@
# multiple Fleets, or a self managed group of GameServers.
#

apiVersion: "stable.agones.dev/v1alpha1"
apiVersion: "allocation.agones.dev/v1alpha1"
kind: GameServerAllocation
metadata:
# We recommend using the following to generate a unique name when creating Allocations
Expand Down
9 changes: 4 additions & 5 deletions examples/simple-udp/gameserverallocation.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -12,14 +12,13 @@
# See the License for the specific language governing permissions and
# limitations under the License.

#
# A GameServerAllocation against a Fleet named 'simple-udp'
# (GameServerAllocation is currently experimental, and likely to change in upcoming releases)
#

apiVersion: "stable.agones.dev/v1alpha1"
apiVersion: "allocation.agones.dev/v1alpha1"
kind: GameServerAllocation
metadata:
# We recommend using the following to generate a unique name when creating Allocations
# This will need to be created with `kubectl create` if using the command line tooling
generateName: simple-udp-
spec:
# GameServer selector from which to choose GameServers from.
# GameServers still have the hard requirement to be `Ready` to be allocated from
Expand Down
2 changes: 1 addition & 1 deletion examples/xonotic/gameserverallocation.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@

# (GameServerAllocation is currently experimental, and likely to change in upcoming releases)

apiVersion: "stable.agones.dev/v1alpha1"
apiVersion: "allocation.agones.dev/v1alpha1"
kind: GameServerAllocation
metadata:
# We recommend using the following to generate a unique name when creating Allocations
Expand Down
1 change: 0 additions & 1 deletion install/helm/agones/scripts/delete_agones_resources.sh
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,6 @@ for ns in $namespaces; do
kubectl -n $ns delete gameserversets --all
kubectl -n $ns delete gameservers --all
kubectl -n $ns delete fleetallocations --all
kubectl -n $ns delete gameserverallocations --all

# Since we don't have the nifty kubectl wait yet, hack one in the meantime
for p in $pods; do
Expand Down
32 changes: 30 additions & 2 deletions install/helm/agones/templates/admissionregistration.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -18,10 +18,40 @@
{{- $cert := genSignedCert $cn nil (list $altName1 $altName2) 3650 $ca }}
{{- if .Values.agones.registerWebhooks }}
---
apiVersion: apiregistration.k8s.io/v1beta1
kind: APIService
metadata:
name: v1alpha1.allocation.agones.dev
labels:
component: controller
app: {{ template "agones.name" . }}
chart: {{ template "agones.chart" . }}
release: {{ .Release.Name }}
heritage: {{ .Release.Service }}
spec:
group: allocation.agones.dev
groupPriorityMinimum: 1000
versionPriority: 15
service:
name: agones-controller-service
namespace: {{ .Release.Namespace }}
{{- if .Values.agones.controller.generateTLS }}
caBundle: {{ b64enc $ca.Cert }}
{{- else }}
caBundle: {{ .Files.Get "certs/server.crt" | b64enc }}
{{- end }}
version: v1alpha1
---
apiVersion: admissionregistration.k8s.io/v1beta1
kind: ValidatingWebhookConfiguration
metadata:
name: agones-validation-webhook
labels:
component: controller
app: {{ template "agones.name" . }}
chart: {{ template "agones.chart" . }}
release: {{ .Release.Name }}
heritage: {{ .Release.Service }}
webhooks:
- name: validations.stable.agones.dev
failurePolicy: Fail
Expand Down Expand Up @@ -55,7 +85,6 @@ webhooks:
- "gameserversets"
- "fleetallocations"
- "fleetautoscalers"
- "gameserverallocations"
apiVersions:
- "v1alpha1"
operations:
Expand Down Expand Up @@ -91,7 +120,6 @@ webhooks:
- "gameservers"
- "fleets"
- "fleetallocations"
- "gameserverallocations"
apiVersions:
- "v1alpha1"
operations:
Expand Down
36 changes: 0 additions & 36 deletions install/helm/agones/templates/crds/_selector.yaml

This file was deleted.

Loading

0 comments on commit 287c225

Please sign in to comment.