Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use multi-version support for CRDs #347

Closed
wants to merge 1 commit into from
Closed

Use multi-version support for CRDs #347

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 12 additions & 12 deletions Gopkg.lock
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

12 changes: 6 additions & 6 deletions Gopkg.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -74,20 +74,20 @@ required = [
name = "golang.org/x/net"

[[constraint]]
branch = "release-1.11"
name = "k8s.io/api"

version = "kubernetes-1.11.2"

[[constraint]]
branch = "release-1.11"
name = "k8s.io/apiextensions-apiserver"
version = "kubernetes-1.11.2"

[[constraint]]
branch = "release-1.11"
name = "k8s.io/apimachinery"

version = "kubernetes-1.11.2"

[[constraint]]
branch = "release-8.0"
name = "k8s.io/client-go"
version = "kubernetes-1.11.2"

[[override]]
name = "github.com/Azure/go-autorest"
Expand Down
6 changes: 2 additions & 4 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,11 +21,9 @@ on how to enable the webhook.

## Prerequisites

* Version >= 1.8 of Kubernetes.
* Version >= 1.9 of Kubernetes if **using the mutating admission webhook for Spark pod customization**.
* Version >= 1.11 of Kubernetes.

The Kubernetes Operator for Apache Spark relies on [garbage collection](https://kubernetes.io/docs/concepts/workloads/controllers/garbage-collection/) support for custom resources that is available in Kubernetes 1.8+
and **optionally** the [Mutating Admission Webhook](https://kubernetes.io/docs/reference/access-authn-authz/extensible-admission-controllers/) which is available in Kubernetes 1.9+.
The Kubernetes Operator for Apache Spark relies on [multi-version support](https://kubernetes.io/docs/tasks/access-kubernetes-api/custom-resources/custom-resource-definition-versioning/) support for CustomResourceDefinitions that is available in Kubernetes 1.11+.

**Due to this [bug](https://github.com/kubernetes/kubernetes/issues/56018) in Kubernetes 1.9 and earlier, CRD objects with
escaped quotes (e.g., `spark.ui.port\"`) in map keys can cause serialization problems in the API server. So please pay
Expand Down
16 changes: 14 additions & 2 deletions manifest/spark-operator-crds.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,13 @@ spec:
- sparkapp
singular: sparkapplication
scope: Namespaced
versions:
- name: v1alpha1
served: true
storage: true
- name: v1beta1
served: true
storage: false
validation:
openAPIV3Schema:
properties:
Expand Down Expand Up @@ -100,7 +107,6 @@ spec:
- Scala
- Python
- R
version: v1alpha1
---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
Expand All @@ -116,6 +122,13 @@ spec:
- scheduledsparkapp
singular: scheduledsparkapplication
scope: Namespaced
versions:
- name: v1alpha1
served: true
storage: true
- name: v1beta1
served: true
storage: false
validation:
openAPIV3Schema:
properties:
Expand Down Expand Up @@ -202,4 +215,3 @@ spec:
- Scala
- Python
- R
version: v1alpha1
19 changes: 15 additions & 4 deletions pkg/crd/scheduledsparkapplication/crd.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ import (

"github.com/GoogleCloudPlatform/spark-on-k8s-operator/pkg/apis/sparkoperator.k8s.io"
"github.com/GoogleCloudPlatform/spark-on-k8s-operator/pkg/apis/sparkoperator.k8s.io/v1alpha1"
"github.com/GoogleCloudPlatform/spark-on-k8s-operator/pkg/apis/sparkoperator.k8s.io/v1beta1"
)

// CRD metadata.
Expand All @@ -32,7 +33,6 @@ const (
Singular = "scheduledsparkapplication"
ShortName = "scheduledsparkapp"
Group = sparkoperator.GroupName
Version = v1alpha1.Version
FullName = Plural + "." + Group
)

Expand All @@ -42,9 +42,20 @@ func GetCRD() *apiextensionsv1beta1.CustomResourceDefinition {
Name: FullName,
},
Spec: apiextensionsv1beta1.CustomResourceDefinitionSpec{
Group: Group,
Version: Version,
Scope: apiextensionsv1beta1.NamespaceScoped,
Group: Group,
Versions: []apiextensionsv1beta1.CustomResourceDefinitionVersion{
{
Name: v1alpha1.Version,
Served: true,
Storage: true,
},
{
Name: v1beta1.Version,
Served: true,
Storage: false,
},
},
Scope: apiextensionsv1beta1.NamespaceScoped,
Names: apiextensionsv1beta1.CustomResourceDefinitionNames{
Plural: Plural,
Singular: Singular,
Expand Down
19 changes: 15 additions & 4 deletions pkg/crd/sparkapplication/crd.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ import (

"github.com/GoogleCloudPlatform/spark-on-k8s-operator/pkg/apis/sparkoperator.k8s.io"
"github.com/GoogleCloudPlatform/spark-on-k8s-operator/pkg/apis/sparkoperator.k8s.io/v1alpha1"
"github.com/GoogleCloudPlatform/spark-on-k8s-operator/pkg/apis/sparkoperator.k8s.io/v1beta1"
)

// CRD metadata.
Expand All @@ -32,7 +33,6 @@ const (
Singular = "sparkapplication"
ShortName = "sparkapp"
Group = sparkoperator.GroupName
Version = v1alpha1.Version
FullName = Plural + "." + Group
)

Expand All @@ -42,9 +42,20 @@ func GetCRD() *apiextensionsv1beta1.CustomResourceDefinition {
Name: FullName,
},
Spec: apiextensionsv1beta1.CustomResourceDefinitionSpec{
Group: Group,
Version: Version,
Scope: apiextensionsv1beta1.NamespaceScoped,
Group: Group,
Versions: []apiextensionsv1beta1.CustomResourceDefinitionVersion{
{
Name: v1alpha1.Version,
Served: true,
Storage: true,
},
{
Name: v1beta1.Version,
Served: true,
Storage: false,
},
},
Scope: apiextensionsv1beta1.NamespaceScoped,
Names: apiextensionsv1beta1.CustomResourceDefinitionNames{
Plural: Plural,
Singular: Singular,
Expand Down