Error when installing m3db operator in Docker for Desktop #310

pat-wong-affirm · 2022-01-18T23:55:56Z

Thanks for opening an issue for the M3DB Operator! We'd love to help you, but we need the following information included
with any issue:

What version of the operator are you running? Please include the docker tag. If using master, please include the git
SHA logged when the operator first starts.

manifest = https://raw.githubusercontent.com/m3db/m3db-operator/master/bundle.yaml

docker image tag = quay.io/m3db/m3db-operator:latest

also tried with docker image tag being v0.13.0, same result

What version of Kubernetes are you running? Please include the output of kubectl version.

Client Version: version.Info{Major:"1", Minor:"23", GitVersion:"v1.23.1", GitCommit:"86ec240af8cbd1b60bcc4c03c20da9b98005b92e", GitTreeState:"clean", BuildDate:"2021-12-16T11:33:37Z", GoVersion:"go1.17.5", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"22", GitVersion:"v1.22.4", GitCommit:"b695d79d4f967c403a96986f1750a35eb75e75f1", GitTreeState:"clean", BuildDate:"2021-11-17T15:42:41Z", GoVersion:"go1.16.10", Compiler:"gc", Platform:"linux/amd64"}

MacOS - Docker For Desktop - built-in kubernetes

What are you trying to do?

Install the m3db operator

What did you expect to happen?

Operator pod would be in Running state

What happened?

{"level":"info","ts":1642550015.150743,"msg":"using InCluster k8s config"}
{"level":"info","ts":1642550015.1522524,"msg":"starting Operator controller","controller":"m3db-cluster-controller"}
E0118 23:53:35.163925       1 reflector.go:153] pkg/mod/k8s.io/[email protected]/tools/cache/reflector.go:105: Failed to list *v1alpha1.M3DBCluster: the server could not find the requested resource (get m3dbclusters.operator.m3db.io)
{"level":"fatal","ts":1642550015.1678174,"msg":"error running controller","error":"could not create or update CRD: error creating CRD 'm3dbclusters.operator.m3db.io': the server could not find the requested resource","errorVerbose":"the server could not find the requested resource\nerror creating CRD 'm3dbclusters.operator.m3db.io'\ncould not create or update CRD"}

The text was updated successfully, but these errors were encountered:

schallert · 2022-02-03T03:48:38Z

Hey @pat-wong-affirm, thanks for reaching out. It looks like we haven't cut a release since merging #302, which makes the operator compatible with Kubernetes 1.21+.

Can you try using the master tag of the operator and letting me know if that fixes it? If so, we can likely solve this by cutting a new release. Thanks!

pat-wong-affirm · 2022-02-03T16:02:06Z

unfortunately, got the same error when using image tag quay.io/m3db/m3db-operator:master

it's possible this error is specific to Docker for Desktop, as another team member was able to get the operator working in an EKS cluster

kendrickclark · 2022-03-22T17:25:36Z

I am having this issue as well

larsf · 2022-03-23T14:37:55Z

I am also seeing this and I am also trying to deploy it on Docker for Desktop

larsf · 2022-03-25T22:42:20Z

could this be because the api server in the desktop environment is not running as localhost? main.go sets masterURL to be 127.0.0:8001, but in the desktop environment you have:

kubectl cluster-info
Kubernetes control plane is running at https://kubernetes.docker.internal:6443
CoreDNS is running at https://kubernetes.docker.internal:6443/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy

I tried adding
args: ["-debug", "-masterhost", "https://kubernetes.docker.internal:6443"]

to bundle.yaml
but it didn't help ...

larsf · 2022-03-25T23:49:29Z

I was able to get the operator to start by passing in:
["-debug=true", "-manage-crd=false", "-masterhost", "https://kubernetes.docker.internal:6443"]

(I believe only -manage-crd=false is really needed to get it to start)

I do see the same errors from the controller as documented earlier periodically, but it does stay running.

Trying to stand up a m3db cluster though, fails with this error:
no matches for kind "M3DBCluster" in version "operator.m3db.io/v1alpha

larsf · 2022-03-29T22:07:29Z

the m3db-operator seems to use interfaces that are no longer supported (k8s v1.22+):
46: crdClient := k.kubeExt.ApiextensionsV1beta1().CustomResourceDefin

see: storageos/cluster-operator#348

Server Version: version.Info{Major:"1", Minor:"22"

kendrickclark · 2022-03-30T02:18:53Z

Would a fix be to change that to ApiextensionsV1()

larsf · 2022-03-30T02:38:28Z

Hi, Partly, I have a diff to 4 files that changes v1beta1 to v1, but I haven't had a chance to build and test it yet

…

On Tue, Mar 29, 2022 at 9:19 PM kendrickclark ***@***.***> wrote: Would a fix be to change that to ApiextensionsV1() — Reply to this email directly, view it on GitHub <#310 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAALESIJB7UFEHYSILVW3DLVCO2ZRANCNFSM5MIPM3HA> . You are receiving this because you commented.Message ID: ***@***.***>

larsf · 2022-03-30T15:11:31Z

Tried to do a build with my changes, but found that the .ci directory is empty, so the build fails because it cannot find go-build-ldflags.sh and common.mk

Any reason they are not in the source tree?

Answered my own question - found the scripts in the m3db/ci-scripts repo
Got further, now I see:
~/src/m3operator$ make
--- clean-all
build .: cannot find module for path .
make: *** [Makefile:194: clean-all] Error 1

kendrickclark · 2022-04-11T12:38:59Z

@larsf Any luck there?

kendrickclark · 2022-05-04T20:21:59Z

Hey @pat-wong-affirm, thanks for reaching out. It looks like we haven't cut a release since merging #302, which makes the operator compatible with Kubernetes 1.21+.

Can you try using the master tag of the operator and letting me know if that fixes it? If so, we can likely solve this by cutting a new release. Thanks!

schallert , Unfortunately this does not resolve the issue.

niravpatel2008 · 2022-07-16T04:43:46Z

Kubernetes version 1.22

i am also facing similar issue on eks while installing operator using following command.

kubectl apply -f https://raw.githubusercontent.com/m3db/m3db-operator/master/bundle.yaml

error:

{"level":"info","ts":1657946530.7369597,"msg":"using InCluster k8s config"}
{"level":"info","ts":1657946530.7382486,"msg":"starting Operator controller","controller":"m3db-cluster-controller"}
{"level":"fatal","ts":1657946530.7782364,"msg":"error running controller","error":"could not create or update CRD: could not fetch CRD 'm3dbclusters.operator.m3db.io': customresourcedefinitions.apiextensions.k8s.io \"m3dbclusters.operator.m3db.io\" is forbidden: User \"system:serviceaccount:default:m3db-operator\" cannot get resource \"customresourcedefinitions\" in API group \"apiextensions.k8s.io\" at the cluster scope","errorVerbose":"customresourcedefinitions.apiextensions.k8s.io \"m3dbclusters.operator.m3db.io\" is forbidden: User \"system:serviceaccount:default:m3db-operator\" cannot get resource \"customresourcedefinitions\" in API group \"apiextensions.k8s.io\" at the cluster scope\ncould not fetch CRD 'm3dbclusters.operator.m3db.io'\ncould not create or update CRD"}

niravpatel2008 · 2022-07-16T04:58:04Z

Kubernetes version 1.22

i am also facing similar issue on eks while installing operator using following command.

kubectl apply -f https://raw.githubusercontent.com/m3db/m3db-operator/master/bundle.yaml

error:

{"level":"info","ts":1657946530.7369597,"msg":"using InCluster k8s config"}
{"level":"info","ts":1657946530.7382486,"msg":"starting Operator controller","controller":"m3db-cluster-controller"}
{"level":"fatal","ts":1657946530.7782364,"msg":"error running controller","error":"could not create or update CRD: could not fetch CRD 'm3dbclusters.operator.m3db.io': customresourcedefinitions.apiextensions.k8s.io \"m3dbclusters.operator.m3db.io\" is forbidden: User \"system:serviceaccount:default:m3db-operator\" cannot get resource \"customresourcedefinitions\" in API group \"apiextensions.k8s.io\" at the cluster scope","errorVerbose":"customresourcedefinitions.apiextensions.k8s.io \"m3dbclusters.operator.m3db.io\" is forbidden: User \"system:serviceaccount:default:m3db-operator\" cannot get resource \"customresourcedefinitions\" in API group \"apiextensions.k8s.io\" at the cluster scope\ncould not fetch CRD 'm3dbclusters.operator.m3db.io'\ncould not create or update CRD"}

using operator image tag 0484dcc instead of 1.13.0 works for me. but it seems it has some security issues. wish it will gets resolved with 1.14.0

pat-wong-affirm changed the title ~~Error when installing m3db operator~~ Error when installing m3db operator in Docker for Desktop Mar 25, 2022

kendrickclark mentioned this issue Mar 30, 2022

v1alpha1.M3DBCluster: the server could not find the requested resource #318

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Error when installing m3db operator in Docker for Desktop #310

Error when installing m3db operator in Docker for Desktop #310

pat-wong-affirm commented Jan 18, 2022

schallert commented Feb 3, 2022

pat-wong-affirm commented Feb 3, 2022

kendrickclark commented Mar 22, 2022

larsf commented Mar 23, 2022

larsf commented Mar 25, 2022 •

edited

Loading

larsf commented Mar 25, 2022

larsf commented Mar 29, 2022 •

edited

Loading

kendrickclark commented Mar 30, 2022

larsf commented Mar 30, 2022 via email

larsf commented Mar 30, 2022 •

edited

Loading

kendrickclark commented Apr 11, 2022

kendrickclark commented May 4, 2022

niravpatel2008 commented Jul 16, 2022

niravpatel2008 commented Jul 16, 2022

Error when installing m3db operator in Docker for Desktop #310

Error when installing m3db operator in Docker for Desktop #310

Comments

pat-wong-affirm commented Jan 18, 2022

schallert commented Feb 3, 2022

pat-wong-affirm commented Feb 3, 2022

kendrickclark commented Mar 22, 2022

larsf commented Mar 23, 2022

larsf commented Mar 25, 2022 • edited Loading

larsf commented Mar 25, 2022

larsf commented Mar 29, 2022 • edited Loading

kendrickclark commented Mar 30, 2022

larsf commented Mar 30, 2022 via email

larsf commented Mar 30, 2022 • edited Loading

kendrickclark commented Apr 11, 2022

kendrickclark commented May 4, 2022

niravpatel2008 commented Jul 16, 2022

niravpatel2008 commented Jul 16, 2022

larsf commented Mar 25, 2022 •

edited

Loading

larsf commented Mar 29, 2022 •

edited

Loading

larsf commented Mar 30, 2022 •

edited

Loading