Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SIGSEGV while creating machine scope #521

Closed
mkumatag opened this issue Jan 25, 2022 · 3 comments · Fixed by #522 or #523
Closed

SIGSEGV while creating machine scope #521

mkumatag opened this issue Jan 25, 2022 · 3 comments · Fixed by #522 or #523
Labels
area/provider/ibmcloud Issues or PRs related to ibmcloud provider kind/bug Categorizes issue or PR as related to a bug.

Comments

@mkumatag
Copy link
Member

mkumatag commented Jan 25, 2022

/kind bug
/area provider/ibmcloud

What steps did you take and what happened:

  1. deploy a machine spec with following content:
apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
kind: IBMPowerVSMachine
metadata:
  annotations:
    cluster.x-k8s.io/cloned-from-groupkind: IBMPowerVSMachineTemplate.infrastructure.cluster.x-k8s.io
    cluster.x-k8s.io/cloned-from-name: capi-ibmcloud-e2e-2ksvpc-md-0
  creationTimestamp: "2022-01-25T06:37:32Z"
  generation: 1
  labels:
    cluster.x-k8s.io/cluster-name: capi-ibmcloud-e2e-2ksvpc
    cluster.x-k8s.io/deployment-name: capi-ibmcloud-e2e-2ksvpc-md-0
    machine-template-hash: "1719017386"
  name: capi-ibmcloud-e2e-2ksvpc-md-0-zqnxs
  namespace: create-workload-cluster-78k9hf
  ownerReferences:
  - apiVersion: cluster.x-k8s.io/v1beta1
    blockOwnerDeletion: true
    controller: true
    kind: Machine
    name: capi-ibmcloud-e2e-2ksvpc-md-0-5c5f45c7db-kklwk
    uid: 6e1bc3b7-3cfd-4b90-8aef-7ed7b4dd8404
  resourceVersion: "1425"
  uid: a7292999-4a0f-4399-ab81-da604bcd3fc1
spec:
  image:
    name: ""
  memory: "8"
  network:
    name: ""
  procType: shared
  processors: "0.25"
  serviceInstanceID: ""
  sysType: s922

error:

E0125 06:40:55.593196       1 controller.go:317] controller/ibmpowervsmachine "msg"="Reconciler error" "error"="failed to create scope: getResourceInstanceOptions failed validation:\nKey: 'GetResourceInstanceOptions.ID' Error:Field validation for 'ID' failed on the 'ne' tag\nfailed to get resource instance\nsigs.k8s.io/cluster-api-provider-ibmcloud/cloud/scope.NewPowerVSMachineScope\n\t/workspace/cloud/scope/powervs_machine.go:115\nsigs.k8s.io/cluster-api-provider-ibmcloud/controllers.(*IBMPowerVSMachineReconciler).Reconcile\n\t/workspace/controllers/ibmpowervsmachine_controller.go:94\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:114\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:311\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:227\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1581" "name"="capi-ibmcloud-e2e-2ksvpc-md-0-zqnxs" "namespace"="create-workload-cluster-78k9hf" "reconciler group"="infrastructure.cluster.x-k8s.io" "reconciler kind"="IBMPowerVSMachine"
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x28 pc=0x16f8042]

goroutine 409 [running]:
sigs.k8s.io/cluster-api-provider-ibmcloud/cloud/scope.(*PowerVSClusterScope).PatchObject(...)
	/workspace/cloud/scope/powervs_cluster.go:141
sigs.k8s.io/cluster-api-provider-ibmcloud/cloud/scope.(*PowerVSClusterScope).Close(...)
	/workspace/cloud/scope/powervs_cluster.go:146
sigs.k8s.io/cluster-api-provider-ibmcloud/controllers.(*IBMPowerVSClusterReconciler).Reconcile.func1()
	/workspace/controllers/ibmpowervscluster_controller.go:81 +0x22
sigs.k8s.io/cluster-api-provider-ibmcloud/controllers.(*IBMPowerVSClusterReconciler).Reconcile(0xc00087d110, {0x1dc47d8, 0xc0008aeba0}, {{{0xc000056500, 0x1e}, {0xc000159308, 0x18}}})
	/workspace/controllers/ibmpowervscluster_controller.go:92 +0x56b
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile(0xc000894000, {0x1dc47d8, 0xc0008aeb70}, {{{0xc000056500, 0x1a51520}, {0xc000159308, 0x413834}}})
	/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:114 +0x26f
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler(0xc000894000, {0x1dc4730, 0xc000655a00}, {0x1960640, 0xc0001a6e00})
	/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:311 +0x33e
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem(0xc000894000, {0x1dc4730, 0xc000655a00})
	/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:266 +0x205
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2()
	/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:227 +0x85
created by sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2
	/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:223 +0x357

What did you expect to happen:
Instead of the above SEGV fault, it should throw a proper error message

Anything else you would like to add:
Not sure why webhook is not catching this error here?!

The following template can be used to generate this error:

IBMPOWERVS_SSHKEY_NAME="mkumatag-pub-key" \
IBMPOWERVS_VIP="192.168.151.5" \
IBMPOWERVS_VIP_EXTERNAL="158.175.162.5" \
IBMPOWERVS_VIP_CIDR="29" \
IBMPOWERVS_IMAGE_NAME="" \
IBMPOWERVS_SERVICE_INSTANCE_ID="" \
IBMPOWERVS_NETWORK_NAME="" \
clusterctl generate cluster ibm-powervs-1 --kubernetes-version v1.22.5 \
--target-namespace default \
--control-plane-machine-count=1 \
--worker-machine-count=1 \
--from ./cluster-template-powervs.yaml

Environment:

  • Cluster-api version:
  • Minikube/KIND version:
  • Kubernetes version: (use kubectl version):
  • OS (e.g. from /etc/os-release):
@k8s-ci-robot k8s-ci-robot added kind/bug Categorizes issue or PR as related to a bug. area/provider/ibmcloud Issues or PRs related to ibmcloud provider labels Jan 25, 2022
@mkumatag
Copy link
Member Author

cc @Prajyot-Parab ^^

@mkumatag
Copy link
Member Author

This happens when NewPowerVSMachineScope/NewPowerVSMachineScope fails before assigning the patchHelper object which is used in the PatchObject function getting called for as deferred function in controller.

Failure case needs to be handled properly by:

  • return the scope instead of nil if error
  • move patchhelper assignment early in the createscope call

@mkumatag
Copy link
Member Author

@Prajyot-Parab added the template used in the description comment

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/provider/ibmcloud Issues or PRs related to ibmcloud provider kind/bug Categorizes issue or PR as related to a bug.
Projects
None yet
2 participants