Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Avoid segfault when scheduler resources is nil #302

Merged
merged 1 commit into from
Jul 18, 2024

Conversation

Sovietaced
Copy link
Contributor

This is a fix for a segfault that occurs when resources are nil in the scheduler spec. The current code tries to dereference *scheduler.Spec.Resources outside of the conditional that checks for nil.

Follow up on #298 with unit tests since @jpoler is no longer responsible for our Armada deployment.

@Sovietaced Sovietaced changed the title Avoid nil pointer when scheduler resources is nil Avoid segfault when scheduler resources is nil May 31, 2024
@Sovietaced Sovietaced changed the title Avoid segfault when scheduler resources is nil fix: Avoid segfault when scheduler resources is nil May 31, 2024
@dejanzele
Copy link
Member

dejanzele commented Jun 2, 2024

Hey @Sovietaced,

Thanks for raising this issue and fixing it!

The core issue was a typo in the Scheduler CRD webhook configuration which wasn't wiring up the mutating webhook properly and it wasn't getting triggered, hence it wasn't defaulting the resources.

This is fixed in #301

This PR also contains updates to work with latest version of Armada which is v0.5.0, it contains a big documentation redo with better examples...

We plan on merging it tomorrow.

Can you please take a look at this PR?

@Sovietaced
Copy link
Contributor Author

This is fixed in #301

Thank you, will close this.

@Sovietaced Sovietaced closed this Jun 5, 2024
@Sovietaced Sovietaced reopened this Jul 15, 2024
@Sovietaced
Copy link
Contributor Author

@dejanzele this does not appear to be fixed in the pull request you mentioned. We are using the latest operator version 0.2.0 and we just hit this when we didn't specify resources for the scheduler.

2024-07-14T23:34:03Z	INFO	setup	Observed a panic in reconciler: runtime error: invalid memory address or nil pointer dereference	{"controller": "scheduler", "controllerGroup": "install.armadaproject.io", "controllerKind": "Scheduler", "Scheduler": {"name":"armada-scheduler","namespace":"armada"}, "namespace": "armada", "name": "armada-scheduler", "reconcileID": "65a6210c-5e4b-4e38-b7bc-d5d1c6bf7929"}
panic: runtime error: invalid memory address or nil pointer dereference [recovered]
	panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x15e401f]

goroutine 433 [running]:
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile.func1()
	/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:111 +0x1e5
panic({0x17a0040?, 0x28ebef0?})
	/opt/hostedtoolcache/go/1.22.3/x64/src/runtime/panic.go:770 +0x132
github.com/armadaproject/armada-operator/internal/controller/install.createSchedulerDeployment(0xc000170f08)
	/home/runner/work/armada-operator/armada-operator/internal/controller/install/scheduler_controller.go:362 +0xe7f
github.com/armadaproject/armada-operator/internal/controller/install.generateSchedulerInstallComponents(0xc000170f08, 0xc0002507e0)

@dejanzele
Copy link
Member

Hi @Sovietaced,

I will recheck today.

@dejanzele dejanzele merged commit 28367eb into armadaproject:main Jul 18, 2024
15 of 17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants