Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MySQL cluster has comaster instances when I recreated it. #207

Closed
juexun opened this issue Jan 17, 2019 · 3 comments
Closed

MySQL cluster has comaster instances when I recreated it. #207

juexun opened this issue Jan 17, 2019 · 3 comments
Labels
Milestone

Comments

@juexun
Copy link

juexun commented Jan 17, 2019

I deleted the mysql cluster using kubectl delete -f demo.yaml --cascade=false and reserved the pvc.
When i recreate the mysql cluster using kubectl apply -f demo.yaml
The cluster was in maintenance mode with two co-master instances.
How can I fix it?

@AMecea
Copy link
Contributor

AMecea commented Jan 17, 2019

I need a little more details, we can chat further on the MySQL operator channel on k8s slack to debug this.

EDIT:
I managed to reproduce it. This is a bug and I will try to figure out a fix for it. Until then when deleted a cluster but want to keep the PVCs then make sure that cluster is scaled to 1 or 0, to keep only one PVC.

The problem is because the master is node-1 and when came up node-0 can't get from orchestrator who was the original master because the operator remove it from the orchestrator. Furthermore the node-0 knows that it has to connect to node-1 but the sidecar containers configure it as master. And when node-1 came up he is configure as slave for node-0 but previously was a master.

Thanks @juexun for reporting this bug!!

@calind calind added the bug label Jan 21, 2019
@calind calind added this to the 0.2.4 milestone Jan 21, 2019
@calind
Copy link
Member

calind commented Feb 1, 2019

This should be solved by #199.

@calind calind modified the milestones: 0.2.4, 0.2.5 Feb 1, 2019
@AMecea
Copy link
Contributor

AMecea commented Feb 1, 2019

Also when deleting a cluster with --cascade=false the stateful set and pods should not be deleted. Updating controller-util fixes this, this is addressed by #221.

To reproduce this issue, in other words to delete the cluster but don't delete the PVCs, you should delete the cluster with --cascade=false and the related objects of the cluster, like stateful set, config maps, etc. Anyway when using cascade deletion you have to be more careful when recreating the cluster.

@calind calind modified the milestones: 0.2.5, 0.2.4 Feb 11, 2019
@AMecea AMecea closed this as completed Feb 13, 2019
chapsuk pushed a commit to chapsuk/mysql-operator that referenced this issue Oct 16, 2023
* Adding TopologySpreadConstraints for VTTablets

Reference:
https://v1-18.docs.kubernetes.io/docs/reference/generated/kubernetes-api/v1.18/#topologyspreadconstraint-v1-core

This adds support for managing/customizing more even spread of
VTTablets across kubernetes nodes
(in addition to affinity/tolerations).

Same feature can probably be added to VTGates, but I'm trying
to keep the changeset small.

Signed-off-by: Artem Vovk <[email protected]>

* Add topologySpreadConstraints to VTGate deployment

Signed-off-by: Artem Vovk <[email protected]>

* Use update for topologySpreadConstraints

and fix some kubebuilder comment formatting

Signed-off-by: Artem Vovk <[email protected]>

Co-authored-by: Artem Vovk <[email protected]>
chapsuk pushed a commit to chapsuk/mysql-operator that referenced this issue Oct 16, 2023
* Add 'make generate-and-diff'

Add a new Makefile target that will exit with a failure if
'make generate' generates any changes from what is currently
committed.

Use this in a new Github workflow, to reject PRs that should
include changes to generated files, but haven't generated and
committed those changes yet.

Signed-off-by: Jordan Moldow <[email protected]>

* Update CRDs from pull bitpoke#207

Signed-off-by: Jordan Moldow <[email protected]>

* Update docs and generated code from pull bitpoke#207

Signed-off-by: Jordan Moldow <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants