Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Update of 1 node with no replication to a few replicated nodes leads to data loss #1578

Closed
18 tasks
atsikham opened this issue Aug 21, 2020 · 0 comments
Closed
18 tasks

Comments

@atsikham
Copy link
Contributor

atsikham commented Aug 21, 2020

Describe the bug
This bug is the result of #1497 spike.
Update of 1 node with no replication to a few replicated nodes leads to data loss. Also no rabbitmqctl forget_cluster_node <node> executed during downscale, so it leads to some issues, such of removed node disk presence in cluster information.

To Reproduce
Steps to reproduce the behavior:

  1. deploy 1 rabbitmq node with disabled clustering (default configuration)
  2. create some entities: users/queues/vhosts (see More information part with useful commands)
  3. extend config file with a part from Config files
  4. apply configuration with 3 instances (count: 3)
  5. check created data, it's been lost
  6. downscale to 2 instances, data is in place

Expected behavior

Config files

---
kind: configuration/rabbitmq
title: "RabbitMQ"
name: default
provider: azure
specification:
  cluster:
    is_clustered: true

Cloud Environment (please complete the following information):
Azure, but should also be tested with other providers.

Additional context
Found in version 0.7.1.

More information
Verification commands

# check cluster status
rabbitmq-diagnostics cluster_status

# create user
rabbitmqctl add_user testuser testpassword
rabbitmqctl set_user_tags testuser administrator
rabbitmqctl set_permissions -p / testuser ".*" ".*" ".*"

# create vhost
rabbitmqctl add_vhost Some_Virtual_Host
rabbitmqctl set_permissions -p Some_Virtual_Host guest ".*" ".*" ".*"

# create queue
rabbitmq-plugins enable rabbitmq_management
wget http://127.0.0.1:15672/cli/rabbitmqadmin
chmod +x rabbitmqadmin
./rabbitmqadmin declare queue --vhost=Some_Virtual_Host name=some_outgoing_queue durable=true

# create exchange
./rabbitmqadmin declare exchange --vhost=Some_Virtual_Host name=some_exchange type=direct

# create binding
./rabbitmqadmin --vhost=Some_Virtual_Host declare binding source=some_exchange destination_type=queue destination=some_outgoing_queue routing_key=some_routing_key

# check queues and vhosts count
rabbitmqctl status

# list
rabbitmqctl list_bindings --vhost "Some_Virtual_Host"
rabbitmqctl list_vhosts
rabbitmqctl list_queues --vhost "Some_Virtual_Host"

DoD checklist

  • Changelog
    • updated
    • not needed
  • COMPONENTS.md
    • updated
    • not needed
  • Schema
    • updated
    • not needed
  • Backport tasks
    • created
    • not needed
  • Documentation
    • added
    • updated
    • not needed
  • Feature has automated tests
  • Automated tests passed (QA pipelines)
    • apply
    • upgrade
    • backup/restore
  • Idempotency tested
  • All conversations in PR resolved
@seriva seriva changed the title [BUG][RabbitMQ] Update of 1 node with no replication to a few replicated nodes leads to data loss [BUG] Update of 1 node with no replication to a few replicated nodes leads to data loss Apr 20, 2022
@seriva seriva closed this as not planned Won't fix, can't repro, duplicate, stale Nov 16, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants