Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[rabbit] enable quorum queues #884

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

[rabbit] enable quorum queues #884

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 7 additions & 13 deletions templates/nova.conf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -55,19 +55,13 @@ key={{.SSLCertificateKeyFile}}
lock_path = /var/lib/nova/tmp

[oslo_messaging_rabbit]
amqp_durable_queues=false
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rabbit notes that durable queue are required for quorm queues
https://www.rabbitmq.com/docs/quorum-queues#feature-comparison
but the oslo messaging option is ignored if quorum queues are enabled
https://github.com/openstack/oslo.messaging/blob/master/oslo_messaging/_drivers/amqp.py#L33-L37
so its more correct to not set this

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh, good point, this wont' work w/o durable queues, and that requires separated vhosts for osp services, right?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The last time I tested a rabbit cluster with quorum queues with my Jepsen fork, it was not possible to do w/o setting durable queues, FYI.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so i think reading the commnet in oslo that oslo will enable durable queue if you request quorma cues and ignore the amqp_druable_queues value

this failed with

amqp.exceptions.PreconditionFailed: Exchange.declare: (406) PRECONDITION_FAILED - inequivalent arg 'durable' for exchange 'nova' in vhost '/': received 'true' but current is 'false'

and i think that was because all the other operators are using false.

ill try expclity setting ti to true but i think this just wont work without vhost

Copy link
Contributor

@bogdando bogdando Oct 28, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this just wont work without vhost

yes, as other services would conflict on creating things (like exchange 'openstack') there with different persistence flags

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

technically, we might want to test the altogether switch of the openstack services to that option in the downstream testproject

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Technically you should not be doing any dev downstream and we should be testing changing this everywhere upstream.

we could just make nova use a separate rabbit for this testing but in reality this is better to test manually in a local deployment then try and do it in the ci.

with that said i wont really have time to work on this so what ever way works for you.

amqp_auto_delete=false
# we should consider using quorum queues instead
# rabbit_quorum_queue=true
{{/*we might just want to make this always false*/}}
{{ if eq .service_name "nova-api"}}
# We cannot set this to true while is
# https://review.opendev.org/c/openstack/oslo.log/+/852443 is not used in the
# nova-api image otherwise logging from the heartbeat thread will cause hangs.
heartbeat_in_pthread=false
{{else}}
heartbeat_in_pthread=false
{{end}}
{{/* we want to use quorum queues but that requires durable queues */}}
rabbit_quorum_queue = true
amqp_durable_queues = true
amqp_auto_delete = true
{{/* heartbeat_in_pthread is known to have several bugs so always set it to flase */}}
heartbeat_in_pthread = false


{{ if eq .service_name "nova-api"}}
[oslo_policy]
Expand Down
5 changes: 5 additions & 0 deletions test/functional/nova_multicell_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -254,6 +254,8 @@ var _ = Describe("Nova multi cell", func() {
novaNames.Namespace, novaNames.Namespace, novaNames.Namespace)))

Expect(configData).To(ContainSubstring("transport_url=rabbit://cell0/fake"))
Expect(configData).To(ContainSubstring("rabbit_quorum_queue = true"))
Expect(configData).To(ContainSubstring("heartbeat_in_pthread = false"))

SimulateReadyOfNovaTopServices()

Expand Down Expand Up @@ -382,6 +384,9 @@ var _ = Describe("Nova multi cell", func() {
Expect(configData).Should(
ContainSubstring("tls_enabled=false"))

Expect(configData).To(ContainSubstring("rabbit_quorum_queue = true"))
Expect(configData).To(ContainSubstring("heartbeat_in_pthread = false"))

myCnf := configDataMap.Data["my.cnf"]
Expect(myCnf).To(
ContainSubstring("[client]\nssl=0"))
Expand Down
Loading