This article will delve into how to implement the Request / Response pattern under the asynchronous message delivery framework of MQTT, with the new features of MQTT 5.0.
New to MQTT 5.0? Please check out our
The publisher-subscriber mechanism of MQTT completely decouples the sender and receiver of messages, allowing messages to be delivered asynchronously. However, this also brings a problem: even with QoS 1 and 2 messages, the publisher can only ensure that the message reaches the server, but cannot know whether the subscriber has ultimately received the message. When executing some requests or commands, the publisher may want to know the execution result of the other end.
The most direct way is to have the subscriber return a response of the request.
In MQTT, this is not difficult to implement. It only requires the two communicating parties to negotiate the request topic and response topic in advance, and then the subscriber returns a response to the response topic after receiving the request. This is also the method generally adopted by clients before MQTT 5.0.
In this scheme, the response topic must be determined in advance and cannot be flexibly changed. When there are multiple different requestors, since they can only subscribe to the same response topic, all requestors will receive the response, and they cannot tell whether the response belongs to themselves:
Although there are many ways to avoid this issue, it also leads to the possibility of completely different implementations among vendors, greatly increasing the difficulty and workload for users when integrating devices from different manufacturers.
To solve these problems, MQTT 5.0 introduced properties such as Response Topic, Correlation Data, and Response Information to standardize the Request / Response pattern in MQTT.
In MQTT 5.0, the requester can specify an expected Response Topic in the request message. After taking appropriate action based on the request message, the responder publishes a response message to the Response Topic carried in the request. If the requester has subscribed to that Response Topic, it will receive the response.
The requester can use its Client ID as part of the Response Topic, effectively avoiding conflicts caused by different requesters inadvertently using the same Response Topic.
The requester can also carry Correlation Data in the request, and the responder must return the Correlation Data intact in the response, allowing the requester to identify the original request to which the response belongs.
This can prevent the requester from incorrectly associating the response with the original request when the responder does not return responses in the order of requests, or when a response (QoS 0) is lost due to network disconnection.
On the other hand, the requester may need to interact with multiple responders, such as controlling various smart devices in the home via a mobile phone. The Correlation Data allows the requester to manage responses asynchronously returned from multiple responders by subscribing to a single Response Topic.
In the above Request / Response process, the MQTT broker does not change the Response Topic or Correlation Data, it only serves as a forwarding agent.
For security reasons, the MQTT server usually restricts the topics that clients can publish and subscribe to. The requester can specify a random Response Topic, but cannot guarantee that it has permission to subscribe to that topic, nor can it guarantee that the responder has permission to publish messages to that Response Topic.
Therefore, MQTT 5.0 also introduced the Response Information property. By setting the Request Response Information identifier to 1 in the CONNECT packet, the client can request the server to return Response Information in the CONNACK packet. The client can use the content of the Response Information as a specific part of the Response Topic, to pass the server's permission check.
MQTT does not further specify the details of this part, such as the content format of the Response Information and how the client creates the Response Topic based on the Response Information, so different server and client implementations may vary.
For example, the server could use the Response Information “FRONT,mytopic” to indicate both the specific content of a certain part of the Response Topic and its position within the Response Topic. It could also agree with the client on how to use this specific part in advance, then use the Response Information “mytopic” to indicate only the specific content of this part.
Taking a smart home scenario as an example, smart devices will not be used across users. We can let the MQTT server return the ID of the user to whom the device belongs as Response Information, and the client uniformly uses this user ID as the prefix of the Response Topic. The MQTT server only needs to ensure that these clients have the publication and subscription permissions for topics starting with this user ID during the lifecycle of their sessions.
Here are some suggestions for using Request / Response in MQTT, following these will help you implement best practices:
- QoS 1 and 2 in MQTT can only ensure that messages reach the server. If you want to confirm whether the message has reached the subscriber, you can use the Request / Response pattern.
- Subscribe to the Response Topic before sending the request to avoid missing the response.
- Ensure that the responder and the requester have the necessary permissions to publish and subscribe to the Response Topic. Response Information can help us build a Response Topic that meets permission requirements.
- When there are multiple requesters, they need to use different Response Topics to avoid response confusion. Using Client ID as part of the topic is a common practice.
- When there are multiple responders, it is best for the requester to set Correlation Data in the request to avoid response confusion.
- We can make the Will Message work with the Request / Response. We just need to set the Response Topic for the Will Message when connecting. This can help the client know whether the Will Message has been consumed during its offline period, so it can make appropriate adjustments.
Next, we will use MQTTX to simulate the scenario of using a mobile phone to remotely control the bedroom light to turn on and receive the response.
Install and open MQTTX, first initiate a client connection to the public MQTT broker to simulate a mobile phone, and subscribe to the response topic state/light-in-bedroom/power:
Create a new client connection to simulate the smart light, and subscribe to the request topic cmnd/light-in-bedroom/power:
Return to the request client, and send a turn-on light command with the Response Topic to the request topic cmnd/light-in-bedroom/power:
In the response client, we can see that the received message carries the Response Topic. So next, we can perform the turn-on light operation according to the command request, and then return the latest status of the light through this Response Topic:
Eventually, the request client will receive this response, and according to the content of the response message, we can know that the light has been successfully turned on:
This is a very simple example, you can also try to increase the number of publishers or responders, to experience how to design the request and response topics in these cases.
In addition, we provide Python sample code for Request / Response in emqx/MQTT-Features-Example, you can use it as a reference.