diff --git a/.gitignore b/.gitignore new file mode 100644 index 00000000..6c5bcd14 --- /dev/null +++ b/.gitignore @@ -0,0 +1,16 @@ +**/target/ +**/.idea/ +**/*.iml +**/.vscode/ +**/bin/ +**/.vertx/ +**/.factorypath + +# Eclipse files +**/.classpath +**/.project +**/.settings/ +.checkstyle + +# OS files +.DS_Store diff --git a/protocol-gateway/README.md b/protocol-gateway/README.md new file mode 100644 index 00000000..ab38f7d4 --- /dev/null +++ b/protocol-gateway/README.md @@ -0,0 +1,9 @@ +# Hono Protocol Gateways + +Contains [protocol gateways](https://www.eclipse.org/hono/docs/concepts/connecting-devices/#connecting-via-a-protocol-gateway) for Eclipse Hono™. + +The directory [mqtt-protocol-gateway-template](mqtt-protocol-gateway-template) provides a template for the creation of +custom MQTT protocol gateways. + +The directory [azure-mqtt-protocol-gateway](azure-mqtt-protocol-gateway) contains an example implementation of the +MQTT protocol gateway template that provides (parts of) the MQTT interface of Azure IoT Hub. diff --git a/protocol-gateway/azure-mqtt-protocol-gateway/README.md b/protocol-gateway/azure-mqtt-protocol-gateway/README.md new file mode 100644 index 00000000..482e71e5 --- /dev/null +++ b/protocol-gateway/azure-mqtt-protocol-gateway/README.md @@ -0,0 +1,167 @@ +# "Azure IoT Hub" Protocol Gateway + +This Protocol Gateway shows how to use Hono's Protocol Gateway Template to implement a production-ready protocol gateway. +The MQTT-API of "Azure IoT Hub" serves as a working example. Parts of its API are mapped to Hono's communication patterns. + +Full compatibility with the Azure IoT Hub is not a design goal of this example. It is supposed to behave similarly for +the "happy path", but cannot treat all errors or misuse in the same way as the former. + +Supported are the following types of messages: + +## Mapping of Azure IoT Hub messages to Hono messages + +**Device-to-cloud communication** + +| Azure IoT Hub message | Hono message | Limitations | +|---|---|---| +| [Device-to-cloud](https://docs.microsoft.com/en-us/azure/iot-hub/iot-hub-devguide-d2c-guidance) with QoS 0 (*AT MOST ONCE*) | [Telemetry](https://www.eclipse.org/hono/docs/api/telemetry/#forward-telemetry-data) | Messages are not brokered | +| [Device-to-cloud](https://docs.microsoft.com/en-us/azure/iot-hub/iot-hub-devguide-d2c-guidance) with QoS 1 (*AT LEAST ONCE*) | [Event](https://www.eclipse.org/hono/docs/api/event/#forward-event) | Messages are not brokered | + + +**Cloud-to-device communication** + +| Hono message | Azure IoT Hub message | Limitations | +|---|---|---| +| [One-way Command](https://www.eclipse.org/hono/docs/api/command-and-control/#send-a-one-way-command) | [Cloud-to-device](https://docs.microsoft.com/en-us/azure/iot-hub/iot-hub-devguide-messages-c2d) | Messages are not brokered (ignores CleanSession flag) | +| [Request/Response Command](https://www.eclipse.org/hono/docs/api/command-and-control/#send-a-request-response-command) | [Direct method](https://docs.microsoft.com/en-us/azure/iot-hub/iot-hub-devguide-direct-methods) | | + +## Limitations + +Not supported are the following features of Azure IoT Hub: + + * "device twins" + * file uploads + * message brokering + * "jobs" + * the back-end application API + * device authentication with client certificates + +## Device Authentication + +A Hono protocol gateway is responsible for the authentication of the devices. +This example implementation does not provide or require data storage for device credentials. +Instead, it can only be configured to use a single demo device, which must already be present in Hono's device registry (see below). +Client certificate based authentication is not implemented. + +Since there is only one device in this example implementation anyway, the credentials for the tenant's gateway client are not looked up dynamically, but are taken from the configuration. + + +## Prerequisites + +### Registering Devices + +The demo device and the gateway need to be registered in Hono's device registry. For the gateway, credentials must be created. +The [Getting started](https://www.eclipse.org/hono/getting-started/#registering-devices) guide shows how to do this. + +Alternatively, the script `scripts/create_demo_devices.sh` can be used to register the devices and create credentials: +~~~sh +# in directory: protocol-gateway/azure-mqtt-protocol-gateway/scripts/ +bash create_demo_devices.sh +~~~ + +After completion the script prints the configuration properties. Copy the output into the +file `protocol-gateway/azure-mqtt-protocol-gateway/src/main/resources/application.properties`. + + +### Configuration + +The protocol gateway needs the configuration of: + +1. the AMQP adapter of a running Hono instance to connect to +2. the MQTT server +3. the demo device to use. + +By default, the gateway will connect to the AMQP adapter of the [Hono Sandbox](https://www.eclipse.org/hono/sandbox/). +However, it can also be configured to connect to a local instance. +The default configuration can be found in the file `protocol-gateway/azure-mqtt-protocol-gateway/src/main/resources/application.properties` +and can be customized using [Spring Boot Configuration](https://docs.spring.io/spring-boot/docs/current/reference/html/spring-boot-features.html#boot-features-external-config). + + +### Starting a Receiver + +Telemetry and event messages need an application that consumes the messages. +The [Getting started](https://www.eclipse.org/hono/getting-started/#starting-the-example-application) guide shows how to start the example application that receives the messages. + + +## Starting the Protocol Gateway + +Build the template project: +~~~sh +# in directory: protocol-gateway/mqtt-protocol-gateway-template/ +mvn clean install +~~~ + +and start the protocol gateway: +~~~sh +# in directory: protocol-gateway/azure-mqtt-protocol-gateway/ +mvn spring-boot:run +~~~ + + +## Enable TLS + +Azure IoT Hub only provides connections with TLS and only offers port 8883. To start the protocol gateway listening +on this port with TLS enabled and a demo certificate, run: + +~~~sh +# in directory: protocol-gateway/azure-mqtt-protocol-gateway/ +mvn spring-boot:run -Dspring-boot.run.profiles=ssl +~~~ +**NB** Do not forget to build the template project before, as shown above. + +With the [Eclipse Mosquitto](https://mosquitto.org/) command line client, for example, sending an event message would then look like this: + +~~~sh +# in directory: protocol-gateway/azure-mqtt-protocol-gateway +mosquitto_pub -d -h localhost -p 8883 -i '4712' -u 'demo1' -P 'demo-secret' -t "devices/4712/messages/events/" -m "hello world" -V mqttv311 --cafile target/config/hono-demo-certs-jar/trusted-certs.pem +~~~ + +Existing hardware devices might need to be configured to accept the used certificate. + +## Example Requests + +With the [Eclipse Mosquitto](https://mosquitto.org/) command line client the requests look like the following. + +**Telemetry** + +~~~sh +mosquitto_pub -d -h localhost -i '4712' -u 'demo1' -P 'demo-secret' -t 'devices/4712/messages/events/?foo%20bar=b%5Fa%5Fz' -m "hello world" -V mqttv311 -q 0 +~~~ + +**Events** + +~~~sh +mosquitto_pub -d -h localhost -i '4712' -u 'demo1' -P 'demo-secret' -t 'devices/4712/messages/events/?foo%20bar=b%5Fa%5Fz' -m '{"alarm": 1}' -V mqttv311 -q 1 +~~~ + +### Commands + +The example application can be used to send commands. +The [Getting started](https://www.eclipse.org/hono/getting-started/#advanced-sending-commands-to-a-device) shows a walk-through example. + +**Subscribe for one-way commands** + +~~~sh +mosquitto_sub -v -h localhost -u "demo1" -P "demo-secret" -t 'devices/4712/messages/devicebound/#' -q 1 +~~~ + +**Subscribe for request-response commands** + +~~~sh +mosquitto_sub -v -h localhost -u "demo1" -P "demo-secret" -t '$iothub/methods/POST/#' -q 1 +~~~ + +When Mosquitto receives the command, the output in the terminal should look like this: +~~~sh +$iothub/methods/POST/setBrightness/?$rid=0100bba05d61-7027-4131-9a9d-30238b9ec9bb {"brightness": 87} +~~~ + +**Respond to a command** + +When sending a response, the request id must be added. The ID after `rid=` can be copied from the received message +and pasted into a new terminal to publish the response like this: +~~~sh +export RID=0100bba05d61-7027-4131-9a9d-30238b9ec9bb +mosquitto_pub -d -h localhost -u 'demo1' -P 'demo-secret' -t "\$iothub/methods/res/200/?\$rid=$RID" -m '{"success": true}' -q 1 +~~~ +Note that the actual identifier from the command must be used. diff --git a/protocol-gateway/azure-mqtt-protocol-gateway/pom.xml b/protocol-gateway/azure-mqtt-protocol-gateway/pom.xml new file mode 100644 index 00000000..fd748d00 --- /dev/null +++ b/protocol-gateway/azure-mqtt-protocol-gateway/pom.xml @@ -0,0 +1,270 @@ + + + + + 4.0.0 + + org.eclipse.hono + azure-protocol-gateway-example + 0.0.1-SNAPSHOT + + Hono Azure IoT Protocol Gateway Example + A simple protocol gateway for connecting Azure IoT Hub compliant devices via MQTT to Eclipse Hono. + + https://www.eclipse.org/hono + 2020 + + + Eclipse Foundation + https://www.eclipse.org/ + + + + + Eclipse Public License - Version 2.0 + http://www.eclipse.org/legal/epl-2.0 + SPDX-License-Identifier: EPL-2.0 + + + + + UTF-8 + UTF-8 + + 11 + 11 + + exec + + + yyyy-MM-dd + ${maven.build.timestamp} + + 3.15.0 + 1.3.0-M3 + 0.0.1-SNAPSHOT + 5.6.0 + 3.3.3 + 1.26 + 2.2.7.RELEASE + + + + + org.eclipse.hono + hono-mqtt-protocol-gateway + ${hono.mqtt-protocol-gateway.version} + + + org.eclipse.hono + hono-legal + ${hono.version} + + + org.eclipse.hono + hono-demo-certs + ${hono.version} + + + org.springframework.boot + spring-boot + ${spring-boot.version} + + + org.springframework.boot + spring-boot-autoconfigure + ${spring-boot.version} + + + org.springframework.boot + spring-boot-starter-logging + ${spring-boot.version} + + + org.yaml + snakeyaml + ${snakeyaml.version} + runtime + + + org.mockito + mockito-core + ${mockito.version} + test + + + org.junit.jupiter + junit-jupiter-engine + ${junit.jupiter.version} + test + + + org.assertj + assertj-core + ${assertj-core.version} + test + + + + + + + maven-compiler-plugin + 3.8.1 + + + default-compile + compile + + compile + + + + default-testCompile + test-compile + + testCompile + + + + + + org.apache.maven.plugins + maven-javadoc-plugin + 3.1.0 + + all,-accessibility + + + + org.apache.maven.plugins + maven-surefire-plugin + 2.22.2 + + + + junit.jupiter.execution.parallel.enabled = true + junit.jupiter.execution.parallel.mode.default = same_thread + junit.jupiter.execution.parallel.mode.classes.default = concurrent + + + + + + org.apache.maven.plugins + maven-failsafe-plugin + 2.22.2 + + + + org.apache.maven.plugins + maven-dependency-plugin + 3.1.1 + + + copy_legal_docs + prepare-package + + unpack-dependencies + + + hono-legal + ${project.build.outputDirectory}/META-INF + legal/** + + + + + copy_demo_certs + generate-resources + + unpack-dependencies + + + + hono-demo-certs + + ${project.build.directory}/config + + *.pem, + *.jks, + *.p12 + + true + true + true + true + + + + + + org.springframework.boot + spring-boot-maven-plugin + 2.1.11.RELEASE + + + + repackage + + + false + ${classifier.spring.boot.artifact} + hono-legal,hono-demo-certs + + + + + + org.apache.maven.plugins + maven-checkstyle-plugin + 3.1.1 + + + org.eclipse.hono + hono-legal + ${hono.version} + + + com.puppycrawl.tools + checkstyle + 8.32 + + + + + checkstyle-check + verify + + check + + + + + checkstyle/default.xml + checkstyle/suppressions.xml + true + + + + + + diff --git a/protocol-gateway/azure-mqtt-protocol-gateway/scripts/create_demo_devices.sh b/protocol-gateway/azure-mqtt-protocol-gateway/scripts/create_demo_devices.sh new file mode 100755 index 00000000..68e75d60 --- /dev/null +++ b/protocol-gateway/azure-mqtt-protocol-gateway/scripts/create_demo_devices.sh @@ -0,0 +1,72 @@ +#!/bin/bash +#******************************************************************************* +# Copyright (c) 2020 Contributors to the Eclipse Foundation +# +# See the NOTICE file(s) distributed with this work for additional +# information regarding copyright ownership. +# +# This program and the accompanying materials are made available under the +# terms of the Eclipse Public License 2.0 which is available at +# http://www.eclipse.org/legal/epl-2.0 +# +# SPDX-License-Identifier: EPL-2.0 +#******************************************************************************* + +################################################################################ +# This simple shell script registers devices to be used to demonstrate a protocol gateway. +# It does the following: +# 1. create a new tenant (or use an existing one) +# 2. register a gateway device +# 3. create credentials for the gateway +# 4. register a demo device and configure it to use the gateway +################################################################################ + +################################################################################ +# CONFIGURATION +# +DEVICE_TO_CREATE="4712" # If changed, change it in the mosquitto requests as well +GATEWAY_TO_CREATE="gw" +GATEWAY_PASSWORD="gw-secret" +# TENANT_TO_USE="" # Set this to use an existing tenant +################################################################################ + +REGISTRY_IP=${REGISTRY_IP:-hono.eclipseprojects.io} + +set -e # exit script on error + +if [ -z "$GATEWAY_PASSWORD" ] || [ -z "$REGISTRY_IP" ] ; then + echo "ERROR: missing configuration. Exit." + exit 1 +fi + +echo "# Using device registry: ${REGISTRY_IP}" + +# register new tenant +if [ -z "$TENANT_TO_USE" ] ; then + TENANT_TO_USE=$(curl --fail -X POST http://$REGISTRY_IP:28080/v1/tenants 2>/dev/null | jq -r .id) + echo "# Registered new tenant: ${TENANT_TO_USE}" +else + echo "# Using configured tenant: ${TENANT_TO_USE}" +fi + +# register new gateway +GATEWAY_TO_CREATE=$(curl --fail -X POST http://$REGISTRY_IP:28080/v1/devices/$TENANT_TO_USE/$GATEWAY_TO_CREATE -d '{"enabled":true}' -H "Content-Type: application/json" 2>/dev/null | jq -r .id) + +# set credentials for gateway +curl --fail -X PUT -H "content-type: application/json" --data-binary '[{ + "type": "hashed-password", + "auth-id": "'$GATEWAY_TO_CREATE'", + "secrets": [{ "pwd-plain": "'$GATEWAY_PASSWORD'" }] +}]' http://$REGISTRY_IP:28080/v1/credentials/$TENANT_TO_USE/$GATEWAY_TO_CREATE +HONO_CLIENT_AMQP_PASSWORD=$GATEWAY_PASSWORD + +# register demo device +HONO_DEMO_DEVICE_DEVICE_ID=$(curl --fail -X POST http://$REGISTRY_IP:28080/v1/devices/$TENANT_TO_USE/$DEVICE_TO_CREATE -d '{"enabled":true,"via":["'$GATEWAY_TO_CREATE'"]}' -H "Content-Type: application/json" 2>/dev/null | jq -r .id) + +echo "# --- DONE ---" +echo "# Please copy the following properties into the configuration of your protocol gateway:" +echo +echo "hono.demo.device.deviceId=${HONO_DEMO_DEVICE_DEVICE_ID}" +echo "hono.client.amqp.password=${HONO_CLIENT_AMQP_PASSWORD}" +echo "hono.client.amqp.username=${GATEWAY_TO_CREATE}@${TENANT_TO_USE}" +echo "hono.demo.device.tenantId=${TENANT_TO_USE}" diff --git a/protocol-gateway/azure-mqtt-protocol-gateway/src/main/java/org/eclipse/hono/gateway/azure/AzureIotHubGatewayApplication.java b/protocol-gateway/azure-mqtt-protocol-gateway/src/main/java/org/eclipse/hono/gateway/azure/AzureIotHubGatewayApplication.java new file mode 100644 index 00000000..dced1ea6 --- /dev/null +++ b/protocol-gateway/azure-mqtt-protocol-gateway/src/main/java/org/eclipse/hono/gateway/azure/AzureIotHubGatewayApplication.java @@ -0,0 +1,50 @@ +/******************************************************************************* + * Copyright (c) 2020 Contributors to the Eclipse Foundation + * + * See the NOTICE file(s) distributed with this work for additional + * information regarding copyright ownership. + * + * This program and the accompanying materials are made available under the + * terms of the Eclipse Public License 2.0 which is available at + * http://www.eclipse.org/legal/epl-2.0 + * + * SPDX-License-Identifier: EPL-2.0 + *******************************************************************************/ + +package org.eclipse.hono.gateway.azure; + +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.boot.ApplicationArguments; +import org.springframework.boot.ApplicationRunner; +import org.springframework.boot.SpringApplication; +import org.springframework.boot.autoconfigure.EnableAutoConfiguration; +import org.springframework.context.annotation.ComponentScan; + +import io.vertx.core.Vertx; + +/** + * The "Azure IoT Hub" Protocol Gateway main application class. + */ +@ComponentScan +@EnableAutoConfiguration +public class AzureIotHubGatewayApplication implements ApplicationRunner { + + private final Vertx vertx = Vertx.vertx(); + + @Autowired + private AzureIotHubMqttGateway azureIotHubMqttGateway; + + /** + * Starts the "Azure IoT Hub" Protocol Gateway application. + * + * @param args Command line arguments passed to the application. + */ + public static void main(final String[] args) { + SpringApplication.run(AzureIotHubGatewayApplication.class, args); + } + + @Override + public void run(final ApplicationArguments args) { + vertx.deployVerticle(azureIotHubMqttGateway); + } +} diff --git a/protocol-gateway/azure-mqtt-protocol-gateway/src/main/java/org/eclipse/hono/gateway/azure/AzureIotHubMqttGateway.java b/protocol-gateway/azure-mqtt-protocol-gateway/src/main/java/org/eclipse/hono/gateway/azure/AzureIotHubMqttGateway.java new file mode 100644 index 00000000..b61c22cc --- /dev/null +++ b/protocol-gateway/azure-mqtt-protocol-gateway/src/main/java/org/eclipse/hono/gateway/azure/AzureIotHubMqttGateway.java @@ -0,0 +1,367 @@ +/******************************************************************************* + * Copyright (c) 2020 Contributors to the Eclipse Foundation + * + * See the NOTICE file(s) distributed with this work for additional + * information regarding copyright ownership. + * + * This program and the accompanying materials are made available under the + * terms of the Eclipse Public License 2.0 which is available at + * http://www.eclipse.org/legal/epl-2.0 + * + * SPDX-License-Identifier: EPL-2.0 + *******************************************************************************/ + +package org.eclipse.hono.gateway.azure; + +import java.net.HttpURLConnection; +import java.util.HashMap; +import java.util.Map; +import java.util.Objects; +import java.util.Optional; + +import org.apache.qpid.proton.amqp.messaging.ApplicationProperties; +import org.eclipse.hono.auth.Device; +import org.eclipse.hono.client.ClientErrorException; +import org.eclipse.hono.config.ClientConfigProperties; +import org.eclipse.hono.gateway.sdk.mqtt2amqp.AbstractMqttProtocolGateway; +import org.eclipse.hono.gateway.sdk.mqtt2amqp.Command; +import org.eclipse.hono.gateway.sdk.mqtt2amqp.MqttCommandContext; +import org.eclipse.hono.gateway.sdk.mqtt2amqp.MqttDownstreamContext; +import org.eclipse.hono.gateway.sdk.mqtt2amqp.MqttProtocolGatewayConfig; +import org.eclipse.hono.gateway.sdk.mqtt2amqp.downstream.CommandResponseMessage; +import org.eclipse.hono.gateway.sdk.mqtt2amqp.downstream.DownstreamMessage; +import org.eclipse.hono.gateway.sdk.mqtt2amqp.downstream.EventMessage; +import org.eclipse.hono.gateway.sdk.mqtt2amqp.downstream.TelemetryMessage; + +import io.netty.handler.codec.mqtt.MqttQoS; +import io.vertx.core.Future; +import io.vertx.core.buffer.Buffer; +import io.vertx.core.json.JsonObject; + +/** + * A Protocol Gateway implementation that shows how production ready protocol gateways can be implemented using the base + * class {@link AbstractMqttProtocolGateway} on the example of the "Azure IoT Hub". It provides parts of the MQTT API of + * the Azure IoT Hub to show how to use the communication patterns that Hono provides. + *

+ * This is not intended to be fully compatible with Azure IoT Hub. Especially it has the following limitations: + *

+ *

+ * Device-to-cloud + * messages are sent as events (if published with QoS 1) or telemetry messages (if published with QoS 0). + *

+ * Received one-way commands are forwarded to the device as + * + * cloud-to-device messages. Received request/response commands are forwarded as + * direct + * method messages and direct method responses are sent as command responses. + */ +public class AzureIotHubMqttGateway extends AbstractMqttProtocolGateway { + + /** + * The topic to which device-to-cloud messages are being sent. + *

+ * The topic name is: {@code devices/{device_id}/messages/events/} or + * {@code devices/{device_id}/messages/events/{property_bag}}. + * + * @see + * Azure IoT Hub Documentation: "Sending device-to-cloud messages" + */ + public static final String EVENT_TOPIC_FORMAT_STRING = "devices/%s/messages/events/"; + + /** + * The topic filter to which devices have to subscribe for receiving cloud-to-device messages. + *

+ * The topic filter is: {@code devices/{device_id}/messages/devicebound/#}. + * + * @see + * Azure IoT Hub Documentation: "Receiving cloud-to-device messages" + */ + public static final String CLOUD_TO_DEVICE_TOPIC_FILTER_FORMAT_STRING = "devices/%s/messages/devicebound/#"; + + /** + * The topic to which cloud-to-device messages are being sent. + *

+ * The topic name is: {@code devices/{device_id}/messages/devicebound/} or + * {@code devices/{device_id}/messages/devicebound/{property_bag}}. + * + * @see + * Azure IoT Hub Documentation: "Receiving cloud-to-device messages" + */ + public static final String CLOUD_TO_DEVICE_TOPIC_FORMAT_STRING = "devices/%s/messages/devicebound/"; + + /** + * The topic filter to which devices have to subscribe for receiving direct method messages. + * + * @see + * Azure IoT Hub Documentation: "Respond to a direct method" + */ + public static final String DIRECT_METHOD_TOPIC_FILTER = "$iothub/methods/POST/#"; + + /** + * The topic to which direct method messages are being sent. + *

+ * The topic name is: {@code $iothub/methods/POST/{method name}/?$rid={request id}}. + *

+ * We use the command's subject as the method name. + * + * @see + * Azure IoT Hub Documentation: "Respond to a direct method" + */ + public static final String DIRECT_METHOD_TOPIC_FORMAT_STRING = "$iothub/methods/POST/%s/?$rid=%s"; + + /** + * The prefix of the topic to which a response to a direct method is being sent. + *

+ * The topic name is: {@code $iothub/methods/res/{status}/?$rid={request id}}. + * + * @see + * Azure IoT Hub Documentation: "Respond to a direct method" + */ + public static final String DIRECT_METHOD_RESPONSE_TOPIC_PREFIX = "$iothub/methods/res/"; + + private static final int DEVICE_TO_CLOUD_SIZE_LIMIT = 256 * 1024; // 256 KB + private static final int DIRECT_METHOD_SIZE_LIMIT = 128 * 1024; // 128 KB + private static final int CLOUD_TO_DEVICE_SIZE_LIMIT = 64 * 1024; // 64 KB + + private final DemoDeviceConfiguration demoDeviceConfig; + + /** + * Creates an instance. + * + * @param amqpClientConfig The AMQP client configuration. + * @param mqttServerConfig The MQTT server configuration. + * @param demoDeviceConfig The configuration for the demo device. + * @throws NullPointerException if any of the parameters is {@code null}. + * @see AbstractMqttProtocolGateway#AbstractMqttProtocolGateway(ClientConfigProperties, MqttProtocolGatewayConfig) + * The constructor of the superclass for details. + */ + public AzureIotHubMqttGateway(final ClientConfigProperties amqpClientConfig, + final MqttProtocolGatewayConfig mqttServerConfig, + final DemoDeviceConfiguration demoDeviceConfig) { + super(amqpClientConfig, mqttServerConfig); + + Objects.requireNonNull(demoDeviceConfig); + this.demoDeviceConfig = demoDeviceConfig; + } + + /** + * {@inheritDoc} + */ + @Override + protected Future authenticateDevice(final String username, final String password, final String clientId) { + if (demoDeviceConfig.getUsername().equals(username) && demoDeviceConfig.getPassword().equals(password)) { + return Future.succeededFuture(new Device(demoDeviceConfig.getTenantId(), demoDeviceConfig.getDeviceId())); + } else { + return Future.failedFuture(String.format("Authentication of device failed [username: %s]", username)); + } + } + + /** + * {@inheritDoc} + */ + @Override + protected Future onPublishedMessage(final MqttDownstreamContext ctx) { + final DownstreamMessage result; + + final String topic = ctx.topic(); + try { + + if (isDeviceToCloudTopic(topic, ctx.authenticatedDevice().getDeviceId())) { + result = createDeviceToCloudMessage(ctx); + } else if (isDirectMethodResponseTopic(topic)) { + result = createDirectMethodResponseMessage(ctx); + } else { + throw new RuntimeException("unknown message type for topic " + topic); + } + + } catch (RuntimeException e) { + return Future.failedFuture(new ClientErrorException(HttpURLConnection.HTTP_BAD_REQUEST, + "published message is invalid", e)); + } + + // TODO Does Azure IoT Hub set the topic somewhere in the message? + result.addApplicationProperty("topic", topic); + + PropertyBag.decode(topic).getPropertyBagIterator() + .forEachRemaining(prop -> result.addApplicationProperty(prop.getKey(), prop.getValue())); + + return Future.succeededFuture(result); + } + + private boolean isDeviceToCloudTopic(final String topic, final String deviceId) { + return topic.startsWith(getEventTopic(deviceId)); + } + + private boolean isDirectMethodResponseTopic(final String topic) { + return topic.startsWith(DIRECT_METHOD_RESPONSE_TOPIC_PREFIX); + } + + private DownstreamMessage createDeviceToCloudMessage(final MqttDownstreamContext ctx) { + final Buffer payload = ctx.message().payload(); + if (payload.length() > DEVICE_TO_CLOUD_SIZE_LIMIT) { + throw new IllegalArgumentException( + String.format("device-to-cloud message is limited to %s KB", DEVICE_TO_CLOUD_SIZE_LIMIT)); + } + + final DownstreamMessage result; + if (MqttQoS.AT_MOST_ONCE.equals(ctx.qosLevel())) { + result = new TelemetryMessage(payload, false); + } else { + result = new EventMessage(payload); + } + + if (ctx.message().isRetain()) { + result.addApplicationProperty("x-opt-retain", true); // TODO Which value does Azure IoT Hub set here? + } + return result; + } + + private DownstreamMessage createDirectMethodResponseMessage(final MqttDownstreamContext ctx) { + + validateDirectMethodPayload(ctx.message().payload()); + + final PropertyBag propertyBag = PropertyBag.decode(ctx.topic()); + final RequestId requestId = RequestId.decode(propertyBag.getProperty("$rid")); + + final String status = ctx.topic().split("\\/")[3]; + + final DownstreamMessage result = new CommandResponseMessage(requestId.getReplyId(), + requestId.getCorrelationId(), status, ctx.message().payload()); + + result.setContentType("application/json"); + return result; + } + + /** + * {@inheritDoc} + */ + @Override + protected boolean isTopicFilterValid(final String topicFilter, final String tenantId, final String deviceId, + final String clientId) { + + final boolean isCloudToDevice = getCloudToDeviceTopicFilter(deviceId).equals(topicFilter); + final boolean isDirectMethod = DIRECT_METHOD_TOPIC_FILTER.equals(topicFilter); + + return isCloudToDevice || isDirectMethod; + } + + /** + * {@inheritDoc} + */ + @Override + protected Command onCommandReceived(final MqttCommandContext ctx) { + + final String topic; + final String topicFilter; + if (ctx.isRequestResponseCommand()) { + validateDirectMethodPayload(ctx.getPayload()); + topic = getDirectMethodTopic(ctx); + topicFilter = DIRECT_METHOD_TOPIC_FILTER; + } else { + validateCloudToDeviceMessage(ctx); + topic = getCloudToDeviceTopic(ctx); + topicFilter = getCloudToDeviceTopicFilter(ctx.getDevice().getDeviceId()); + } + + return new Command(topic, topicFilter, ctx.getPayload()); + } + + private void validateDirectMethodPayload(final Buffer payload) { + if (payload.length() > DIRECT_METHOD_SIZE_LIMIT) { + throw new IllegalArgumentException( + String.format("direct method response is limited to %s KB", DIRECT_METHOD_SIZE_LIMIT)); + } + + if (payload.length() > 0) { + // validates that it is a JSON object + final JsonObject jsonObject = payload.toJsonObject(); + log.trace("payload is JSON with {} entries", jsonObject.size()); + } + } + + private void validateCloudToDeviceMessage(final MqttCommandContext ctx) { + if (ctx.getPayload().length() > CLOUD_TO_DEVICE_SIZE_LIMIT) { + throw new RuntimeException( + String.format("cloud-to-device message is limited to %s KB", CLOUD_TO_DEVICE_SIZE_LIMIT)); + } + } + + private String getDirectMethodTopic(final MqttCommandContext ctx) { + return getDirectMethodTopic(ctx.getSubject(), ctx.getReplyTo(), ctx.getCorrelationId()); + } + + private String getCloudToDeviceTopic(final MqttCommandContext ctx) { + + final String baseTopic = getCloudToDeviceTopic(ctx.getDevice().getDeviceId()); + + final Map properties = Optional.ofNullable(ctx.getApplicationProperties()) + .map(ApplicationProperties::getValue) + .orElse(new HashMap<>()); + + addPropertyToMap(properties, "subject", ctx.getSubject()); + + addPropertyToMap(properties, "$correlationId", ctx.getCorrelationId()); + addPropertyToMap(properties, "$messageId", ctx.getMessageId()); + + return PropertyBag.encode(baseTopic, properties); + } + + /** + * Returns the event topic for the given device id. + * + * @param deviceId The id of the device that send messages to this topic. + * @return The topic without a property bag. + */ + public static String getEventTopic(final String deviceId) { + return String.format(EVENT_TOPIC_FORMAT_STRING, deviceId); + } + + /** + * Returns the topic to which direct method messages are being sent for the given method name and request id. + * + * @param methodName The method name. This implementation uses the subject of the command message. + * @param replyToAddress The reply-to address from the command message. + * @param correlationId The correlation id from the command message. + * @return The topic. + */ + public static String getDirectMethodTopic(final String methodName, final String replyToAddress, + final Object correlationId) { + final String requestId = RequestId.encode(replyToAddress, correlationId); + + return String.format(DIRECT_METHOD_TOPIC_FORMAT_STRING, methodName, requestId); + } + + /** + * Returns the cloud-to-device topic for the given device id. + * + * @param deviceId The ID of the device to which the message is addressed. + * @return The topic without a property bag. + */ + public static String getCloudToDeviceTopic(final String deviceId) { + return String.format(CLOUD_TO_DEVICE_TOPIC_FORMAT_STRING, deviceId); + } + + /** + * Returns the cloud-to-device topic filter for the given device id. + * + * @param deviceId The ID of the device that subscribes for cloud-to-device messages. + * @return The topic filter. + */ + public static String getCloudToDeviceTopicFilter(final String deviceId) { + return String.format(CLOUD_TO_DEVICE_TOPIC_FILTER_FORMAT_STRING, deviceId); + } + + private void addPropertyToMap(final Map map, final String key, final Object value) { + if (value != null) { + map.put(key, value); + } + } + +} diff --git a/protocol-gateway/azure-mqtt-protocol-gateway/src/main/java/org/eclipse/hono/gateway/azure/Config.java b/protocol-gateway/azure-mqtt-protocol-gateway/src/main/java/org/eclipse/hono/gateway/azure/Config.java new file mode 100644 index 00000000..7e07e906 --- /dev/null +++ b/protocol-gateway/azure-mqtt-protocol-gateway/src/main/java/org/eclipse/hono/gateway/azure/Config.java @@ -0,0 +1,98 @@ +/******************************************************************************* + * Copyright (c) 2020 Contributors to the Eclipse Foundation + * + * See the NOTICE file(s) distributed with this work for additional + * information regarding copyright ownership. + * + * This program and the accompanying materials are made available under the + * terms of the Eclipse Public License 2.0 which is available at + * http://www.eclipse.org/legal/epl-2.0 + * + * SPDX-License-Identifier: EPL-2.0 + *******************************************************************************/ + +package org.eclipse.hono.gateway.azure; + +import java.util.List; + +import org.eclipse.hono.config.ClientConfigProperties; +import org.eclipse.hono.gateway.sdk.mqtt2amqp.MqttProtocolGatewayConfig; +import org.springframework.boot.context.properties.ConfigurationProperties; +import org.springframework.context.annotation.Bean; +import org.springframework.context.annotation.Configuration; + +/** + * Spring Boot configuration for the the "Azure IoT Hub" Protocol Gateway. + */ +@Configuration +public class Config { + + /** + * These are the default secure protocols in Vertx. + */ + public static final List enabledProtocols = List.of("TLSv1", "TLSv1.1", "TLSv1.2"); + + /** + * Exposes configuration properties for providing an MQTT server as a Spring bean. + *

+ * Sets the TLS protocols from {@link #enabledProtocols} as the enabled secure protocols of the MQTT server if not + * set explicitly. + * + * @return The properties. + */ + @Bean + @ConfigurationProperties(prefix = "hono.server.mqtt") + public MqttProtocolGatewayConfig mqttGatewayConfig() { + final MqttProtocolGatewayConfig mqttProtocolGatewayConfig = new MqttProtocolGatewayConfig(); + mqttProtocolGatewayConfig.setSecureProtocols(enabledProtocols); + return mqttProtocolGatewayConfig; + } + + /** + * Exposes configuration properties for accessing Hono's AMQP adapter as a Spring bean. + * + * @return The properties. + */ + @Bean + @ConfigurationProperties(prefix = "hono.client.amqp") + public ClientConfigProperties amqpClientConfig() { + final ClientConfigProperties amqpClientConfig = new ClientConfigProperties(); + amqpClientConfig.setServerRole("AMQP Adapter"); + return amqpClientConfig; + } + + /** + * Creates a new Azure IoT Hub protocol gateway instance. + * + * @return The new instance. + */ + @Bean + public AzureIotHubMqttGateway azureIotHubMqttGateway() { + final DemoDeviceConfiguration demoDeviceConfig = demoDevice(); + final ClientConfigProperties amqpClientConfig = amqpClientConfig(); + + if (demoDeviceConfig.getTenantId() == null || demoDeviceConfig.getDeviceId() == null) { + throw new IllegalArgumentException("Demo device is not configured."); + } + if (amqpClientConfig.getUsername() == null || amqpClientConfig.getPassword() == null) { + throw new IllegalArgumentException("Gateway credentials are not configured."); + } + if (amqpClientConfig.getHost() == null) { + throw new IllegalArgumentException("AMQP host is not configured."); + } + + return new AzureIotHubMqttGateway(amqpClientConfig, mqttGatewayConfig(), demoDeviceConfig); + } + + /** + * Exposes configuration properties for a demo device as a Spring bean. + * + * @return The demo device configuration against which the authentication of a connecting device is being performed. + */ + @Bean + @ConfigurationProperties(prefix = "hono.demo.device") + public DemoDeviceConfiguration demoDevice() { + return new DemoDeviceConfiguration(); + } + +} diff --git a/protocol-gateway/azure-mqtt-protocol-gateway/src/main/java/org/eclipse/hono/gateway/azure/DemoDeviceConfiguration.java b/protocol-gateway/azure-mqtt-protocol-gateway/src/main/java/org/eclipse/hono/gateway/azure/DemoDeviceConfiguration.java new file mode 100644 index 00000000..54253285 --- /dev/null +++ b/protocol-gateway/azure-mqtt-protocol-gateway/src/main/java/org/eclipse/hono/gateway/azure/DemoDeviceConfiguration.java @@ -0,0 +1,97 @@ +/******************************************************************************* + * Copyright (c) 2020 Contributors to the Eclipse Foundation + * + * See the NOTICE file(s) distributed with this work for additional + * information regarding copyright ownership. + * + * This program and the accompanying materials are made available under the + * terms of the Eclipse Public License 2.0 which is available at + * http://www.eclipse.org/legal/epl-2.0 + * + * SPDX-License-Identifier: EPL-2.0 + *******************************************************************************/ + +package org.eclipse.hono.gateway.azure; + +/** + * A collection of properties to configure a static device for demonstration purposes. + */ +public class DemoDeviceConfiguration { + + private String tenantId; + private String deviceId; + private String username; + private String password; + + /** + * Gets the tenant to which the device belongs. + * + * @return The tenant id. + */ + public String getTenantId() { + return tenantId; + } + + /** + * Sets the tenant to which the device belongs. + * + * @param tenantId The tenant id. + */ + public void setTenantId(final String tenantId) { + this.tenantId = tenantId; + } + + /** + * Sets the device id. + * + * @return The device id. + */ + public String getDeviceId() { + return deviceId; + } + + /** + * Sets the device id. + * + * @param deviceId The device id. + */ + public void setDeviceId(final String deviceId) { + this.deviceId = deviceId; + } + + /** + * Gets the allowed username for the device. + * + * @return The username. + */ + public String getUsername() { + return username; + } + + /** + * Sets the allowed username for the device. + * + * @param username The username. + */ + public void setUsername(final String username) { + this.username = username; + } + + /** + * Gets the allowed password for the device. + * + * @return Sets the allowed username for the device.. + */ + public String getPassword() { + return password; + } + + /** + * Sets the allowed password for the device. + * + * @param password The password. + */ + public void setPassword(final String password) { + this.password = password; + } +} diff --git a/protocol-gateway/azure-mqtt-protocol-gateway/src/main/java/org/eclipse/hono/gateway/azure/PropertyBag.java b/protocol-gateway/azure-mqtt-protocol-gateway/src/main/java/org/eclipse/hono/gateway/azure/PropertyBag.java new file mode 100644 index 00000000..060bfc24 --- /dev/null +++ b/protocol-gateway/azure-mqtt-protocol-gateway/src/main/java/org/eclipse/hono/gateway/azure/PropertyBag.java @@ -0,0 +1,117 @@ +/******************************************************************************* + * Copyright (c) 2020 Contributors to the Eclipse Foundation + * + * See the NOTICE file(s) distributed with this work for additional + * information regarding copyright ownership. + * + * This program and the accompanying materials are made available under the + * terms of the Eclipse Public License 2.0 which is available at + * http://www.eclipse.org/legal/epl-2.0 + * + * SPDX-License-Identifier: EPL-2.0 + *******************************************************************************/ +package org.eclipse.hono.gateway.azure; + +import java.util.AbstractMap; +import java.util.Collections; +import java.util.Iterator; +import java.util.List; +import java.util.Map; +import java.util.Objects; +import java.util.Optional; + +import io.netty.handler.codec.http.QueryStringDecoder; +import io.netty.handler.codec.http.QueryStringEncoder; + +/** + * A collection of methods for processing a property-bag in MQTT topics. + * + */ +public final class PropertyBag { + + private final Map> properties; + private final String topicWithoutPropertyBag; + + private PropertyBag(final String topicWithoutPropertyBag, final Map> properties) { + this.properties = properties; + this.topicWithoutPropertyBag = topicWithoutPropertyBag; + } + + /** + * Creates a property bag object from the given topic by retrieving all the properties from the + * property-bag. + * + * @param topic The topic that the message has been published to. + * @return The property bag object or {@code null} if no property-bag is set in the topic. + * @throws NullPointerException if topic is {@code null}. + */ + public static PropertyBag decode(final String topic) { + + Objects.requireNonNull(topic); + + final int index = topic.lastIndexOf("?"); + if (index > 0) { + return new PropertyBag( + topic.substring(0, index), + new QueryStringDecoder(topic.substring(index)).parameters()); + } + return new PropertyBag(topic, null); + } + + /** + * Creates a topic with the given properties as an URL-encoded property bag. + * + * @param baseTopic The topic to which the properties are appended. + * @param properties The properties to encode into the result - may be {@code null}. + * @return A topic string ending with the property bag or the base topic if no properties passed in. + * @throws NullPointerException if the base topic is {@code null}. + */ + public static String encode(final String baseTopic, final Map properties) { + Objects.requireNonNull(baseTopic); + + if (properties == null) { + return baseTopic; + } else { + final QueryStringEncoder queryStringEncoder = new QueryStringEncoder(baseTopic); + properties.forEach((k, v) -> queryStringEncoder.addParam(k, v.toString())); + return queryStringEncoder.toString(); + } + } + + /** + * Gets a property value from the property-bag. + * + * @param name The property name. + * @return The property value or {@code null} if the property is not set. + */ + public String getProperty(final String name) { + return Optional.ofNullable(properties) + .map(props -> props.get(name)) + .map(values -> values.get(0)) + .orElse(null); + } + + /** + * Gets an iterator iterating over the properties. + * + * @return The properties iterator. + */ + public Iterator> getPropertyBagIterator() { + return Optional.ofNullable(properties) + .map(props -> props.entrySet().stream() + .map(entry -> (Map.Entry) new AbstractMap.SimpleEntry<>(entry.getKey(), + entry.getValue() != null ? entry.getValue().get(0) : null)) + .iterator()) + .orElse(Collections.emptyIterator()); + } + + /** + * Returns the topic without the property-bag. + * + * @return The topic without the property-bag. + */ + public String topicWithoutPropertyBag() { + return topicWithoutPropertyBag; + } + +} diff --git a/protocol-gateway/azure-mqtt-protocol-gateway/src/main/java/org/eclipse/hono/gateway/azure/RequestId.java b/protocol-gateway/azure-mqtt-protocol-gateway/src/main/java/org/eclipse/hono/gateway/azure/RequestId.java new file mode 100644 index 00000000..72500e64 --- /dev/null +++ b/protocol-gateway/azure-mqtt-protocol-gateway/src/main/java/org/eclipse/hono/gateway/azure/RequestId.java @@ -0,0 +1,115 @@ +/******************************************************************************* + * Copyright (c) 2020 Contributors to the Eclipse Foundation + * + * See the NOTICE file(s) distributed with this work for additional + * information regarding copyright ownership. + * + * This program and the accompanying materials are made available under the + * terms of the Eclipse Public License 2.0 which is available at + * http://www.eclipse.org/legal/epl-2.0 + * + * SPDX-License-Identifier: EPL-2.0 + *******************************************************************************/ + +package org.eclipse.hono.gateway.azure; + +import java.util.Objects; + +/** + * This class encodes the reply-to-id and the correlation-id of a command into a single string and decodes it from a + * command response back into the two values. + *

+ * The reply-to-id as well as the correlation-id can be freely chosen by the backend application and are not restricted + * regarding the used characters. Therefore no reserved delimiters can be used here, instead fixed positions are used. + * The first two characters encode the length of the correlation-id, then the correlation-id is appended and at the end + * the reply-to-id. + *

+ * The maximal expected length is 255 characters. + */ +public class RequestId { + + /** + * The maximal length of the correlation-id. + */ + public static final int MAX_LENGTH_CORRELATION_ID_HEX = 2; + + private final String replyId; + private final String correlationId; + + private RequestId(final String replyId, final String correlationId) { + this.replyId = replyId; + this.correlationId = correlationId; + } + + /** + * Gets the reply-id that has been decoded from a request-id. + * + * @return The reply-id. + */ + public String getReplyId() { + return replyId; + } + + /** + * Gets the correlation-id that has been decoded from a request-id. + * + * @return The correlation-id. + */ + public String getCorrelationId() { + return correlationId; + } + + /** + * Decodes the given request-id back into separate reply-id and correlation-id. + * + * @param requestId The encoded request-id. + * @return The object containing reply-id and correlation-id. + * @throws IllegalArgumentException if parsing or decoding fails. + */ + public static RequestId decode(final String requestId) { + + try { + final int correlationIdEnd = Integer.parseInt(requestId.substring(0, MAX_LENGTH_CORRELATION_ID_HEX), 16) + + MAX_LENGTH_CORRELATION_ID_HEX; + final String correlationId = requestId.substring(MAX_LENGTH_CORRELATION_ID_HEX, correlationIdEnd); + final String replyId = requestId.substring(correlationIdEnd); + + return new RequestId(replyId, correlationId); + } catch (final RuntimeException e) { + throw new IllegalArgumentException("Failed to decode request-id [" + requestId + "]", e); + } + } + + /** + * Combines the reply-id (which is part of the reply-to address) with the correlation-id into a single string. + * + * @param replyTo the reply-to of the command message. + * @param correlationId the correlation-id of the command message. + * @return the encoded request-id. + * @throws NullPointerException if any of the params is {@code null}. + * @throws IllegalArgumentException if the parameters do not comply to Hono's Command and Control API or if the + * correlation-id is longer than 255 chars. + * @see #decode(String) + */ + public static String encode(final String replyTo, final Object correlationId) { + Objects.requireNonNull(replyTo); + Objects.requireNonNull(correlationId); + + if (!(correlationId instanceof String)) { + throw new IllegalArgumentException("correlation-id must be a string"); + } else { + final String correlationIdString = ((String) correlationId); + if (correlationIdString.length() > 255) { + throw new IllegalArgumentException("correlationId is too long"); + } + + final String[] replyToElements = replyTo.split("\\/"); + if (replyToElements.length <= 3) { + throw new IllegalArgumentException("reply-to address is malformed"); + } else { + return String.format("%02x%s%s", correlationIdString.length(), correlationIdString, replyToElements[3]); + } + } + } + +} diff --git a/protocol-gateway/azure-mqtt-protocol-gateway/src/main/resources/application-ssl.properties b/protocol-gateway/azure-mqtt-protocol-gateway/src/main/resources/application-ssl.properties new file mode 100644 index 00000000..31705d48 --- /dev/null +++ b/protocol-gateway/azure-mqtt-protocol-gateway/src/main/resources/application-ssl.properties @@ -0,0 +1,17 @@ +#******************************************************************************* +# Copyright (c) 2020 Contributors to the Eclipse Foundation +# +# See the NOTICE file(s) distributed with this work for additional +# information regarding copyright ownership. +# +# This program and the accompanying materials are made available under the +# terms of the Eclipse Public License 2.0 which is available at +# http://www.eclipse.org/legal/epl-2.0 +# +# SPDX-License-Identifier: EPL-2.0 +#******************************************************************************* + +hono.server.mqtt.bindAddress=0.0.0.0 +hono.server.mqtt.port=8883 +hono.server.mqtt.keyPath=target/config/hono-demo-certs-jar/example-gateway-key.pem +hono.server.mqtt.certPath=target/config/hono-demo-certs-jar/example-gateway-cert.pem diff --git a/protocol-gateway/azure-mqtt-protocol-gateway/src/main/resources/application.properties b/protocol-gateway/azure-mqtt-protocol-gateway/src/main/resources/application.properties new file mode 100644 index 00000000..32ee99db --- /dev/null +++ b/protocol-gateway/azure-mqtt-protocol-gateway/src/main/resources/application.properties @@ -0,0 +1,28 @@ +#******************************************************************************* +# Copyright (c) 2020 Contributors to the Eclipse Foundation +# +# See the NOTICE file(s) distributed with this work for additional +# information regarding copyright ownership. +# +# This program and the accompanying materials are made available under the +# terms of the Eclipse Public License 2.0 which is available at +# http://www.eclipse.org/legal/epl-2.0 +# +# SPDX-License-Identifier: EPL-2.0 +#******************************************************************************* + +logging.level.org.eclipse.hono.gateway=TRACE +logging.level.org.eclipse.hono.gateway.sdk=DEBUG + +hono.server.mqtt.bindAddress=0.0.0.0 +hono.server.mqtt.port=1883 +hono.client.amqp.host=hono.eclipseprojects.io +hono.client.amqp.port=5671 +hono.client.amqp.tlsEnabled=true +hono.demo.device.username=demo1 +hono.demo.device.password=demo-secret + +#hono.demo.device.deviceId=4712 +#hono.client.amqp.password=gw-secret +#hono.client.amqp.username=gw@DEFAULT_TENANT +#hono.demo.device.tenantId=DEFAULT_TENANT diff --git a/protocol-gateway/azure-mqtt-protocol-gateway/src/test/java/org/eclipse/hono/gateway/azure/AzureIotHubMqttGatewayTest.java b/protocol-gateway/azure-mqtt-protocol-gateway/src/test/java/org/eclipse/hono/gateway/azure/AzureIotHubMqttGatewayTest.java new file mode 100644 index 00000000..61572255 --- /dev/null +++ b/protocol-gateway/azure-mqtt-protocol-gateway/src/test/java/org/eclipse/hono/gateway/azure/AzureIotHubMqttGatewayTest.java @@ -0,0 +1,286 @@ +/******************************************************************************* + * Copyright (c) 2020 Contributors to the Eclipse Foundation + * + * See the NOTICE file(s) distributed with this work for additional + * information regarding copyright ownership. + * + * This program and the accompanying materials are made available under the + * terms of the Eclipse Public License 2.0 which is available at + * http://www.eclipse.org/legal/epl-2.0 + * + * SPDX-License-Identifier: EPL-2.0 + *******************************************************************************/ + +package org.eclipse.hono.gateway.azure; + +import static org.assertj.core.api.Assertions.assertThat; +import static org.mockito.Mockito.mock; +import static org.mockito.Mockito.when; + +import java.util.HashMap; + +import org.apache.qpid.proton.amqp.Binary; +import org.apache.qpid.proton.amqp.messaging.ApplicationProperties; +import org.apache.qpid.proton.amqp.messaging.Data; +import org.apache.qpid.proton.message.Message; +import org.eclipse.hono.auth.Device; +import org.eclipse.hono.config.ClientConfigProperties; +import org.eclipse.hono.gateway.sdk.mqtt2amqp.Command; +import org.eclipse.hono.gateway.sdk.mqtt2amqp.MqttCommandContext; +import org.eclipse.hono.gateway.sdk.mqtt2amqp.MqttDownstreamContext; +import org.eclipse.hono.gateway.sdk.mqtt2amqp.MqttProtocolGatewayConfig; +import org.eclipse.hono.gateway.sdk.mqtt2amqp.downstream.CommandResponseMessage; +import org.eclipse.hono.gateway.sdk.mqtt2amqp.downstream.DownstreamMessage; +import org.eclipse.hono.gateway.sdk.mqtt2amqp.downstream.EventMessage; +import org.eclipse.hono.gateway.sdk.mqtt2amqp.downstream.TelemetryMessage; +import org.eclipse.hono.util.CommandConstants; +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; + +import io.netty.handler.codec.mqtt.MqttQoS; +import io.vertx.core.Future; +import io.vertx.core.buffer.Buffer; +import io.vertx.core.json.JsonObject; +import io.vertx.mqtt.MqttEndpoint; +import io.vertx.mqtt.messages.MqttPublishMessage; + +/** + * Verifies behavior of {@link AzureIotHubMqttGateway}. + */ +public class AzureIotHubMqttGatewayTest { + + private static final String CORRELATION_ID = "666"; + private static final String MESSAGE_ID = "777"; + private static final String APPLICATION_PROPERTY_KEY = "foo"; + private static final String APPLICATION_PROPERTY_VALUE = "bar"; + + private static final String REPLY_ID = "XXXX"; + private static final String REPLY_TO_ADDRESS = "command_response/test-tenant/test-device/" + REPLY_ID; + private static final String REQUEST_ID = RequestId.encode(REPLY_TO_ADDRESS, CORRELATION_ID); + + private static final String TENANT_ID = "test-tenant"; + private static final String DEVICE_ID = "device1"; + private static final String CLIENT_ID = "the-client-id"; + + private static final String cloudToDeviceTopicFilter = String + .format(AzureIotHubMqttGateway.CLOUD_TO_DEVICE_TOPIC_FILTER_FORMAT_STRING, DEVICE_ID); + + private static final String directMessageTopicFilter = AzureIotHubMqttGateway.DIRECT_METHOD_TOPIC_FILTER; + + private final Device device = new Device(TENANT_ID, DEVICE_ID); + private final Buffer payload = new JsonObject().put("a-key", "a-value").toBuffer(); + private final DemoDeviceConfiguration demoDeviceConfig = new DemoDeviceConfiguration(); + + private Message commandMessage; + private AzureIotHubMqttGateway underTest; + + /** + * Sets up the fixture. + */ + @BeforeEach + public void setUp() { + + commandMessage = mock(Message.class); + when(commandMessage.getBody()).thenReturn(new Data(new Binary(payload.getBytes()))); + when(commandMessage.getMessageId()).thenReturn(MESSAGE_ID); + when(commandMessage.getCorrelationId()).thenReturn(CORRELATION_ID); + final HashMap propertiesMap = new HashMap<>(); + propertiesMap.put(APPLICATION_PROPERTY_KEY, APPLICATION_PROPERTY_VALUE); + final ApplicationProperties applicationProperties = new ApplicationProperties(propertiesMap); + when(commandMessage.getApplicationProperties()).thenReturn(applicationProperties); + + demoDeviceConfig.setDeviceId(DEVICE_ID); + demoDeviceConfig.setTenantId(TENANT_ID); + demoDeviceConfig.setUsername("the-username"); + demoDeviceConfig.setPassword("super secret"); + + underTest = new AzureIotHubMqttGateway(new ClientConfigProperties(), new MqttProtocolGatewayConfig(), + demoDeviceConfig); + } + + /** + * Verifies the command object when a one-way command message is received. + */ + @Test + public void testReceiveOneWayCommand() { + + // GIVEN a command context for a one-way command + final MqttCommandContext commandContext = MqttCommandContext.fromAmqpMessage(commandMessage, device); + + // WHEN the message is received + final Command command = underTest.onCommandReceived(commandContext); + + // THEN the returned command object contains the correct topic filter and payload... + assertThat(command.getTopicFilter()).isEqualTo(cloudToDeviceTopicFilter); + assertThat(command.getPayload()).isEqualTo(payload); + + // ...AND the topic contains the expected properties + final PropertyBag propertyBag = PropertyBag.decode(command.getTopic()); + + assertThat(propertyBag.topicWithoutPropertyBag()).isEqualTo("devices/" + DEVICE_ID + "/messages/devicebound/"); + + assertThat(propertyBag.getProperty("$messageId")).isEqualTo(MESSAGE_ID); + assertThat(propertyBag.getProperty("$correlationId")).isEqualTo(CORRELATION_ID); + assertThat(propertyBag.getProperty(APPLICATION_PROPERTY_KEY)).isEqualTo(APPLICATION_PROPERTY_VALUE); + + } + + /** + * Verifies the command object when a request/response command message is received. + */ + @Test + public void testReceiveRequestResponseCommand() { + final String subject = "the-subject"; + + // GIVEN a command context for a request/response command + when(commandMessage.getReplyTo()).thenReturn(REPLY_TO_ADDRESS); + when(commandMessage.getSubject()).thenReturn(subject); + + final MqttCommandContext commandContext = MqttCommandContext.fromAmqpMessage(commandMessage, device); + + // WHEN the message is received + final Command command = underTest.onCommandReceived(commandContext); + + // THEN the returned command object contains the correct topic filter, payload and topic + assertThat(command.getTopicFilter()).isEqualTo(directMessageTopicFilter); + assertThat(command.getPayload()).isEqualTo(payload); + + assertThat(command.getTopic()).isEqualTo("$iothub/methods/POST/" + subject + "/?$rid=" + REQUEST_ID); + + } + + /** + * Verifies that authentication succeeds for the credentials in the demo-device config. + */ + @Test + public void authenticateDeviceSucceeds() { + + final Future deviceFuture = underTest.authenticateDevice(demoDeviceConfig.getUsername(), + demoDeviceConfig.getPassword(), CLIENT_ID); + + assertThat(deviceFuture.succeeded()).isTrue(); + final Device result = deviceFuture.result(); + + assertThat(result).isNotNull(); + assertThat(result.getTenantId()).isEqualTo(TENANT_ID); + assertThat(result.getDeviceId()).isEqualTo(DEVICE_ID); + + } + + /** + * Verifies that authentication fails for unknown credentials. + */ + @Test + public void authenticateDeviceFailsForWrongCredentials() { + final String user = demoDeviceConfig.getUsername(); + assertThat(underTest.authenticateDevice(user, "wrong-password", CLIENT_ID).succeeded()).isFalse(); + + final String password = demoDeviceConfig.getPassword(); + assertThat(underTest.authenticateDevice("wrong-username", password, CLIENT_ID).succeeded()).isFalse(); + } + + /** + * Verifies that MQTT messages with QoS 0 on the event topic are sent downstream as telemetry messages. + */ + @Test + public void testOnPublishedMessageForTelemetryMessage() { + + // GIVEN an MQTT message with QoS 0 + final MqttPublishMessage mqttPublishMessage = mock(MqttPublishMessage.class); + when(mqttPublishMessage.payload()).thenReturn(payload); + when(mqttPublishMessage.topicName()).thenReturn(AzureIotHubMqttGateway.getEventTopic(DEVICE_ID)); + when(mqttPublishMessage.qosLevel()).thenReturn(MqttQoS.AT_MOST_ONCE); + + final MqttDownstreamContext downstreamContext = MqttDownstreamContext.fromPublishPacket(mqttPublishMessage, + mock(MqttEndpoint.class), device); + + // WHEN the message is received + final Future messageFuture = underTest.onPublishedMessage(downstreamContext); + + // THEN a telemetry message with the payload is returned + assertThat(messageFuture.succeeded()).isTrue(); + final DownstreamMessage result = messageFuture.result(); + + assertThat(result).isInstanceOfAny(TelemetryMessage.class); + assertThat(result.getPayload()).isEqualTo(payload.getBytes()); + + } + + /** + * Verifies that MQTT messages with QoS 1 on the event topic are sent downstream as event messages. + */ + @Test + public void testOnPublishedMessageForEventMessage() { + + // GIVEN an MQTT message with QoS 1 + final MqttPublishMessage mqttPublishMessage = mock(MqttPublishMessage.class); + when(mqttPublishMessage.payload()).thenReturn(payload); + when(mqttPublishMessage.topicName()).thenReturn(AzureIotHubMqttGateway.getEventTopic(DEVICE_ID)); + when(mqttPublishMessage.qosLevel()).thenReturn(MqttQoS.AT_LEAST_ONCE); + + final MqttDownstreamContext downstreamContext = MqttDownstreamContext.fromPublishPacket(mqttPublishMessage, + mock(MqttEndpoint.class), device); + + // WHEN the message is received + final Future messageFuture = underTest.onPublishedMessage(downstreamContext); + + // THEN an event message with the payload is returned + assertThat(messageFuture.succeeded()).isTrue(); + final DownstreamMessage result = messageFuture.result(); + + assertThat(result).isInstanceOfAny(EventMessage.class); + assertThat(result.getPayload()).isEqualTo(payload.getBytes()); + + } + + /** + * Verifies that MQTT messages on the direct method response topic are sent downstream as command response messages. + */ + @Test + public void testOnPublishedMessageForCommandResponse() { + final int status = 200; + + // GIVEN an MQTT message with the direct method response topic + final MqttPublishMessage mqttPublishMessage = mock(MqttPublishMessage.class); + when(mqttPublishMessage.payload()).thenReturn(payload); + when(mqttPublishMessage.qosLevel()).thenReturn(MqttQoS.AT_LEAST_ONCE); + when(mqttPublishMessage.topicName()).thenReturn(AzureIotHubMqttGateway.DIRECT_METHOD_RESPONSE_TOPIC_PREFIX + + status + "/?$rid=" + REQUEST_ID); + + final MqttDownstreamContext downstreamContext = MqttDownstreamContext.fromPublishPacket(mqttPublishMessage, + mock(MqttEndpoint.class), device); + + // WHEN the message is received + final Future messageFuture = underTest.onPublishedMessage(downstreamContext); + + // THEN a command response message with the payload is returned... + assertThat(messageFuture.succeeded()).isTrue(); + final DownstreamMessage result = messageFuture.result(); + + assertThat(result).isInstanceOfAny(CommandResponseMessage.class); + assertThat(result.getPayload()).isEqualTo(payload.getBytes()); + + // ...AND its parameters are set correctly + final CommandResponseMessage responseMessage = (CommandResponseMessage) result; + assertThat(responseMessage.getCorrelationId()).isEqualTo(CORRELATION_ID); + assertThat(responseMessage.getStatus()).isEqualTo(status); + assertThat(responseMessage.getTargetAddress(TENANT_ID, DEVICE_ID)).isEqualTo(String.format("%s/%s/%s/%s", + CommandConstants.NORTHBOUND_COMMAND_RESPONSE_ENDPOINT, TENANT_ID, DEVICE_ID, REPLY_ID)); + assertThat(responseMessage.getContentType()).isEqualTo("application/json"); + } + + /** + * Verifies that the topic filters for cloud-to-device messages and for direct method responses are validated + * successfully and other topic filters fail. + */ + @Test + public void isTopicFilterValid() { + + assertThat(underTest.isTopicFilterValid(cloudToDeviceTopicFilter, null, DEVICE_ID, null)).isTrue(); + assertThat(underTest.isTopicFilterValid(directMessageTopicFilter, null, null, null)).isTrue(); + + final String unknownTopicFilter = "foo/#"; + assertThat(underTest.isTopicFilterValid(unknownTopicFilter, null, DEVICE_ID, null)).isFalse(); + } + +} diff --git a/protocol-gateway/azure-mqtt-protocol-gateway/src/test/java/org/eclipse/hono/gateway/azure/PropertyBagTest.java b/protocol-gateway/azure-mqtt-protocol-gateway/src/test/java/org/eclipse/hono/gateway/azure/PropertyBagTest.java new file mode 100644 index 00000000..56216561 --- /dev/null +++ b/protocol-gateway/azure-mqtt-protocol-gateway/src/test/java/org/eclipse/hono/gateway/azure/PropertyBagTest.java @@ -0,0 +1,104 @@ +/** + * Copyright (c) 2019, 2020 Contributors to the Eclipse Foundation + * + * See the NOTICE file(s) distributed with this work for additional + * information regarding copyright ownership. + * + * This program and the accompanying materials are made available under the + * terms of the Eclipse Public License 2.0 which is available at + * http://www.eclipse.org/legal/epl-2.0 + * + * SPDX-License-Identifier: EPL-2.0 + */ + +package org.eclipse.hono.gateway.azure; + +import static org.assertj.core.api.Assertions.assertThat; + +import java.util.HashMap; +import java.util.Iterator; +import java.util.Map; + +import org.junit.jupiter.api.Test; + +/** + * Verifies behavior of {@link PropertyBag}. + * + */ +public class PropertyBagTest { + + private static final String TOPIC_WITHOUT_PROPERTY_BAG = "devices/4712/messages/events/"; + private static final String KEY1 = "a"; + private static final String VALUE1 = "b"; + private static final String ENCODED1 = "a=b"; + private static final String KEY2 = "foo bar"; + private static final String VALUE2 = "b/a/z"; + private static final String ENCODED2 = "foo%20bar=b%2Fa%2Fz"; + private static final String ENCODED_TOPIC = TOPIC_WITHOUT_PROPERTY_BAG + "?" + ENCODED1 + "&" + ENCODED2; + + /** + * Verifies that properties are set in the property-bag of the message's topic. + */ + @Test + public void testDecodePropertyBag() { + final PropertyBag propertyBag = PropertyBag.decode(ENCODED_TOPIC); + assertThat((Object) propertyBag.topicWithoutPropertyBag()).isEqualTo(TOPIC_WITHOUT_PROPERTY_BAG); + assertThat((Object) propertyBag.getProperty(KEY1)).isEqualTo(VALUE1); + assertThat((Object) propertyBag.getProperty(KEY2)).isEqualTo(VALUE2); + } + + /** + * Verifies that the property-bag is trimmed from a topic string and the rest is returned. + */ + @Test + public void testDecodeTopicWithoutPropertyBag() { + assertThat((Object) PropertyBag.decode(TOPIC_WITHOUT_PROPERTY_BAG).topicWithoutPropertyBag()) + .isEqualTo(TOPIC_WITHOUT_PROPERTY_BAG); + } + + /** + * Verifies that getPropertiesIterator returns an iterator with the expected entries. + */ + @Test + public void testGetPropertyBagIterator() { + final PropertyBag propertyBag = PropertyBag.decode(ENCODED_TOPIC); + assertThat(propertyBag).isNotNull(); + final Iterator> propertiesIterator = propertyBag.getPropertyBagIterator(); + final Map tmpMap = new HashMap<>(); + propertiesIterator.forEachRemaining((entry) -> tmpMap.put(entry.getKey(), entry.getValue())); + assertThat((Object) tmpMap.size()).isEqualTo(2); + assertThat((Object) tmpMap.get(KEY1)).isEqualTo(VALUE1); + assertThat((Object) tmpMap.get(KEY2)).isEqualTo(VALUE2); + } + + /** + * Verifies that properties get encoded into the topic. + */ + @Test + public void testEncode() { + final String topicWithPropertyBag = PropertyBag.encode(TOPIC_WITHOUT_PROPERTY_BAG, + Map.of(KEY1, VALUE1, KEY2, VALUE2)); + + // the order of the properties might differ + assertThat(topicWithPropertyBag).startsWith(TOPIC_WITHOUT_PROPERTY_BAG + "?"); + assertThat(topicWithPropertyBag).contains(ENCODED1); + assertThat(topicWithPropertyBag).contains(ENCODED2); + assertThat(topicWithPropertyBag).contains("&"); + assertThat(topicWithPropertyBag).hasSize(53); + } + + /** + * Verifies that the encoded topic can be decoded with the expected results. + */ + @Test + public void testDecodeEncodedTopic() { + final String topicWithPropertyBag = PropertyBag.encode(TOPIC_WITHOUT_PROPERTY_BAG, + Map.of(KEY1, VALUE1, KEY2, VALUE2)); + + final PropertyBag propertyBag = PropertyBag.decode(topicWithPropertyBag); + assertThat((Object) propertyBag.topicWithoutPropertyBag()).isEqualTo(TOPIC_WITHOUT_PROPERTY_BAG); + assertThat((Object) propertyBag.getProperty(KEY2)).isEqualTo(VALUE2); + assertThat((Object) propertyBag.getProperty(KEY1)).isEqualTo(VALUE1); + } + +} diff --git a/protocol-gateway/azure-mqtt-protocol-gateway/src/test/java/org/eclipse/hono/gateway/azure/RequestIdTest.java b/protocol-gateway/azure-mqtt-protocol-gateway/src/test/java/org/eclipse/hono/gateway/azure/RequestIdTest.java new file mode 100644 index 00000000..595c037e --- /dev/null +++ b/protocol-gateway/azure-mqtt-protocol-gateway/src/test/java/org/eclipse/hono/gateway/azure/RequestIdTest.java @@ -0,0 +1,78 @@ +/******************************************************************************* + * Copyright (c) 2020 Contributors to the Eclipse Foundation + * + * See the NOTICE file(s) distributed with this work for additional + * information regarding copyright ownership. + * + * This program and the accompanying materials are made available under the + * terms of the Eclipse Public License 2.0 which is available at + * http://www.eclipse.org/legal/epl-2.0 + * + * SPDX-License-Identifier: EPL-2.0 + *******************************************************************************/ + +package org.eclipse.hono.gateway.azure; + +import static org.assertj.core.api.Assertions.assertThat; +import static org.junit.jupiter.api.Assertions.assertThrows; + +import java.util.Arrays; + +import org.junit.jupiter.api.Test; + +/** + * Verifies the behavior of {@link RequestId}. + */ +public class RequestIdTest { + + private static final String REPLY_ID = "999"; + private static final String CORRELATION_ID = "888"; + private static final String REQUEST_ID = "03888999"; + private static final int MAX_LENGTH = 255; + + /** + * Verifies that encoding a correlation id and reply id (from a reply-to address) results in the expected request + * id. + */ + @Test + public void testEncode() { + final String replyTo = "command_response/test-tenant/test-device/" + REPLY_ID; + + assertThat(RequestId.encode(replyTo, CORRELATION_ID)).isEqualTo("03888999"); + + } + + /** + * Verifies that decoding the request id results in the expected reply id and correlation id. + */ + @Test + public void testDecode() { + + final RequestId requestId = RequestId.decode(REQUEST_ID); + assertThat(requestId.getCorrelationId()).isEqualTo(CORRELATION_ID); + assertThat(requestId.getReplyId()).isEqualTo(REPLY_ID); + } + + /** + * Verifies that trying to encode too long correlation ids result in the expected exception. + */ + @Test + public void testMaxCorrelationIdLength() { + + // GIVEN a correlation id that is longer than the max length allowed ... + final char[] tooManyChars = new char[MAX_LENGTH + 1]; + Arrays.fill(tooManyChars, '8'); + final String tooLongCorrelationId = new String(tooManyChars); + + // ... AND a correlation id that is 1 character shorter + final char[] maxChars = Arrays.copyOf(tooManyChars, MAX_LENGTH); + final String maxCorrelationId = new String(maxChars); + + // WHEN encoding the short correlation id THEN no exception is thrown ... + RequestId.encode("command_response/test-tenant/test-device/" + REPLY_ID, maxCorrelationId); + + // ... WHEN encoding longer correlation id THEN the expected exception is thrown + assertThrows(IllegalArgumentException.class, + () -> RequestId.encode("command_response/test-tenant/test-device/" + REPLY_ID, tooLongCorrelationId)); + } +} diff --git a/protocol-gateway/mqtt-protocol-gateway-template/README.md b/protocol-gateway/mqtt-protocol-gateway-template/README.md new file mode 100644 index 00000000..7c29a843 --- /dev/null +++ b/protocol-gateway/mqtt-protocol-gateway-template/README.md @@ -0,0 +1,118 @@ +# MQTT Protocol Gateway Template + + +This project provides a template for creating MQTT +[protocol gateways](https://www.eclipse.org/hono/docs/concepts/connecting-devices/#connecting-via-a-protocol-gateway) for Eclipse Hono™. +While Hono contains an [MQTT protocol adapter](https://www.eclipse.org/hono/docs/user-guide/mqtt-adapter/), there are cases when +it cannot be used, e.g., when existing MQTT-enabled devices cannot be updated to use the supported topics of the default protocol adapter. +Other use cases for an MQTT protocol gateway are special requirements that the protocol adapter does not support, +such as custom authentication or message transformations, like compression or encryption of the payload. + +This template allows creating protocol gateways quickly and easily. It accepts messages on custom topics and sends +them to Hono's [AMQP adapter](https://www.eclipse.org/hono/docs/user-guide/amqp-adapter/). +Received commands can be published on custom topics to the device. + +The following diagram shows how two devices are connected to an MQTT protocol gateway in the backend, +which in turn is connected to Hono's AMQP protocol adapter. + +![Diagram shows devices connected to protocol gateway via MQTT, which connects to the AMQP adapter](device-via-mqtt-protocol-gw.svg) + + +## Implement a custom MQTT Protocol Gateway + +The template is provided in form of the abstract class `AbstractMqttProtocolGateway`. +The abstract methods must be implemented to handle the following events: + +1. when a device connects: authenticate the device +1. when a device subscribes: validate the topic filters +1. when a business application sends a command: + * determine the topic on which the device expects it + * select the corresponding subscription + * (optional) modify the payload +1. when a device sends a message: + * select the message type (telemetry, event, command response) + * (if command response) correlate the message to the command + * (optional) modify the payload + * (optional) specify the content type + * (optional) add "application properties" + +The abstract class is configured by its constructor parameters. +The `ClientConfigProperties` configure the connection to the AMQP protocol adapter and the `MqttProtocolGatewayConfig` +configures the protocol gateway including the MQTT server. + +**NB** When receiving commands, the AMQP message is settled (and accepted) as soon as the message has been successfully +published to the device. The implementation does not wait for an acknowledgement from the device, regardless of the +QoS with which the device has subscribed. +If the business application requires a higher delivery guarantee, it is recommended to use request-response commands. + +### Device Registration + +The template presumes that the gateway device as well as the devices connecting to it are registered in Hono's +device registry and that the gateway is authorized for the device using the _via_ property, as described on the +[concept page](https://www.eclipse.org/hono/docs/concepts/connecting-devices/#connecting-via-a-device-gateway). + +It is not necessary to provision credentials for the devices to Hono's device registry. + + +### Device Authentication + +The protocol gateway is responsible for establishing and verifying the identity of devices. +This template supports both the authentication based on the username and password provided in +an MQTT CONNECT packet and client certificate-based authentication as part of a TLS handshake for this purpose. + +For authentication with username, the abstract method _authenticateDevice_ must be implemented. +The provided credentials must be checked there and the tenant ID and device ID of the device must be returned. +Authentication with username is not executed if authentication with client certificate was successful. + +To authenticate devices that use X.509 client certificates, two methods must be overridden: one provides the +issuer certificates to be used as trust anchors for validation, and the other must determine the tenant ID and +device ID for the validated client certificate. + +To authenticate client certificates that are not based on the X.509 standard, the 'authenticateDeviceCertificate' +method must be overwritten and the entire validation and authentication process implemented. + +Authentication with client certificates is only invoked if the connection is secured with TLS. +If it fails, the authentication with username is then invoked. + + +### Correlation of Commands and Responses + +Hono's [Command & Control](https://www.eclipse.org/hono/docs/api/command-and-control) API requires that a +command response must include the correlation ID and the response ID of the command. +When using MQTT, there is no canonical way to do this. The following three approaches are conceivable: + +1. encode the parameters in the topic and have the device send them back +2. encode them into the payload and have them returned by the device +3. keep them in the protocol gateway and add them there to the response + +If the values are sent to the device (first two options), the device must support this and add the +correct data to the response in the required manner. +The 3rd approach means that the protocol gateway is stateful and has the disadvantage that the protocol gateway +can no longer simply be scaled horizontally. The template allows to use any of these strategies. + + +### Gateway Authentication + +Gateways must be registered as devices in Hono's device registry, and the corresponding credentials for authentication must be created. + +If all devices connecting to the protocol gateway belong to the same tenant, the credentials of the gateway device +can be configured in the `ClientConfigProperties`, which are passed in the constructor of the class `AbstractMqttProtocolGateway`. +If devices of different tenants are to be connected, the credentials must be determined dynamically, as described below. + + +### Multi-tenancy + +The protocol gateway supports the handling of devices belonging to different tenants. +For this purpose, for each tenant that is supposed to use the protocol gateway, a gateway device must be registered +in the device registry and corresponding credentials must be created. + +The method _provideGatewayCredentials_ must be overwritten. This method is invoked after the (successful) authentication +of a device to provide the gateway credentials for this tenant. + +**NB** If credentials for the gateway are present in the configuration, the method _provideGatewayCredentials_ is _not_ invoked. + + +### Optional Extension Points + +The abstract base class exposes some `protected` methods that may be used to extend the behavior of the protocol gateway. +Please refer to the JavaDoc for details. diff --git a/protocol-gateway/mqtt-protocol-gateway-template/device-via-mqtt-protocol-gw.svg b/protocol-gateway/mqtt-protocol-gateway-template/device-via-mqtt-protocol-gw.svg new file mode 100644 index 00000000..ca6fc0cd --- /dev/null +++ b/protocol-gateway/mqtt-protocol-gateway-template/device-via-mqtt-protocol-gw.svg @@ -0,0 +1,106 @@ + + + + + + + + + + + + + + Device 1 + + + + Device 2 + + + Internet + + + + + + MQTT Protocol Gateway + + + + + + AMQP Adapter + + + MQTT + + + MQTT + + + MQTT + + + AMQP + + + diff --git a/protocol-gateway/mqtt-protocol-gateway-template/pom.xml b/protocol-gateway/mqtt-protocol-gateway-template/pom.xml new file mode 100644 index 00000000..c917205c --- /dev/null +++ b/protocol-gateway/mqtt-protocol-gateway-template/pom.xml @@ -0,0 +1,242 @@ + + + + + 4.0.0 + + org.eclipse.hono + hono-mqtt-protocol-gateway + 0.0.1-SNAPSHOT + bundle + + Hono MQTT Protocol Gateway Template + A template for MQTT protocol gateways for Hono. + https://www.eclipse.org/hono + 2020 + + + Eclipse Foundation + https://www.eclipse.org/ + + + + + Eclipse Public License - Version 2.0 + http://www.eclipse.org/legal/epl-2.0 + SPDX-License-Identifier: EPL-2.0 + + + + + UTF-8 + UTF-8 + 1.8 + + + yyyy-MM-dd + ${maven.build.timestamp} + + 3.15.0 + 1.3.0-M3 + 5.6.0 + 3.3.3 + 3.9.1 + + + + + org.eclipse.hono + hono-client + ${hono.version} + + + org.eclipse.hono + hono-legal + ${hono.version} + + + io.vertx + vertx-mqtt + ${vertx.version} + + + org.mockito + mockito-core + ${mockito.version} + test + + + org.junit.jupiter + junit-jupiter-engine + ${junit.jupiter.version} + test + + + io.vertx + vertx-junit5 + ${vertx.version} + test + + + org.assertj + assertj-core + ${assertj-core.version} + test + + + + + + + + org.apache.maven.plugins + maven-javadoc-plugin + 3.1.0 + + ${java.level} + + + + + + + org.apache.maven.plugins + maven-compiler-plugin + 3.8.1 + + ${java.level} + ${java.level} + UTF-8 + + + + org.codehaus.mojo + animal-sniffer-maven-plugin + 1.18 + + + verify_java8_compatibility + test + + check + + + + + + org.apache.maven.plugins + maven-javadoc-plugin + 3.1.0 + + all,-accessibility + + + + org.apache.maven.plugins + maven-surefire-plugin + 2.22.2 + + + + junit.jupiter.execution.parallel.enabled = true + junit.jupiter.execution.parallel.mode.default = same_thread + junit.jupiter.execution.parallel.mode.classes.default = concurrent + + + + + + org.apache.maven.plugins + maven-failsafe-plugin + 2.22.2 + + + + org.apache.maven.plugins + maven-dependency-plugin + 3.1.1 + + + copy_legal_docs + prepare-package + + unpack-dependencies + + + hono-legal + ${project.build.outputDirectory}/META-INF + legal/** + + + + + + org.apache.felix + maven-bundle-plugin + 3.5.0 + true + + + + + META-INF=${project.build.outputDirectory}/META-INF + + + {local-packages} + + + osgi.ee;filter:="(&(osgi.ee=JavaSE)(version=${java.level}))" + + + + + + org.apache.maven.plugins + maven-checkstyle-plugin + 3.1.1 + + + org.eclipse.hono + hono-legal + ${hono.version} + + + com.puppycrawl.tools + checkstyle + 8.32 + + + + + checkstyle-check + verify + + check + + + + + checkstyle/default.xml + checkstyle/suppressions.xml + true + + + + + + diff --git a/protocol-gateway/mqtt-protocol-gateway-template/src/main/java/org/eclipse/hono/gateway/sdk/mqtt2amqp/AbstractMqttProtocolGateway.java b/protocol-gateway/mqtt-protocol-gateway-template/src/main/java/org/eclipse/hono/gateway/sdk/mqtt2amqp/AbstractMqttProtocolGateway.java new file mode 100644 index 00000000..aa775d56 --- /dev/null +++ b/protocol-gateway/mqtt-protocol-gateway-template/src/main/java/org/eclipse/hono/gateway/sdk/mqtt2amqp/AbstractMqttProtocolGateway.java @@ -0,0 +1,926 @@ +/******************************************************************************* + * Copyright (c) 2020 Contributors to the Eclipse Foundation + * + * See the NOTICE file(s) distributed with this work for additional + * information regarding copyright ownership. + * + * This program and the accompanying materials are made available under the + * terms of the Eclipse Public License 2.0 which is available at + * http://www.eclipse.org/legal/epl-2.0 + * + * SPDX-License-Identifier: EPL-2.0 + *******************************************************************************/ + +package org.eclipse.hono.gateway.sdk.mqtt2amqp; + +import java.net.HttpURLConnection; +import java.security.cert.Certificate; +import java.security.cert.TrustAnchor; +import java.security.cert.X509Certificate; +import java.util.ArrayList; +import java.util.Arrays; +import java.util.HashMap; +import java.util.LinkedHashSet; +import java.util.List; +import java.util.Map; +import java.util.Objects; +import java.util.Set; +import java.util.stream.Collectors; + +import javax.net.ssl.SSLPeerUnverifiedException; + +import org.apache.qpid.proton.message.Message; +import org.eclipse.hono.auth.Device; +import org.eclipse.hono.client.ClientErrorException; +import org.eclipse.hono.client.HonoConnection; +import org.eclipse.hono.client.MessageConsumer; +import org.eclipse.hono.client.ServiceInvocationException; +import org.eclipse.hono.client.device.amqp.AmqpAdapterClientFactory; +import org.eclipse.hono.config.ClientConfigProperties; +import org.eclipse.hono.gateway.sdk.mqtt2amqp.downstream.CommandResponseMessage; +import org.eclipse.hono.gateway.sdk.mqtt2amqp.downstream.DownstreamMessage; +import org.eclipse.hono.gateway.sdk.mqtt2amqp.downstream.EventMessage; +import org.eclipse.hono.gateway.sdk.mqtt2amqp.downstream.TelemetryMessage; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import io.netty.handler.codec.mqtt.MqttConnectReturnCode; +import io.netty.handler.codec.mqtt.MqttQoS; +import io.vertx.core.AbstractVerticle; +import io.vertx.core.CompositeFuture; +import io.vertx.core.Future; +import io.vertx.core.Handler; +import io.vertx.core.Promise; +import io.vertx.core.Vertx; +import io.vertx.core.http.ClientAuth; +import io.vertx.core.net.KeyCertOptions; +import io.vertx.core.net.NetServerOptions; +import io.vertx.core.net.TrustOptions; +import io.vertx.mqtt.MqttAuth; +import io.vertx.mqtt.MqttEndpoint; +import io.vertx.mqtt.MqttServer; +import io.vertx.mqtt.MqttServerOptions; +import io.vertx.mqtt.messages.MqttSubscribeMessage; +import io.vertx.mqtt.messages.MqttUnsubscribeMessage; +import io.vertx.proton.ProtonDelivery; + +/** + * Base class for implementing a protocol gateway that connects to a Hono AMQP adapter and provides a custom MQTT server + * for devices to connect to. + *

+ * This implementation does not support MQTT QoS 2; when a device requests QoS 2 in its SUBSCRIBE message, only + * QoS 1 is granted. + *

+ * When receiving commands, the AMQP message is settled with the outcome accepted as soon as the message has + * been successfully published to the device. The implementation does not wait for an acknowledgement from the device, + * regardless of the QoS with which the device has subscribed. + */ +public abstract class AbstractMqttProtocolGateway extends AbstractVerticle { + + /** + * A logger to be shared with subclasses. + */ + protected final Logger log = LoggerFactory.getLogger(getClass()); + + private final ClientConfigProperties amqpClientConfig; + private final MqttProtocolGatewayConfig mqttGatewayConfig; + private final Map clientFactoryPerTenant = new HashMap<>(); + + private MqttServer server; + + /** + * Creates an instance. + *

+ * The AMQP client configuration needs to contain the properties that are required to connect to the Hono AMQP + * adapter. If it contains a username and password, those are used to authenticate the amqp client with. Otherwise + * {@link #provideGatewayCredentials(String)} needs to be overridden in order to dynamically resolve credentials for + * the tenant of a device request ("multi-tenant mode"). + * + * @param amqpClientConfig The AMQP client configuration. + * @param mqttGatewayConfig The configuration of the protocol gateway. + * @throws NullPointerException if any of the parameters is {@code null}. + * @see ClientConfigProperties#setTlsEnabled(boolean) + * @see ClientConfigProperties#setTrustStorePath(String) + */ + public AbstractMqttProtocolGateway(final ClientConfigProperties amqpClientConfig, + final MqttProtocolGatewayConfig mqttGatewayConfig) { + Objects.requireNonNull(amqpClientConfig); + Objects.requireNonNull(mqttGatewayConfig); + + this.amqpClientConfig = amqpClientConfig; + this.mqttGatewayConfig = mqttGatewayConfig; + } + + /** + * Authenticates a device that has provided the specified credentials in its CONNECT packet. This method is not + * invoked if the client certificate-based authentication was already successful. + *

+ * Implementations must return a (succeeded) future with the authenticated device if authentication was + * successful or a failed future otherwise. {@code Null} must never be returned. + * + * @param username The username. + * @param password The password. + * @param clientId The client id. + * @return A future indicating the outcome of the operation. + */ + protected abstract Future authenticateDevice(String username, String password, String clientId); + + /** + * Validates the topic filter that a device sent in its subscription message. Additional information is provided + * with the parameters that can be used for validation. + * + * @param topicFilter the topic filter provided by the device. + * @param tenantId the tenant id of the authenticated device. + * @param deviceId the device id of the authenticated device. + * @param clientId the MQTT client id of the device. + * @return {@code true} if the topic filter is valid. + */ + protected abstract boolean isTopicFilterValid(String topicFilter, String tenantId, String deviceId, + String clientId); + + /** + * This method is called when a message has been published by a device via MQTT. It prepares the data to be uploaded + * to Hono. + *

+ * Subclasses determine the message type by returning one of the subclasses of {@link DownstreamMessage}. + * + * @param ctx The context in which the MQTT message has been published. + * @return A future indicating the outcome of the operation. If an error occurs, a failed future is returned, but + * never {@code null}. If the failure has been caused by the device that published the message, the (failed) + * future contains a {@link ClientErrorException}. + */ + protected abstract Future onPublishedMessage(MqttDownstreamContext ctx); + + /** + * This method is called when a command message that has been received from Hono. It prepares the data to be + * published to the device via MQTT. + *

+ * If the implementation throws an exception, the AMQP command message will be released. + * + * @param ctx The context in which the command has been received. + * @return The command to be published to the device - must not be {@code null}. + */ + protected abstract Command onCommandReceived(MqttCommandContext ctx); + + /** + * Gets credentials for authentication against the AMQP adapter to which this protocol gateway connects. If username + * and password are specified in the AMQP client configuration of this gateway, then these are used and this method + * is not invoked. + *

+ * Subclasses should overwrite this method to resolve the credentials for the given client. + *

+ * This default implementation returns a failed future because it is only called if no configuration with username + * and password is provided and it is not overwritten by an alternative implementation. + *

+ * The method must never return {@code null}. + * + * @param tenantId The tenant for which a connection is required (from the device authentication). + * @return A future indicating the outcome of the operation. + * @see ClientConfigProperties#setUsername(String) + * @see ClientConfigProperties#setPassword(String) + */ + protected Future provideGatewayCredentials(final String tenantId) { + return Future.failedFuture("credentials of the protocol gateway not found in the provided configuration."); + } + + /** + * Invoked when a message has been forwarded downstream successfully. + *

+ * This default implementation does nothing. + *

+ * Subclasses should override this method in order to e.g. update metrics counters. + * + * @param ctx The context in which the MQTT message has been published. + */ + protected void onMessageSent(final MqttDownstreamContext ctx) { + } + + /** + * Invoked when a message could not be forwarded downstream. + *

+ * This method will only be invoked if the failure to forward the message has not been caused by the device that + * published the message. + *

+ * This default implementation does nothing. + *

+ * Subclasses should override this method in order to e.g. update metrics counters. + * + * @param ctx The context in which the MQTT message has been published. + */ + protected void onMessageUndeliverable(final MqttDownstreamContext ctx) { + } + + /** + * Invoked when a message has been sent to the device successfully. + *

+ * This default implementation does nothing. + *

+ * Subclasses should override this method in order to e.g. update metrics counters. + * + * @param command The received command message. + * @param subscription The corresponding subscription. + */ + protected void onCommandPublished(final Message command, final CommandSubscription subscription) { + } + + /** + * Invoked before the connection with a device is closed. + *

+ * Subclasses should override this method in order to release any device specific resources. + *

+ * This default implementation does nothing. + * + * @param endpoint The connection to be closed. + */ + protected void onDeviceConnectionClose(final MqttEndpoint endpoint) { + } + + /** + * Authenticates a device using its TLS client certificate. This method is only invoked if the device establishes a + * connection with TLS and presents a client certificate. + *

+ * If authentication fails, the username/password based authentication + * ({@link #authenticateDevice(String, String, String)}) will be invoked afterwards. + *

+ * To authenticate devices using client certificates, subclasses must either (a) override methods + * {@link #getTrustAnchors(List)} and {@link #authenticateClientCertificate(X509Certificate)} if only X.509 + * certificates are used, or (b) override this method if other certificate types are to be used. + *

+ * This default implementation only validates X.509 certificates. It performs the following steps if the previous + * steps were successful: + *

    + *
  1. invoke {@link #getTrustAnchors(List)}
    2. + *
  2. validate the given certificate chain against the trust anchors
    3. + *
  3. invoke {@link #authenticateClientCertificate(X509Certificate)}
    4. + *
+ * If one of the steps fails (which they do, unless the above methods are overridden in a subclass), this method + * returns a failed future, which causes username/password based authentication to be invoked. + * + * @param path The certificate path from the TLS session with the client certificate first - not {@code null}. + * @return A future indicating the outcome of the operation. The future will succeed with the device data belonging + * to the authentication or it will fail with a failure message indicating the cause of the failure. + * {@code Null} must never be returned. + * + * @see #getTrustAnchors(List) + * @see #authenticateClientCertificate(X509Certificate) + */ + protected Future authenticateDeviceCertificate(final Certificate[] path) { + + final List certificates = Arrays.stream(path) + .filter(cert -> cert instanceof X509Certificate) + .map(cert -> ((X509Certificate) cert)) + .collect(Collectors.toList()); + + final X509CertificateValidator validator = new X509CertificateValidator(); + return getTrustAnchors(certificates) + .compose(trustAnchors -> validator.validate(certificates, trustAnchors)) + .compose(ok -> authenticateClientCertificate(certificates.get(0))); + } + + /** + * Returns the trust anchors to be used to validate the X.509 client certificate of a device. + *

+ * Subclasses should override this method to provide trust anchors against which the device certificate can be + * validated. + *

+ * To authenticate devices using client certificates, subclasses must either (a) override this method and + * {@link #authenticateClientCertificate(X509Certificate)} if only X.509 certificates are used, or (b) override + * method {@link #authenticateDeviceCertificate(Certificate[])} if other certificate types are to be used. + *

+ * This default implementation always returns a failed future because there are no default trust anchors. + * + * @param certificates The certificate chain to be validated, which - depending on the actual implementation - may + * be necessary to select the relevant trust anchors. + * @return A future indicating the outcome of the operation. The future will succeed with the trust anchors to be + * used for the validation or it will fail with a failure message indicating the cause of the failure. + * {@code Null} must never be returned. + * + * @see #authenticateDeviceCertificate(Certificate[]) + * @see #authenticateClientCertificate(X509Certificate) + */ + protected Future> getTrustAnchors(final List certificates) { + return Future.failedFuture("Client certificate can not be validated: no trust anchors provided"); + } + + /** + * Authenticates the X.509 client certificate and returns the authenticated device. + *

+ * Subclasses should override this method to check if the given certificate identifies is a known and authorized + * device and to retrieve the tenant id and the device id for it. + *

+ * To authenticate devices using client certificates, subclasses must either (a) override this method and + * {@link #getTrustAnchors(List)} if only X.509 certificates are used, or (b) override method + * {@link #authenticateDeviceCertificate(Certificate[])} if other certificate types are to be used. + *

+ * This default implementation always returns a failed future. + * + * @param deviceCertificate The already validated client certificate. + * @return A future indicating the outcome of the operation. The future will succeed with the authenticated device + * or it will fail with a failure message indicating the cause of the failure. {@code Null} must never be + * returned. + * + * @see #authenticateDeviceCertificate(Certificate[]) + * @see #getTrustAnchors(List) + */ + protected Future authenticateClientCertificate(final X509Certificate deviceCertificate) { + return Future.failedFuture("Cannot establish device identity"); + } + + /** + * Invoked when a device sends its CONNECT packet. + *

+ * Authenticates the device, connects the gateway to Hono's AMQP adapter and registers handlers for processing + * messages published by the client. + * + * @param endpoint The MQTT endpoint representing the client. + * @throws NullPointerException if the endpoint is {@code null}. + */ + final void handleEndpointConnection(final MqttEndpoint endpoint) { + + Objects.requireNonNull(endpoint); + + log.debug("connection request from client [client-id: {}]", endpoint.clientIdentifier()); + + if (!endpoint.isCleanSession()) { + log.debug("ignoring client's intent to resume existing session"); + } + if (endpoint.will() != null) { + log.debug("ignoring client's last will"); + } + + final Future authAttempt = tryAuthenticationWithClientCertificate(endpoint) + .recover(ex -> authenticateWithUsernameAndPassword(endpoint)) + .compose(authenticateDevice -> (authenticateDevice == null) + ? Future.failedFuture("device authentication failed") + : Future.succeededFuture(authenticateDevice)); + + authAttempt + .compose(this::connectGatewayToAmqpAdapter) + .onComplete(result -> { + if (result.succeeded()) { + registerHandlers(endpoint, authAttempt.result()); + log.debug("connection accepted from {}", authAttempt.result().toString()); + endpoint.accept(false); // we do not maintain session state + } else { + final MqttConnectReturnCode returnCode; + if (authAttempt.failed()) { + log.debug("connection request from client [clientId: {}] rejected, authentication failed", + endpoint.clientIdentifier(), authAttempt.cause()); + returnCode = MqttConnectReturnCode.CONNECTION_REFUSED_NOT_AUTHORIZED; + } else { + log.debug( + "connection request from client [clientId: {}] rejected, connection to backend failed", + endpoint.clientIdentifier(), result.cause()); + returnCode = MqttConnectReturnCode.CONNECTION_REFUSED_SERVER_UNAVAILABLE; + } + + endpoint.reject(returnCode); + } + }); + } + + private Future tryAuthenticationWithClientCertificate(final MqttEndpoint endpoint) { + if (endpoint.isSsl()) { + try { + final Certificate[] path = endpoint.sslSession().getPeerCertificates(); + if (path != null && path.length > 0) { + final Future authAttempt = authenticateDeviceCertificate(path); + log.debug("authentication with client certificate: {}.", + (authAttempt.succeeded()) ? "succeeded" : "failed"); + return authAttempt; + } + } catch (RuntimeException | SSLPeerUnverifiedException e) { + log.debug("could not retrieve client certificate from device endpoint: {}", e.getMessage()); + } + } + return Future.failedFuture(new ClientErrorException(HttpURLConnection.HTTP_UNAUTHORIZED)); + } + + private Future authenticateWithUsernameAndPassword(final MqttEndpoint endpoint) { + final MqttAuth auth = endpoint.auth(); + if (auth == null || auth.getUsername() == null || auth.getPassword() == null) { + return Future.failedFuture(new ClientErrorException(HttpURLConnection.HTTP_UNAUTHORIZED, + "device did not provide credentials in CONNECT packet")); + } else { + final Future authenticatedDevice = authenticateDevice(auth.getUsername(), auth.getPassword(), + endpoint.clientIdentifier()); + if (authenticatedDevice == null) { + return Future.failedFuture(new ClientErrorException(HttpURLConnection.HTTP_INTERNAL_ERROR)); + } else { + log.debug("authentication with username/password {}.", + (authenticatedDevice.succeeded()) ? "succeeded" : "failed"); + return authenticatedDevice; + } + } + } + + private Future connectGatewayToAmqpAdapter(final Device authenticatedDevice) { + + final String tenantId = authenticatedDevice.getTenantId(); + + if (amqpClientConfig.getUsername() != null && amqpClientConfig.getPassword() != null) { + return connectGatewayToAmqpAdapter(tenantId, amqpClientConfig); + } else { + return provideGatewayCredentials(tenantId) + .compose(credentials -> { + final ClientConfigProperties tenantConfig = new ClientConfigProperties(amqpClientConfig); + tenantConfig.setUsername(credentials.getUsername()); + tenantConfig.setPassword(credentials.getPassword()); + + return connectGatewayToAmqpAdapter(tenantId, tenantConfig); + }); + } + } + + private Future connectGatewayToAmqpAdapter(final String tenantId, final ClientConfigProperties clientConfig) { + + final AmqpAdapterClientFactory amqpAdapterClientFactory = clientFactoryPerTenant.get(tenantId); + if (amqpAdapterClientFactory != null) { + return amqpAdapterClientFactory.isConnected(clientConfig.getConnectTimeout()); + } else { + + final AmqpAdapterClientFactory factory = createTenantClientFactory(tenantId, clientConfig); + clientFactoryPerTenant.put(tenantId, factory); + + return factory.connect() + .map(con -> { + log.debug("Connected to AMQP adapter"); + return null; + }); + } + } + + /** + * Returns a new {@link AmqpAdapterClientFactory} with a new AMQP connection for the given tenant. + *

+ * This method is only visible for testing purposes. + * + * @param tenantId The tenant to be connected. + * @param clientConfig The client properties to use for the connection. + * @return The factory. Note that the underlying AMQP connection will not be established until + * {@link AmqpAdapterClientFactory#connect()} is invoked. + */ + AmqpAdapterClientFactory createTenantClientFactory(final String tenantId, + final ClientConfigProperties clientConfig) { + final HonoConnection connection = HonoConnection.newConnection(vertx, clientConfig); + return AmqpAdapterClientFactory.create(connection, tenantId); + } + + private void registerHandlers(final MqttEndpoint endpoint, final Device authenticatedDevice) { + + endpoint.publishHandler( + message -> handlePublishedMessage( + MqttDownstreamContext.fromPublishPacket(message, endpoint, authenticatedDevice))); + + final CommandSubscriptionsManager cmdSubscriptionsManager = createCommandHandler(vertx); + endpoint.closeHandler(v -> close(endpoint, cmdSubscriptionsManager)); + endpoint.publishAcknowledgeHandler(cmdSubscriptionsManager::handlePubAck); + endpoint.subscribeHandler(msg -> onSubscribe(endpoint, authenticatedDevice, msg, cmdSubscriptionsManager)); + endpoint.unsubscribeHandler(msg -> onUnsubscribe(endpoint, authenticatedDevice, msg, cmdSubscriptionsManager)); + + } + + private void close(final MqttEndpoint endpoint, final CommandSubscriptionsManager cmdSubscriptionsManager) { + onDeviceConnectionClose(endpoint); + cmdSubscriptionsManager.removeAllSubscriptions(); + if (endpoint.isConnected()) { + log.debug("closing connection with client [client ID: {}]", endpoint.clientIdentifier()); + endpoint.close(); + } else { + log.trace("connection to client is already closed"); + } + } + + /** + * Invoked when a device connects, after authentication. + *

+ * This method is only visible for testing purposes. + * + * @param vertx The vert.x instance + * @return The command handler for the given device. + */ + CommandSubscriptionsManager createCommandHandler(final Vertx vertx) { + return new CommandSubscriptionsManager(vertx, mqttGatewayConfig); + } + + /** + * Invoked when a device publishes a message. + * + * Invokes {@link #onPublishedMessage(MqttDownstreamContext)}, uploads the message to Hono's AMQP adapter. + * Afterwards it invokes {@link #onMessageSent(MqttDownstreamContext)} if the message has been forwarded + * successfully or if a the message could not be delivered, {@link #onMessageUndeliverable(MqttDownstreamContext)}. + * + * @param ctx The context in which the MQTT message has been published. + * @throws NullPointerException if the context is {@code null}. + */ + private void handlePublishedMessage(final MqttDownstreamContext ctx) { + + Objects.requireNonNull(ctx); + + onPublishedMessage(ctx) + .compose(downstreamMessage -> uploadMessage(downstreamMessage, ctx)) + .onComplete(processing -> { + if (processing.succeeded()) { + onUploadSuccess(ctx); + onMessageSent(ctx); + } else { + onUploadFailure(ctx, processing.cause()); + } + }); + } + + private Future uploadMessage(final DownstreamMessage downstreamMessage, + final MqttDownstreamContext ctx) { + + final String tenantId = ctx.authenticatedDevice().getTenantId(); + final String deviceId = ctx.authenticatedDevice().getDeviceId(); + final Map properties = downstreamMessage.getApplicationProperties(); + final byte[] payload = downstreamMessage.getPayload(); + final String contentType = downstreamMessage.getContentType(); + + if (downstreamMessage instanceof TelemetryMessage) { + + final TelemetryMessage telemetryMessage = (TelemetryMessage) downstreamMessage; + return sendTelemetry(tenantId, deviceId, properties, payload, contentType, + telemetryMessage.shouldWaitForOutcome()); + + } else if (downstreamMessage instanceof EventMessage) { + + return sendEvent(tenantId, deviceId, properties, payload, contentType); + + } else if (downstreamMessage instanceof CommandResponseMessage) { + + final CommandResponseMessage response = (CommandResponseMessage) downstreamMessage; + return sendCommandResponse(tenantId, deviceId, response.getTargetAddress(tenantId, deviceId), + response.getCorrelationId(), response.getStatus(), payload, contentType, properties); + + } else { + return Future.failedFuture(new ClientErrorException(HttpURLConnection.HTTP_BAD_REQUEST, + String.format("uploading message failed [topic: %s]. Unknown message type [%s]", ctx.topic(), + downstreamMessage.getClass().getSimpleName()))); + } + } + + private void onUploadSuccess(final MqttDownstreamContext ctx) { + log.debug("successfully processed message [topic: {}, QoS: {}] from device [tenantId: {}, deviceId: {}]", + ctx.topic(), ctx.qosLevel(), ctx.authenticatedDevice().getTenantId(), + ctx.authenticatedDevice().getDeviceId()); + // check that the remote MQTT client is still connected before sending PUBACK + if (MqttQoS.AT_LEAST_ONCE.equals(ctx.qosLevel()) && ctx.deviceEndpoint().isConnected()) { + log.debug("sending PUBACK"); + ctx.acknowledge(); + } + } + + private void onUploadFailure(final MqttDownstreamContext ctx, final Throwable cause) { + + final int statusCode = ServiceInvocationException.extractStatusCode(cause); + + if (statusCode < 500) { + log.debug("Publish message [topic: {}] from {} failed with client error: ", ctx.topic(), + ctx.authenticatedDevice(), cause); + } else { + log.info("Publish message [topic: {}] from {} failed: ", ctx.topic(), ctx.authenticatedDevice(), cause); + onMessageUndeliverable(ctx); + } + + if (ctx.deviceEndpoint().isConnected()) { + log.info("closing connection to device {}", ctx.authenticatedDevice().toString()); + ctx.deviceEndpoint().close(); + } + } + + private Future sendTelemetry(final String tenantId, final String deviceId, + final Map properties, final byte[] payload, final String contentType, + final boolean waitForOutcome) { + + return clientFactoryPerTenant.get(tenantId).getOrCreateTelemetrySender() + .compose(sender -> { + if (waitForOutcome) { + log.trace( + "sending telemetry message and wait for outcome [tenantId: {}, deviceId: {}, contentType: {}, properties: {}]", + tenantId, deviceId, contentType, properties); + return sender.sendAndWaitForOutcome(deviceId, payload, contentType, properties); + } else { + log.trace( + "sending telemetry message [tenantId: {}, deviceId: {}, contentType: {}, properties: {}]", + tenantId, deviceId, contentType, properties); + return sender.send(deviceId, payload, contentType, properties); + } + }); + } + + private Future sendEvent(final String tenantId, final String deviceId, + final Map properties, final byte[] payload, final String contentType) { + + log.trace("sending event message [tenantId: {}, deviceId: {}, contentType: {}, properties: {}]", + tenantId, deviceId, contentType, properties); + + return clientFactoryPerTenant.get(tenantId).getOrCreateEventSender() + .compose(sender -> sender.send(deviceId, payload, contentType, properties)); + } + + private Future sendCommandResponse(final String tenantId, final String deviceId, + final String targetAddress, final String correlationId, final int status, final byte[] payload, + final String contentType, final Map properties) { + + log.trace( + "sending command response [tenantId: {}, deviceId: {}, targetAddress: {}, correlationId: {}, status: {}, contentType: {}, properties: {}]", + tenantId, deviceId, targetAddress, correlationId, status, contentType, properties); + + return clientFactoryPerTenant.get(tenantId).getOrCreateCommandResponseSender() + .compose(sender -> sender.sendCommandResponse(deviceId, targetAddress, correlationId, status, payload, + contentType, properties)); + } + + /** + * Invoked when a device sends an MQTT SUBSCRIBE packet. + * + * It invokes {@link #isTopicFilterValid(String, String, String, String)} for each topic filter in the subscribe + * packet. If there is a valid topic filter and no command consumer already exists for this device, this method + * opens a device-specific command consumer for receiving commands from applications for the device. + * + * @param endpoint The endpoint representing the connection to the device. + * @param authenticatedDevice The authenticated identity of the device. + * @param subscribeMsg The subscribe request received from the device. + * @param cmdSubscriptionsManager The CommandSubscriptionsManager to track command subscriptions, unsubscriptions + * and handle PUBACKs. + * @throws NullPointerException if any of the parameters is {@code null}. + */ + private void onSubscribe(final MqttEndpoint endpoint, final Device authenticatedDevice, + final MqttSubscribeMessage subscribeMsg, final CommandSubscriptionsManager cmdSubscriptionsManager) { + + Objects.requireNonNull(endpoint); + Objects.requireNonNull(authenticatedDevice); + Objects.requireNonNull(subscribeMsg); + Objects.requireNonNull(cmdSubscriptionsManager); + + @SuppressWarnings("rawtypes") + final List subscriptionOutcome = new ArrayList<>(subscribeMsg.topicSubscriptions().size()); + + subscribeMsg.topicSubscriptions().forEach(subscription -> { + + final Future result; + + if (isTopicFilterValid(subscription.topicName(), authenticatedDevice.getTenantId(), + authenticatedDevice.getDeviceId(), endpoint.clientIdentifier())) { + + // we do not support subscribing to commands using QoS 2 + final MqttQoS grantedQos = MqttQoS.EXACTLY_ONCE.equals(subscription.qualityOfService()) + ? MqttQoS.AT_LEAST_ONCE + : subscription.qualityOfService(); + + final CommandSubscription cmdSub = new CommandSubscription(subscription.topicName(), grantedQos, + endpoint.clientIdentifier()); + + result = cmdSubscriptionsManager.addSubscription(cmdSub, + () -> createCommandConsumer(endpoint, cmdSubscriptionsManager, authenticatedDevice)); + } else { + log.debug("cannot create subscription [filter: {}, requested QoS: {}]: unsupported topic filter", + subscription.topicName(), subscription.qualityOfService()); + result = Future.succeededFuture(MqttQoS.FAILURE); + } + subscriptionOutcome.add(result); + }); + + // wait for all futures to complete before sending SUBACK + CompositeFuture.join(subscriptionOutcome).onComplete(v -> { + + // return a status code for each topic filter contained in the SUBSCRIBE packet + final List grantedQosLevels = subscriptionOutcome.stream() + .map(Future::result) + .map(result -> (MqttQoS) result) + .collect(Collectors.toList()); + + if (endpoint.isConnected()) { + endpoint.subscribeAcknowledge(subscribeMsg.messageId(), grantedQosLevels); + } + }); + } + + /** + * Invoked when a device sends an MQTT UNSUBSCRIBE packet. + * + * @param endpoint The endpoint representing the connection to the device. + * @param authenticatedDevice The authenticated identity of the device. + * @param unsubscribeMsg The unsubscribe request received from the device. + * @param cmdSubscriptionsManager The CommandSubscriptionsManager to track command subscriptions, unsubscriptions + * and handle PUBACKs. + * @throws NullPointerException if any of the parameters is {@code null}. + */ + private void onUnsubscribe(final MqttEndpoint endpoint, final Device authenticatedDevice, + final MqttUnsubscribeMessage unsubscribeMsg, final CommandSubscriptionsManager cmdSubscriptionsManager) { + + Objects.requireNonNull(endpoint); + Objects.requireNonNull(authenticatedDevice); + Objects.requireNonNull(unsubscribeMsg); + Objects.requireNonNull(cmdSubscriptionsManager); + + unsubscribeMsg.topics().forEach(topic -> { + if (!isTopicFilterValid(topic, authenticatedDevice.getTenantId(), authenticatedDevice.getDeviceId(), + endpoint.clientIdentifier())) { + log.debug("ignoring unsubscribe request for unsupported topic filter [{}]", topic); + } else { + log.debug("unsubscribing device [tenant-id: {}, device-id: {}] from topic [{}]", + authenticatedDevice.getTenantId(), authenticatedDevice.getDeviceId(), topic); + cmdSubscriptionsManager.removeSubscription(topic); + } + }); + if (endpoint.isConnected()) { + endpoint.unsubscribeAcknowledge(unsubscribeMsg.messageId()); + } + } + + private Future createCommandConsumer(final MqttEndpoint endpoint, + final CommandSubscriptionsManager cmdSubscriptionsManager, final Device authenticatedDevice) { + return clientFactoryPerTenant.get(authenticatedDevice.getTenantId()).createDeviceSpecificCommandConsumer( + authenticatedDevice.getDeviceId(), + cmd -> handleCommand(endpoint, cmd, cmdSubscriptionsManager, authenticatedDevice)); + } + + private void handleCommand(final MqttEndpoint endpoint, final Message message, + final CommandSubscriptionsManager cmdSubscriptionsManager, final Device authenticatedDevice) { + + if (message.getReplyTo() != null) { + log.debug("Received request/response command [subject: {}, correlationID: {}, messageID: {}, reply-to: {}]", + message.getSubject(), message.getCorrelationId(), message.getMessageId(), message.getReplyTo()); + } else { + log.debug("Received one-way command [subject: {}]", message.getSubject()); + } + + final MqttCommandContext ctx = MqttCommandContext.fromAmqpMessage(message, authenticatedDevice); + final Command command = onCommandReceived(ctx); + + if (command == null) { + throw new IllegalStateException("onCommandReceived returned null"); + } + + final CommandSubscription subscription = cmdSubscriptionsManager.getSubscriptions() + .get(command.getTopicFilter()); + if (subscription == null) { + throw new IllegalStateException( + String.format("No subscription found for topic filter %s. Discarding message from %s", + command.getTopicFilter(), authenticatedDevice.toString())); + } + + log.debug("Publishing command on topic [{}] to device {} [MQTT client-id: {}, QoS: {}]", command.getTopic(), + authenticatedDevice.toString(), endpoint.clientIdentifier(), subscription.getQos()); + + endpoint.publish(command.getTopic(), command.getPayload(), subscription.getQos(), false, false, + ar -> afterCommandPublished(ar.result(), message, authenticatedDevice, subscription, + cmdSubscriptionsManager)); + + } + + // Vert.x only calls this handler after it successfully published the message, otherwise it throws an exception + // which causes the AMQP Command Consumer not to be settled (and the backend application to receive an error) + private void afterCommandPublished(final Integer publishedMsgId, final Message message, + final Device authenticatedDevice, final CommandSubscription subscription, + final CommandSubscriptionsManager cmdSubscriptionsManager) { + + if (MqttQoS.AT_LEAST_ONCE.equals(subscription.getQos())) { + + final Handler onAckHandler = msgId -> { + + onCommandPublished(message, subscription); + + log.debug( + "Acknowledged [Msg-id: {}] command to device [tenant-id: {}, device-id: {}, MQTT client-id: {}, QoS: {}]", + msgId, authenticatedDevice.getTenantId(), authenticatedDevice.getDeviceId(), + subscription.getClientId(), subscription.getQos()); + }; + + final Handler onAckTimeoutHandler = v -> log.debug( + "Timed out waiting for acknowledgment for command sent to device [tenant-id: {}, device-id: {}, MQTT client-id: {}, QoS: {}]", + authenticatedDevice.getTenantId(), authenticatedDevice.getDeviceId(), + subscription.getClientId(), subscription.getQos()); + + cmdSubscriptionsManager.addToWaitingForAcknowledgement(publishedMsgId, onAckHandler, onAckTimeoutHandler); + } else { + onCommandPublished(message, subscription); + } + } + + /** + * {@inheritDoc} + *

+ * Creates and starts the MQTT server and invokes {@link #afterStartup(Promise)} afterwards. + */ + @Override + public final void start(final Promise startPromise) { + + if (mqttGatewayConfig.getKeyCertOptions() == null + && mqttGatewayConfig.getPort() == MqttServerOptions.DEFAULT_TLS_PORT) { + log.error("configuration must have key & certificate if port 8883 is configured"); + startPromise.fail("TLS configuration invalid"); + } + + MqttServer.create(vertx, getMqttServerOptions()) + .endpointHandler(this::handleEndpointConnection) + .listen(asyncResult -> { + if (asyncResult.succeeded()) { + final MqttServer startedServer = asyncResult.result(); + log.info("MQTT server running on {}:{}", mqttGatewayConfig.getBindAddress(), + startedServer.actualPort()); + server = startedServer; + afterStartup(startPromise); + } else { + log.error("error while starting up MQTT server", asyncResult.cause()); + startPromise.fail(asyncResult.cause()); + } + }); + } + + /** + * Returns the options for the MQTT server. + *

+ * This method is only visible for testing purposes. + * + * @return The options configured with the values of the {@link MqttProtocolGatewayConfig}. + */ + MqttServerOptions getMqttServerOptions() { + final MqttServerOptions options = new MqttServerOptions() + .setHost(mqttGatewayConfig.getBindAddress()) + .setPort(mqttGatewayConfig.getPort()); + + addTlsKeyCertOptions(options); + addTlsTrustOptions(options); + return options; + } + + private void addTlsKeyCertOptions(final NetServerOptions serverOptions) { + + final KeyCertOptions keyCertOptions = mqttGatewayConfig.getKeyCertOptions(); + + if (keyCertOptions != null) { + serverOptions.setSsl(true).setKeyCertOptions(keyCertOptions); + log.info("Enabling TLS"); + + final LinkedHashSet enabledProtocols = new LinkedHashSet<>(mqttGatewayConfig.getSecureProtocols()); + serverOptions.setEnabledSecureTransportProtocols(enabledProtocols); + log.info("Enabling secure protocols [{}]", enabledProtocols); + + serverOptions.setSni(mqttGatewayConfig.isSni()); + log.info("Supporting TLS ServerNameIndication: {}", mqttGatewayConfig.isSni()); + } + } + + private void addTlsTrustOptions(final NetServerOptions serverOptions) { + + if (serverOptions.isSsl()) { + + final TrustOptions trustOptions = mqttGatewayConfig.getTrustOptions(); + if (trustOptions != null) { + serverOptions.setTrustOptions(trustOptions).setClientAuth(ClientAuth.REQUEST); + log.info("Enabling client authentication using certificates [{}]", trustOptions.getClass().getName()); + } + } + } + + /** + * {@inheritDoc} + *

+ * Invokes {@link #beforeShutdown(Promise)} and stops the MQTT server. + */ + @Override + public final void stop(final Promise stopPromise) { + + final Promise stopTracker = Promise.promise(); + beforeShutdown(stopTracker); + stopTracker.future().onComplete(v -> { + if (server != null) { + server.close(stopPromise); + } else { + stopPromise.complete(); + } + }); + + } + + /** + * Invoked directly before the gateway is shut down. + *

+ * This default implementation always completes the promise. + *

+ * Subclasses should override this method to perform any work required before shutting down this protocol gateway. + * + * @param stopPromise The promise to complete once all work is done and shut down should commence. + */ + protected void beforeShutdown(final Promise stopPromise) { + stopPromise.complete(); + } + + /** + * Invoked after the gateway has started up. + *

+ * This default implementation simply completes the promise. + *

+ * Subclasses should override this method to perform any work required on start-up of this protocol gateway. + * + * @param startPromise The promise to complete once start up is complete. + */ + protected void afterStartup(final Promise startPromise) { + startPromise.complete(); + } + +} diff --git a/protocol-gateway/mqtt-protocol-gateway-template/src/main/java/org/eclipse/hono/gateway/sdk/mqtt2amqp/Command.java b/protocol-gateway/mqtt-protocol-gateway-template/src/main/java/org/eclipse/hono/gateway/sdk/mqtt2amqp/Command.java new file mode 100644 index 00000000..998a6ae5 --- /dev/null +++ b/protocol-gateway/mqtt-protocol-gateway-template/src/main/java/org/eclipse/hono/gateway/sdk/mqtt2amqp/Command.java @@ -0,0 +1,77 @@ +/******************************************************************************* + * Copyright (c) 2020 Contributors to the Eclipse Foundation + * + * See the NOTICE file(s) distributed with this work for additional + * information regarding copyright ownership. + * + * This program and the accompanying materials are made available under the + * terms of the Eclipse Public License 2.0 which is available at + * http://www.eclipse.org/legal/epl-2.0 + * + * SPDX-License-Identifier: EPL-2.0 + *******************************************************************************/ + +package org.eclipse.hono.gateway.sdk.mqtt2amqp; + +import java.util.Objects; + +import io.vertx.core.buffer.Buffer; + +/** + * The command message that will be published via MQTT to a device. + *

+ * Devices can use wildcards in the topic filter, so there is no easy way to uniquely identify the corresponding + * subscription and thus the required Qos. To ensure this, the topic filter must match exactly the one the device has + * subscribed to. + */ +public class Command { + + private final String topic; + private final String topicFilter; + private final Buffer payload; + + /** + * Creates an instance. + * + * @param topic The topic on which the command should be sent to the device. + * @param topicFilter The topic filter to which the device has subscribed. + * @param payload The payload of the command. + */ + public Command(final String topic, final String topicFilter, final Buffer payload) { + Objects.requireNonNull(topic); + Objects.requireNonNull(topicFilter); + Objects.requireNonNull(payload); + + this.topic = topic; + this.topicFilter = topicFilter; + this.payload = payload; + } + + /** + * Gets the topic on which the command should be sent to the device. + * + * @return The topic. + */ + public String getTopic() { + return topic; + } + + /** + * Returns the topic filter to which the device has subscribed. + * + * @return The topic filter. + */ + public String getTopicFilter() { + return topicFilter; + } + + /** + * Returns the payload to be published to the device. + * + * @return The payload. + */ + public Buffer getPayload() { + return payload; + } + +} diff --git a/protocol-gateway/mqtt-protocol-gateway-template/src/main/java/org/eclipse/hono/gateway/sdk/mqtt2amqp/CommandSubscription.java b/protocol-gateway/mqtt-protocol-gateway-template/src/main/java/org/eclipse/hono/gateway/sdk/mqtt2amqp/CommandSubscription.java new file mode 100644 index 00000000..0f74b56b --- /dev/null +++ b/protocol-gateway/mqtt-protocol-gateway-template/src/main/java/org/eclipse/hono/gateway/sdk/mqtt2amqp/CommandSubscription.java @@ -0,0 +1,95 @@ +/** + * Copyright (c) 2018, 2020 Contributors to the Eclipse Foundation + * + * See the NOTICE file(s) distributed with this work for additional + * information regarding copyright ownership. + * + * This program and the accompanying materials are made available under the + * terms of the Eclipse Public License 2.0 which is available at + * https://www.eclipse.org/legal/epl-2.0 + * + * SPDX-License-Identifier: EPL-2.0 + */ + +package org.eclipse.hono.gateway.sdk.mqtt2amqp; + +import java.util.Objects; + +import io.netty.handler.codec.mqtt.MqttQoS; + +/** + * The MQTT subscription of devices, to get commands. + * + */ +public class CommandSubscription { + + private final String topicFilter; + private final MqttQoS qos; + private final String clientId; + + /** + * Creates a command subscription object for the given topic filter. + * + * @param topicFilter The topic filter in the subscription request from device. + * @param qos The MQTT QoS of the subscription. + * @param clientId The client identifier as provided by the remote MQTT client. + * @throws NullPointerException if one of the arguments is {@code null}. + * @throws IllegalArgumentException if the topic filter does not match the rules or any of the arguments is not + * valid. + **/ + public CommandSubscription(final String topicFilter, final MqttQoS qos, final String clientId) { + Objects.requireNonNull(topicFilter); + Objects.requireNonNull(qos); + Objects.requireNonNull(clientId); + + this.topicFilter = topicFilter; + this.qos = qos; + this.clientId = clientId; + } + + /** + * Gets the QoS of the subscription. + * + * @return The QoS value. + */ + public MqttQoS getQos() { + return qos; + } + + /** + * Gets the clientId of the Mqtt subscription. + * + * @return The clientId. + */ + public String getClientId() { + return clientId; + } + + /** + * Gets the subscription topic filter. + * + * @return The topic filter. + */ + public String getTopicFilter() { + return topicFilter; + } + + @Override + public boolean equals(final Object o) { + if (this == o) { + return true; + } + if (o == null || getClass() != o.getClass()) { + return false; + } + final CommandSubscription that = (CommandSubscription) o; + return topicFilter.equals(that.topicFilter) && + qos == that.qos && + clientId.equals(that.clientId); + } + + @Override + public int hashCode() { + return Objects.hash(topicFilter, qos, clientId); + } +} diff --git a/protocol-gateway/mqtt-protocol-gateway-template/src/main/java/org/eclipse/hono/gateway/sdk/mqtt2amqp/CommandSubscriptionsManager.java b/protocol-gateway/mqtt-protocol-gateway-template/src/main/java/org/eclipse/hono/gateway/sdk/mqtt2amqp/CommandSubscriptionsManager.java new file mode 100644 index 00000000..255a8b04 --- /dev/null +++ b/protocol-gateway/mqtt-protocol-gateway-template/src/main/java/org/eclipse/hono/gateway/sdk/mqtt2amqp/CommandSubscriptionsManager.java @@ -0,0 +1,230 @@ +/******************************************************************************* + * Copyright (c) 2016, 2020 Contributors to the Eclipse Foundation + * + * See the NOTICE file(s) distributed with this work for additional + * information regarding copyright ownership. + * + * This program and the accompanying materials are made available under the + * terms of the Eclipse Public License 2.0 which is available at + * http://www.eclipse.org/legal/epl-2.0 + * + * SPDX-License-Identifier: EPL-2.0 + *******************************************************************************/ + +package org.eclipse.hono.gateway.sdk.mqtt2amqp; + +import java.util.Collections; +import java.util.Map; +import java.util.Objects; +import java.util.Optional; +import java.util.concurrent.ConcurrentHashMap; +import java.util.function.Supplier; + +import org.eclipse.hono.client.MessageConsumer; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import io.netty.handler.codec.mqtt.MqttQoS; +import io.vertx.core.Future; +import io.vertx.core.Handler; +import io.vertx.core.Vertx; + +/** + * A class that tracks command subscriptions, unsubscriptions and handles PUBACKs. + */ +final class CommandSubscriptionsManager { + + private static final Logger LOG = LoggerFactory.getLogger(CommandSubscriptionsManager.class); + /** + * Map of the current subscriptions. Key is the topic name. + */ + private final Map subscriptions = new ConcurrentHashMap<>(); + /** + * Map of the requests waiting for an acknowledgement. Key is the command message id. + */ + private final Map waitingForAcknowledgement = new ConcurrentHashMap<>(); + private final Vertx vertx; + private final MqttProtocolGatewayConfig config; + private Future commandConsumer; + + /** + * Creates a new CommandSubscriptionsManager instance. + * + * @param vertx The Vert.x instance to execute the client on. + * @param config The configuration properties to use. + * @throws NullPointerException if any of the parameters are {@code null}. + */ + CommandSubscriptionsManager(final Vertx vertx, final MqttProtocolGatewayConfig config) { + this.vertx = Objects.requireNonNull(vertx); + this.config = Objects.requireNonNull(config); + } + + /** + * Invoked when a device sends an MQTT PUBACK packet. + * + * @param msgId The msgId of the command published with QoS 1. + * @throws NullPointerException if msgId is {@code null}. + */ + public void handlePubAck(final Integer msgId) { + Objects.requireNonNull(msgId); + LOG.trace("Acknowledgement received for command [Msg-id: {}] that has been sent to device.", msgId); + Optional.ofNullable(removeFromWaitingForAcknowledgement(msgId)).ifPresent(value -> { + cancelTimer(value.timerId); + value.onAckHandler.handle(msgId); + }); + } + + /** + * Registers handlers to be invoked when the command message with the given id is either acknowledged or a timeout + * occurs. + * + * @param msgId The id of the command (message) that has been published. + * @param onAckHandler Handler to invoke when the device has acknowledged the command. + * @param onAckTimeoutHandler Handler to invoke when there is a timeout waiting for the acknowledgement from the + * device. + * @throws NullPointerException if any of the parameters is {@code null}. + */ + public void addToWaitingForAcknowledgement(final Integer msgId, final Handler onAckHandler, + final Handler onAckTimeoutHandler) { + + Objects.requireNonNull(msgId); + Objects.requireNonNull(onAckHandler); + Objects.requireNonNull(onAckTimeoutHandler); + + waitingForAcknowledgement.put(msgId, + new PendingCommandRequest(startTimer(msgId), onAckHandler, onAckTimeoutHandler)); + } + + /** + * Removes the entry from the waitingForAcknowledgement map for the given msgId. + * + * @param msgId The id of the command (message) that has been published. + * @return The PendingCommandRequest object containing timer-id and event handlers. + */ + private PendingCommandRequest removeFromWaitingForAcknowledgement(final Integer msgId) { + return waitingForAcknowledgement.remove(msgId); + } + + /** + * Stores the command subscription and creates a command consumer for it. Multiple subscription share the same + * consumer. + * + * @param subscription The device's command subscription. + * @param commandConsumerSupplier A function to create a client for consuming messages. + * @throws NullPointerException if any of the parameters are {@code null}. + * @return The QoS of the subscription or {@link MqttQoS#FAILURE} if the consumer could not be opened. + */ + public Future addSubscription(final CommandSubscription subscription, + final Supplier> commandConsumerSupplier) { + + Objects.requireNonNull(subscription); + Objects.requireNonNull(commandConsumerSupplier); + + LOG.trace("Adding subscription for topic filter [{}]", subscription.getTopicFilter()); + + if (commandConsumer == null) { + commandConsumer = commandConsumerSupplier.get(); + } + + return commandConsumer + .map(messageConsumer -> { + subscriptions.put(subscription.getTopicFilter(), subscription); + LOG.debug("Added subscription for topic filter [{}] with QoS {}", subscription.getTopicFilter(), + subscription.getQos()); + return subscription.getQos(); + }) + .otherwise(MqttQoS.FAILURE); + } + + /** + * Removes the subscription entry for the given topic filter. If this was the only subscription, the command + * consumer will be closed. + * + * @param topicFilter The topic filter string to unsubscribe. + * @throws NullPointerException if the topic filter is {@code null}. + */ + public void removeSubscription(final String topicFilter) { + Objects.requireNonNull(topicFilter); + + final CommandSubscription value = subscriptions.remove(topicFilter); + if (value != null) { + LOG.debug("Remove subscription for topic filter [{}]", topicFilter); + + if (subscriptions.isEmpty() && commandConsumer != null && commandConsumer.succeeded()) { + closeCommandConsumer(commandConsumer.result()); + } + } else { + LOG.debug("Cannot remove subscription; none registered for topic filter [{}].", topicFilter); + } + } + + /** + * Removes all the subscription entries and closes the command consumer. + * + **/ + public void removeAllSubscriptions() { + subscriptions.keySet().forEach(this::removeSubscription); + } + + private void closeCommandConsumer(final MessageConsumer consumer) { + consumer.close(cls -> { + if (cls.succeeded()) { + LOG.debug("Command consumer closed"); + commandConsumer = null; + } else { + LOG.error("Error closing command consumer", cls.cause()); + } + }); + } + + private long startTimer(final Integer msgId) { + + return vertx.setTimer(config.getCommandAckTimeout(), timerId -> { + Optional.ofNullable(removeFromWaitingForAcknowledgement(msgId)) + .ifPresent(value -> value.onAckTimeoutHandler.handle(null)); + }); + } + + private void cancelTimer(final Long timerId) { + vertx.cancelTimer(timerId); + LOG.trace("Canceled Timer [timer-id: {}}", timerId); + } + + /** + * Returns all subscriptions of this device. + * + * @return An unmodifiable view of the subscriptions map with the topic filter as key and the subscription object as + * value. + */ + public Map getSubscriptions() { + return Collections.unmodifiableMap(subscriptions); + } + + /** + * A class to facilitate storing of information in connection with the pending command requests. The pending command + * requests are tracked using a map in the enclosing class {@link CommandSubscriptionsManager}. + */ + private static class PendingCommandRequest { + + private final Long timerId; + private final Handler onAckHandler; + private final Handler onAckTimeoutHandler; + + /** + * Creates a new PendingCommandRequest instance. + * + * @param timerId The unique ID of the timer. + * @param onAckHandler Handler to invoke when the device has acknowledged the command. + * @param onAckTimeoutHandler Handler to invoke when there is a timeout waiting for the acknowledgement from the + * device. + * @throws NullPointerException if any of the parameters is {@code null}. + */ + private PendingCommandRequest(final Long timerId, final Handler onAckHandler, + final Handler onAckTimeoutHandler) { + this.timerId = Objects.requireNonNull(timerId); + this.onAckHandler = Objects.requireNonNull(onAckHandler); + this.onAckTimeoutHandler = Objects.requireNonNull(onAckTimeoutHandler); + } + + } +} diff --git a/protocol-gateway/mqtt-protocol-gateway-template/src/main/java/org/eclipse/hono/gateway/sdk/mqtt2amqp/Credentials.java b/protocol-gateway/mqtt-protocol-gateway-template/src/main/java/org/eclipse/hono/gateway/sdk/mqtt2amqp/Credentials.java new file mode 100644 index 00000000..861de081 --- /dev/null +++ b/protocol-gateway/mqtt-protocol-gateway-template/src/main/java/org/eclipse/hono/gateway/sdk/mqtt2amqp/Credentials.java @@ -0,0 +1,58 @@ +/******************************************************************************* + * Copyright (c) 2020 Contributors to the Eclipse Foundation + * + * See the NOTICE file(s) distributed with this work for additional + * information regarding copyright ownership. + * + * This program and the accompanying materials are made available under the + * terms of the Eclipse Public License 2.0 which is available at + * http://www.eclipse.org/legal/epl-2.0 + * + * SPDX-License-Identifier: EPL-2.0 + *******************************************************************************/ + +package org.eclipse.hono.gateway.sdk.mqtt2amqp; + +import java.util.Objects; + +/** + * Simple data structure that wraps a username an a password. + */ +public class Credentials { + + private final String username; + private final String password; + + /** + * Creates an instance. + * + * @param username The user name. + * @param password The password. + * @throws NullPointerException if any of the parameters is {@code null}. + */ + public Credentials(final String username, final String password) { + Objects.requireNonNull(username); + Objects.requireNonNull(password); + + this.username = username; + this.password = password; + } + + /** + * Returns the username. + * + * @return The username - not {@code null}. + */ + public String getUsername() { + return username; + } + + /** + * Returns the password. + * + * @return The password - not {@code null}. + */ + public String getPassword() { + return password; + } +} diff --git a/protocol-gateway/mqtt-protocol-gateway-template/src/main/java/org/eclipse/hono/gateway/sdk/mqtt2amqp/MqttCommandContext.java b/protocol-gateway/mqtt-protocol-gateway-template/src/main/java/org/eclipse/hono/gateway/sdk/mqtt2amqp/MqttCommandContext.java new file mode 100644 index 00000000..b42a71d6 --- /dev/null +++ b/protocol-gateway/mqtt-protocol-gateway-template/src/main/java/org/eclipse/hono/gateway/sdk/mqtt2amqp/MqttCommandContext.java @@ -0,0 +1,138 @@ +/******************************************************************************* + * Copyright (c) 2020 Contributors to the Eclipse Foundation + * + * See the NOTICE file(s) distributed with this work for additional + * information regarding copyright ownership. + * + * This program and the accompanying materials are made available under the + * terms of the Eclipse Public License 2.0 which is available at + * http://www.eclipse.org/legal/epl-2.0 + * + * SPDX-License-Identifier: EPL-2.0 + *******************************************************************************/ + +package org.eclipse.hono.gateway.sdk.mqtt2amqp; + +import java.util.Objects; + +import org.apache.qpid.proton.amqp.messaging.ApplicationProperties; +import org.apache.qpid.proton.message.Message; +import org.eclipse.hono.auth.Device; +import org.eclipse.hono.util.MessageHelper; + +import io.vertx.core.buffer.Buffer; + +/** + * A dictionary of data for the processing of a command message received from Hono's AMQP adapter. + */ +public class MqttCommandContext { + + private final Message message; + private final Device authenticatedDevice; + + private MqttCommandContext(final Message message, final Device authenticatedDevice) { + this.message = message; + this.authenticatedDevice = authenticatedDevice; + } + + /** + * Creates a new context for a command message. + * + * @param message The received command message. + * @param authenticatedDevice The authenticated device identity. + * @return The context. + * @throws NullPointerException if any of the parameters is {@code null}. + */ + public static MqttCommandContext fromAmqpMessage(final Message message, final Device authenticatedDevice) { + Objects.requireNonNull(message); + Objects.requireNonNull(authenticatedDevice); + + return new MqttCommandContext(message, authenticatedDevice); + } + + /** + * Gets the identity of the device to which the command is addressed to. + * + * @return The authenticated device. + */ + public Device getDevice() { + return authenticatedDevice; + } + + /** + * Indicates if the message represents a request/response or an one-way command. + * + * @return {@code true} if a response is expected. + */ + public boolean isRequestResponseCommand() { + return message.getReplyTo() != null; + } + + /** + * Returns the subject of the command message. + * + * @return The subject. + */ + public String getSubject() { + return message.getSubject(); + } + + /** + * Returns the content type of the payload if this information is available from the message. + * + * @return The content type or {@code null}. + */ + public String getContentType() { + return message.getContentType(); + } + + /** + * Returns the correlation id of the message. + * + * @return The correlation id. + */ + public Object getCorrelationId() { + return message.getCorrelationId(); + } + + /** + * Returns the message id of the message. + * + * @return The message id. + */ + public Object getMessageId() { + return message.getMessageId(); + } + + /** + * Returns the reply-to address of the message. + * + * @return The reply-to address. + */ + public String getReplyTo() { + return message.getReplyTo(); + } + + /** + * Returns the application properties of the message. + * + * @return The application properties. + */ + public ApplicationProperties getApplicationProperties() { + return message.getApplicationProperties(); + } + + /** + * Returns the received payload. + * + * @return The payload - not {@code null}. + */ + public Buffer getPayload() { + final Buffer payload = MessageHelper.getPayload(message); + if (payload != null) { + return payload; + } else { + return Buffer.buffer(); + } + } +} diff --git a/protocol-gateway/mqtt-protocol-gateway-template/src/main/java/org/eclipse/hono/gateway/sdk/mqtt2amqp/MqttDownstreamContext.java b/protocol-gateway/mqtt-protocol-gateway-template/src/main/java/org/eclipse/hono/gateway/sdk/mqtt2amqp/MqttDownstreamContext.java new file mode 100644 index 00000000..d2dfd9ce --- /dev/null +++ b/protocol-gateway/mqtt-protocol-gateway-template/src/main/java/org/eclipse/hono/gateway/sdk/mqtt2amqp/MqttDownstreamContext.java @@ -0,0 +1,119 @@ +/******************************************************************************* + * Copyright (c) 2016, 2020 Contributors to the Eclipse Foundation + * + * See the NOTICE file(s) distributed with this work for additional + * information regarding copyright ownership. + * + * This program and the accompanying materials are made available under the + * terms of the Eclipse Public License 2.0 which is available at + * http://www.eclipse.org/legal/epl-2.0 + * + * SPDX-License-Identifier: EPL-2.0 + *******************************************************************************/ + +package org.eclipse.hono.gateway.sdk.mqtt2amqp; + +import java.util.Objects; + +import org.eclipse.hono.auth.Device; + +import io.netty.handler.codec.mqtt.MqttQoS; +import io.vertx.mqtt.MqttEndpoint; +import io.vertx.mqtt.messages.MqttPublishMessage; + +/** + * A dictionary of relevant information required during the processing of an MQTT message published by a device. + * + */ +public class MqttDownstreamContext { + + private final MqttPublishMessage message; + private final MqttEndpoint deviceEndpoint; + private final Device authenticatedDevice; + private final String topic; + private final MqttQoS qos; + + private MqttDownstreamContext(final Device authenticatedDevice, final MqttPublishMessage publishedMessage, + final MqttEndpoint deviceEndpoint, final String topic) { + this.authenticatedDevice = authenticatedDevice; + this.message = publishedMessage; + this.deviceEndpoint = deviceEndpoint; + this.topic = topic; + this.qos = publishedMessage.qosLevel(); + } + + /** + * Creates a new context for a published message. + * + * @param message The published MQTT message. + * @param deviceEndpoint The endpoint representing the device that has published the message. + * @param authenticatedDevice The authenticated device identity. + * @return The context. + * @throws NullPointerException if any of the parameters are {@code null}. + */ + public static MqttDownstreamContext fromPublishPacket( + final MqttPublishMessage message, + final MqttEndpoint deviceEndpoint, + final Device authenticatedDevice) { + + Objects.requireNonNull(message); + Objects.requireNonNull(deviceEndpoint); + Objects.requireNonNull(authenticatedDevice); + + return new MqttDownstreamContext(authenticatedDevice, message, deviceEndpoint, message.topicName()); + } + + /** + * Gets the MQTT message to process. + * + * @return The message. + */ + public MqttPublishMessage message() { + return message; + } + + /** + * Gets the MQTT endpoint over which the message has been received. + * + * @return The endpoint. + */ + MqttEndpoint deviceEndpoint() { + return deviceEndpoint; + } + + /** + * Gets the identity of the authenticated device that has published the message. + * + * @return The identity or {@code null} if the device has not been authenticated. + */ + public Device authenticatedDevice() { + return authenticatedDevice; + } + + /** + * Gets the topic that the message has been published to. + * + * @return The topic. + */ + public String topic() { + return topic; + } + + /** + * Gets the QoS level of the published MQTT message. + * + * @return The QoS. + */ + public MqttQoS qosLevel() { + return qos; + } + + /** + * Sends a PUBACK for the message to the device. + */ + public void acknowledge() { + if (message != null && deviceEndpoint != null) { + deviceEndpoint.publishAcknowledge(message.messageId()); + } + } +} diff --git a/protocol-gateway/mqtt-protocol-gateway-template/src/main/java/org/eclipse/hono/gateway/sdk/mqtt2amqp/MqttProtocolGatewayConfig.java b/protocol-gateway/mqtt-protocol-gateway-template/src/main/java/org/eclipse/hono/gateway/sdk/mqtt2amqp/MqttProtocolGatewayConfig.java new file mode 100644 index 00000000..159fcc92 --- /dev/null +++ b/protocol-gateway/mqtt-protocol-gateway-template/src/main/java/org/eclipse/hono/gateway/sdk/mqtt2amqp/MqttProtocolGatewayConfig.java @@ -0,0 +1,134 @@ +/******************************************************************************* + * Copyright (c) 2020 Contributors to the Eclipse Foundation + * + * See the NOTICE file(s) distributed with this work for additional + * information regarding copyright ownership. + * + * This program and the accompanying materials are made available under the + * terms of the Eclipse Public License 2.0 which is available at + * http://www.eclipse.org/legal/epl-2.0 + * + * SPDX-License-Identifier: EPL-2.0 + *******************************************************************************/ + +package org.eclipse.hono.gateway.sdk.mqtt2amqp; + +import java.util.Objects; + +import org.eclipse.hono.config.AbstractConfig; +import org.eclipse.hono.util.Constants; + +/** + * Configuration of server properties for {@link AbstractMqttProtocolGateway}. + */ +public class MqttProtocolGatewayConfig extends AbstractConfig { + + /** + * The default number of milliseconds to wait for PUBACK. + */ + protected static final int DEFAULT_COMMAND_ACK_TIMEOUT = 100; + + private int commandAckTimeout = DEFAULT_COMMAND_ACK_TIMEOUT; + private int port = 0; + private String bindAddress = Constants.LOOPBACK_DEVICE_ADDRESS; + private boolean sni; + + /** + * Gets the host name or literal IP address of the network interface that this server's secure port is configured to + * be bound to. + * + * @return The host name. + */ + public final String getBindAddress() { + return bindAddress; + } + + /** + * Sets the host name or literal IP address of the network interface that this server's secure port should be bound + * to. + *

+ * The default value of this property is {@link Constants#LOOPBACK_DEVICE_ADDRESS} on IPv4 stacks. + * + * @param address The host name or IP address. + * @throws NullPointerException if host is {@code null}. + */ + public final void setBindAddress(final String address) { + this.bindAddress = Objects.requireNonNull(address); + } + + /** + * Gets the port this server is configured to listen on. + * + * @return The port number. + */ + public final int getPort() { + return port; + } + + /** + * Sets the port that this server should listen on. + *

+ * If the port is set to 0 (the default value), then this server will bind to an arbitrary free port chosen by the + * operating system during startup. + * + * @param port The port number. + * @throws IllegalArgumentException if port < 0 or port > 65535. + */ + public final void setPort(final int port) { + if (isValidPort(port)) { + this.port = port; + } else { + throw new IllegalArgumentException("invalid port number"); + } + } + + /** + * Sets whether the server should support Server Name Indication for TLS connections. + * + * @param sni {@code true} if the server should support SNI. + */ + public final void setSni(final boolean sni) { + this.sni = sni; + } + + /** + * Checks if the server supports Server Name Indication for TLS connections. + * + * @return {@code true} if the server supports SNI. + */ + public final boolean isSni() { + return this.sni; + } + + /** + * Gets the waiting for acknowledgement time out in milliseconds for commands published with QoS 1. + *

+ * This time out is used by the MQTT protocol gateway for commands published with QoS 1. If there is no + * acknowledgement within this time limit, then the command is settled with the released outcome. + *

+ * The default value is {@link #DEFAULT_COMMAND_ACK_TIMEOUT}. + * + * @return The time out in milliseconds. + */ + public final int getCommandAckTimeout() { + return commandAckTimeout; + } + + /** + * Sets the waiting for acknowledgement time out in milliseconds for commands published with QoS 1. + *

+ * This time out is used by the MQTT protocol gateway for commands published with QoS 1. If there is no + * acknowledgement within this time limit, then the command is settled with the released outcome. + *

+ * The default value is {@link #DEFAULT_COMMAND_ACK_TIMEOUT}. + * + * @param timeout The time out in milliseconds. + * @throws IllegalArgumentException if the timeout is negative. + */ + public final void setCommandAckTimeout(final int timeout) { + if (timeout < 0) { + throw new IllegalArgumentException("timeout must not be negative"); + } + this.commandAckTimeout = timeout; + } +} diff --git a/protocol-gateway/mqtt-protocol-gateway-template/src/main/java/org/eclipse/hono/gateway/sdk/mqtt2amqp/X509CertificateValidator.java b/protocol-gateway/mqtt-protocol-gateway-template/src/main/java/org/eclipse/hono/gateway/sdk/mqtt2amqp/X509CertificateValidator.java new file mode 100644 index 00000000..c4d87e55 --- /dev/null +++ b/protocol-gateway/mqtt-protocol-gateway-template/src/main/java/org/eclipse/hono/gateway/sdk/mqtt2amqp/X509CertificateValidator.java @@ -0,0 +1,86 @@ +/******************************************************************************* + * Copyright (c) 2016, 2020 Contributors to the Eclipse Foundation + * + * See the NOTICE file(s) distributed with this work for additional + * information regarding copyright ownership. + * + * This program and the accompanying materials are made available under the + * terms of the Eclipse Public License 2.0 which is available at + * http://www.eclipse.org/legal/epl-2.0 + * + * SPDX-License-Identifier: EPL-2.0 + *******************************************************************************/ + +package org.eclipse.hono.gateway.sdk.mqtt2amqp; + +import java.security.GeneralSecurityException; +import java.security.cert.CertPath; +import java.security.cert.CertPathValidator; +import java.security.cert.CertificateException; +import java.security.cert.CertificateFactory; +import java.security.cert.PKIXParameters; +import java.security.cert.TrustAnchor; +import java.security.cert.X509Certificate; +import java.util.List; +import java.util.Objects; +import java.util.Set; + +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import io.vertx.core.Future; +import io.vertx.core.Promise; + +/** + * Validates a device's certificate chain using a {@link CertPathValidator}. + * + */ +public class X509CertificateValidator { + + private static final Logger LOG = LoggerFactory.getLogger(X509CertificateValidator.class); + + /** + * Validates a certificate path based on a list of trust anchors. + * + * @param chain The certificate chain to validate. The end certificate must be at position 0. + * @param trustAnchors The list of trust anchors to use for validating the chain. + * @return A completed future if the path is valid (according to the implemented tests). Otherwise, the future will + * be failed with a {@link CertificateException}. + * @throws NullPointerException if any of the parameters are {@code null}. + * @throws IllegalArgumentException if the chain or trust anchor list are empty. + */ + public Future validate(final List chain, final Set trustAnchors) { + + Objects.requireNonNull(chain); + Objects.requireNonNull(trustAnchors); + + if (chain.isEmpty()) { + throw new IllegalArgumentException("certificate chain must not be empty"); + } else if (trustAnchors.isEmpty()) { + throw new IllegalArgumentException("trust anchor list must not be empty"); + } + + final Promise result = Promise.promise(); + + try { + final PKIXParameters params = new PKIXParameters(trustAnchors); + params.setRevocationEnabled(false); + final CertificateFactory factory = CertificateFactory.getInstance("X.509"); + final CertPath path = factory.generateCertPath(chain); + final CertPathValidator validator = CertPathValidator.getInstance("PKIX"); + validator.validate(path, params); + LOG.debug("validation of device certificate [subject DN: {}] succeeded", + chain.get(0).getSubjectX500Principal().getName()); + result.complete(); + } catch (GeneralSecurityException e) { + LOG.debug("validation of device certificate [subject DN: {}] failed", + chain.get(0).getSubjectX500Principal().getName(), e); + if (e instanceof CertificateException) { + result.fail(e); + } else { + result.fail(new CertificateException("validation of device certificate failed", e)); + } + } + return result.future(); + } +} diff --git a/protocol-gateway/mqtt-protocol-gateway-template/src/main/java/org/eclipse/hono/gateway/sdk/mqtt2amqp/downstream/CommandResponseMessage.java b/protocol-gateway/mqtt-protocol-gateway-template/src/main/java/org/eclipse/hono/gateway/sdk/mqtt2amqp/downstream/CommandResponseMessage.java new file mode 100644 index 00000000..9378e8d8 --- /dev/null +++ b/protocol-gateway/mqtt-protocol-gateway-template/src/main/java/org/eclipse/hono/gateway/sdk/mqtt2amqp/downstream/CommandResponseMessage.java @@ -0,0 +1,110 @@ +/******************************************************************************* + * Copyright (c) 2020 Contributors to the Eclipse Foundation + * + * See the NOTICE file(s) distributed with this work for additional + * information regarding copyright ownership. + * + * This program and the accompanying materials are made available under the + * terms of the Eclipse Public License 2.0 which is available at + * http://www.eclipse.org/legal/epl-2.0 + * + * SPDX-License-Identifier: EPL-2.0 + *******************************************************************************/ + +package org.eclipse.hono.gateway.sdk.mqtt2amqp.downstream; + +import java.util.Objects; + +import org.eclipse.hono.util.CommandConstants; + +import io.vertx.core.buffer.Buffer; + +/** + * This class holds required data of a command response. + */ +public final class CommandResponseMessage extends DownstreamMessage { + + private final String replyId; + private final String correlationId; + private final int status; + + /** + * Creates an instance. + * + * @param replyId The reply id of the command. + * @param correlationId The correlation id (taken from the command). + * @param status The outcome of the command as a HTTP status code. + * @param payload The payload to be used. + * @throws NullPointerException if any of the parameters is {@code null}. + * @throws IllegalArgumentException if status does not represent a valid HTTP status code. + */ + public CommandResponseMessage(final String replyId, final String correlationId, final String status, + final Buffer payload) { + super(payload); + + Objects.requireNonNull(replyId); + Objects.requireNonNull(correlationId); + Objects.requireNonNull(status); + + this.replyId = replyId; + this.correlationId = correlationId; + + final Integer statusCode = parseStatus(status); + validateStatus(statusCode); + this.status = statusCode; + + } + + /** + * Gets the target address of the response. + * + * @param tenantId The tenant. + * @param deviceId The device that sends the response. + * @return The command reply address as expected by Hono'sCommand & Control API. + */ + public String getTargetAddress(final String tenantId, final String deviceId) { + return String.format("%s/%s/%s/%s", CommandConstants.NORTHBOUND_COMMAND_RESPONSE_ENDPOINT, tenantId, deviceId, + replyId); + } + + /** + * Gets the correlation id of the response. + * + * @return The correlation id. + */ + public String getCorrelationId() { + return correlationId; + } + + /** + * Gets the status code indicating the outcome of the request. + * + * @return The code. + */ + public int getStatus() { + return status; + } + + @Override + public String toString() { + return "command response [replyId=" + replyId + + ", correlationId=" + correlationId + + ", status=" + status + + "]"; + } + + private void validateStatus(final int status) { + if ((status < 200) || (status >= 600)) { + throw new IllegalArgumentException("status is invalid"); + } + } + + private Integer parseStatus(final String status) { + try { + return Integer.parseInt(status); + } catch (NumberFormatException e) { + throw new IllegalArgumentException(e); + } + } + +} diff --git a/protocol-gateway/mqtt-protocol-gateway-template/src/main/java/org/eclipse/hono/gateway/sdk/mqtt2amqp/downstream/DownstreamMessage.java b/protocol-gateway/mqtt-protocol-gateway-template/src/main/java/org/eclipse/hono/gateway/sdk/mqtt2amqp/downstream/DownstreamMessage.java new file mode 100644 index 00000000..b1f02f0b --- /dev/null +++ b/protocol-gateway/mqtt-protocol-gateway-template/src/main/java/org/eclipse/hono/gateway/sdk/mqtt2amqp/downstream/DownstreamMessage.java @@ -0,0 +1,89 @@ +/******************************************************************************* + * Copyright (c) 2020 Contributors to the Eclipse Foundation + * + * See the NOTICE file(s) distributed with this work for additional + * information regarding copyright ownership. + * + * This program and the accompanying materials are made available under the + * terms of the Eclipse Public License 2.0 which is available at + * http://www.eclipse.org/legal/epl-2.0 + * + * SPDX-License-Identifier: EPL-2.0 + *******************************************************************************/ + +package org.eclipse.hono.gateway.sdk.mqtt2amqp.downstream; + +import java.util.HashMap; +import java.util.Map; + +import io.vertx.core.buffer.Buffer; + +/** + * Data to be sent to Hono's AMQP adapter. + */ +public abstract class DownstreamMessage { + + private final Buffer payload; + private Map applicationProperties; + private String contentType; + + /** + * Creates an instance. + * + * @param payload The payload to be used. + * @throws NullPointerException if payload is {@code null}. + */ + public DownstreamMessage(final Buffer payload) { + this.payload = payload; + } + + /** + * Gets the payload of the message as a byte array. + * + * @return The payload. + */ + public byte[] getPayload() { + return (payload == null) ? null : payload.getBytes(); + } + + /** + * Gets the content type of the message payload. + * + * @return The type or {@code null} if the content type is unknown. + */ + public String getContentType() { + return contentType; + } + + /** + * Sets the content type of the message payload. + * + * @param contentType The type or {@code null} if the content type is unknown. + */ + public void setContentType(final String contentType) { + this.contentType = contentType; + } + + /** + * Adds the given property to the AMQP application properties to be added to a message. + * + * @param key The key of the property. + * @param value The value of the property. + */ + public void addApplicationProperty(final String key, final Object value) { + if (applicationProperties == null) { + applicationProperties = new HashMap<>(); + } + applicationProperties.put(key, value); + } + + /** + * Gets the application properties to be added to a message. + * + * @return The application properties. + */ + public Map getApplicationProperties() { + return applicationProperties; + } + +} diff --git a/protocol-gateway/mqtt-protocol-gateway-template/src/main/java/org/eclipse/hono/gateway/sdk/mqtt2amqp/downstream/EventMessage.java b/protocol-gateway/mqtt-protocol-gateway-template/src/main/java/org/eclipse/hono/gateway/sdk/mqtt2amqp/downstream/EventMessage.java new file mode 100644 index 00000000..85a39062 --- /dev/null +++ b/protocol-gateway/mqtt-protocol-gateway-template/src/main/java/org/eclipse/hono/gateway/sdk/mqtt2amqp/downstream/EventMessage.java @@ -0,0 +1,35 @@ +/******************************************************************************* + * Copyright (c) 2020 Contributors to the Eclipse Foundation + * + * See the NOTICE file(s) distributed with this work for additional + * information regarding copyright ownership. + * + * This program and the accompanying materials are made available under the + * terms of the Eclipse Public License 2.0 which is available at + * http://www.eclipse.org/legal/epl-2.0 + * + * SPDX-License-Identifier: EPL-2.0 + *******************************************************************************/ + +package org.eclipse.hono.gateway.sdk.mqtt2amqp.downstream; + +import java.util.Objects; + +import io.vertx.core.buffer.Buffer; + +/** + * This class holds required data of an event message. + */ +public final class EventMessage extends DownstreamMessage { + + /** + * Creates an instance. + * + * @param payload The payload to be used. + * @throws NullPointerException if payload is {@code null}. + */ + public EventMessage(final Buffer payload) { + super(Objects.requireNonNull(payload)); + } + +} diff --git a/protocol-gateway/mqtt-protocol-gateway-template/src/main/java/org/eclipse/hono/gateway/sdk/mqtt2amqp/downstream/TelemetryMessage.java b/protocol-gateway/mqtt-protocol-gateway-template/src/main/java/org/eclipse/hono/gateway/sdk/mqtt2amqp/downstream/TelemetryMessage.java new file mode 100644 index 00000000..d6b71d1c --- /dev/null +++ b/protocol-gateway/mqtt-protocol-gateway-template/src/main/java/org/eclipse/hono/gateway/sdk/mqtt2amqp/downstream/TelemetryMessage.java @@ -0,0 +1,48 @@ +/******************************************************************************* + * Copyright (c) 2020 Contributors to the Eclipse Foundation + * + * See the NOTICE file(s) distributed with this work for additional + * information regarding copyright ownership. + * + * This program and the accompanying materials are made available under the + * terms of the Eclipse Public License 2.0 which is available at + * http://www.eclipse.org/legal/epl-2.0 + * + * SPDX-License-Identifier: EPL-2.0 + *******************************************************************************/ + +package org.eclipse.hono.gateway.sdk.mqtt2amqp.downstream; + +import java.util.Objects; + +import io.vertx.core.buffer.Buffer; + +/** + * This class holds required data of a telemetry message. + */ +public final class TelemetryMessage extends DownstreamMessage { + + private final boolean waitForOutcome; + + /** + * Creates an instance. + * + * @param payload The payload to be used. + * @param waitForOutcome True if the sender should wait for the outcome of the send operation. + * @throws NullPointerException if payload is {@code null}. + */ + public TelemetryMessage(final Buffer payload, final boolean waitForOutcome) { + super(Objects.requireNonNull(payload)); + + this.waitForOutcome = waitForOutcome; + } + + /** + * Returns if the result of the sending should be waited for. + * + * @return {@code true} if the sender should wait for the outcome of the send operation. + */ + public boolean shouldWaitForOutcome() { + return waitForOutcome; + } +} diff --git a/protocol-gateway/mqtt-protocol-gateway-template/src/test/java/org/eclipse/hono/gateway/sdk/mqtt2amqp/AbstractMqttProtocolGatewayTest.java b/protocol-gateway/mqtt-protocol-gateway-template/src/test/java/org/eclipse/hono/gateway/sdk/mqtt2amqp/AbstractMqttProtocolGatewayTest.java new file mode 100644 index 00000000..e5dcca78 --- /dev/null +++ b/protocol-gateway/mqtt-protocol-gateway-template/src/test/java/org/eclipse/hono/gateway/sdk/mqtt2amqp/AbstractMqttProtocolGatewayTest.java @@ -0,0 +1,930 @@ +/** + * Copyright (c) 2020 Contributors to the Eclipse Foundation + * + * See the NOTICE file(s) distributed with this work for additional + * information regarding copyright ownership. + * + * This program and the accompanying materials are made available under the + * terms of the Eclipse Public License 2.0 which is available at + * http://www.eclipse.org/legal/epl-2.0 + * + * SPDX-License-Identifier: EPL-2.0 + */ + +package org.eclipse.hono.gateway.sdk.mqtt2amqp; + +import static org.assertj.core.api.Assertions.assertThat; +import static org.mockito.ArgumentMatchers.any; +import static org.mockito.ArgumentMatchers.anyInt; +import static org.mockito.ArgumentMatchers.anyLong; +import static org.mockito.ArgumentMatchers.anyString; +import static org.mockito.ArgumentMatchers.eq; +import static org.mockito.Mockito.doAnswer; +import static org.mockito.Mockito.mock; +import static org.mockito.Mockito.verify; +import static org.mockito.Mockito.when; + +import java.security.cert.TrustAnchor; +import java.security.cert.X509Certificate; +import java.util.Arrays; +import java.util.Collections; +import java.util.LinkedHashSet; +import java.util.List; +import java.util.Map; +import java.util.Set; +import java.util.concurrent.TimeUnit; +import java.util.function.Consumer; + +import org.apache.qpid.proton.amqp.messaging.Source; +import org.apache.qpid.proton.amqp.transport.Target; +import org.apache.qpid.proton.message.Message; +import org.apache.qpid.proton.message.impl.MessageImpl; +import org.eclipse.hono.client.HonoConnection; +import org.eclipse.hono.client.device.amqp.AmqpAdapterClientFactory; +import org.eclipse.hono.client.device.amqp.CommandResponder; +import org.eclipse.hono.client.device.amqp.EventSender; +import org.eclipse.hono.client.device.amqp.TelemetrySender; +import org.eclipse.hono.client.device.amqp.internal.AmqpAdapterClientCommandConsumer; +import org.eclipse.hono.client.device.amqp.internal.AmqpAdapterClientCommandResponseSender; +import org.eclipse.hono.client.device.amqp.internal.AmqpAdapterClientEventSenderImpl; +import org.eclipse.hono.client.device.amqp.internal.AmqpAdapterClientTelemetrySenderImpl; +import org.eclipse.hono.config.ClientConfigProperties; +import org.eclipse.hono.gateway.sdk.mqtt2amqp.downstream.CommandResponseMessage; +import org.eclipse.hono.gateway.sdk.mqtt2amqp.downstream.DownstreamMessage; +import org.eclipse.hono.gateway.sdk.mqtt2amqp.downstream.TelemetryMessage; +import org.eclipse.hono.util.MessageHelper; +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.extension.ExtendWith; +import org.mockito.ArgumentCaptor; + +import io.netty.handler.codec.mqtt.MqttConnectReturnCode; +import io.netty.handler.codec.mqtt.MqttQoS; +import io.netty.handler.codec.mqtt.MqttTopicSubscription; +import io.opentracing.Tracer; +import io.opentracing.noop.NoopTracerFactory; +import io.vertx.core.AsyncResult; +import io.vertx.core.Future; +import io.vertx.core.Handler; +import io.vertx.core.Promise; +import io.vertx.core.Vertx; +import io.vertx.core.buffer.Buffer; +import io.vertx.core.http.ClientAuth; +import io.vertx.core.json.JsonObject; +import io.vertx.core.net.NetServer; +import io.vertx.core.net.PemTrustOptions; +import io.vertx.core.net.PfxOptions; +import io.vertx.junit5.Timeout; +import io.vertx.junit5.VertxExtension; +import io.vertx.junit5.VertxTestContext; +import io.vertx.mqtt.MqttEndpoint; +import io.vertx.mqtt.MqttServerOptions; +import io.vertx.proton.ProtonQoS; +import io.vertx.proton.ProtonReceiver; +import io.vertx.proton.ProtonSender; + +/** + * Verifies behavior of {@link AbstractMqttProtocolGateway}. + */ +@ExtendWith(VertxExtension.class) +@Timeout(value = 10, timeUnit = TimeUnit.SECONDS) +public class AbstractMqttProtocolGatewayTest { + + private ClientConfigProperties amqpClientConfig; + private Vertx vertx; + private ProtonSender protonSender; + private NetServer netServer; + private AmqpAdapterClientFactory amqpAdapterClientFactory; + private Consumer commandHandler; + + /** + * Sets up common fixture. + */ + @BeforeEach + public void setUp() { + amqpAdapterClientFactory = mock(AmqpAdapterClientFactory.class); + netServer = mock(NetServer.class); + vertx = mock(Vertx.class); + protonSender = mockProtonSender(); + + when(amqpAdapterClientFactory.connect()).thenReturn(Future.succeededFuture()); + + amqpClientConfig = new ClientConfigProperties(); + final HonoConnection connection = mockHonoConnection(vertx, amqpClientConfig, protonSender); + + final Future eventSender = AmqpAdapterClientEventSenderImpl + .createWithAnonymousLinkAddress(connection, TestMqttProtocolGateway.TENANT_ID, s -> { + }); + when(amqpAdapterClientFactory.getOrCreateEventSender()).thenReturn(eventSender); + + final Future telemetrySender = AmqpAdapterClientTelemetrySenderImpl + .createWithAnonymousLinkAddress(connection, TestMqttProtocolGateway.TENANT_ID, s -> { + }); + when(amqpAdapterClientFactory.getOrCreateTelemetrySender()).thenReturn(telemetrySender); + + final Future commandResponseSender = AmqpAdapterClientCommandResponseSender + .createWithAnonymousLinkAddress(connection, TestMqttProtocolGateway.TENANT_ID, s -> { + }); + when(amqpAdapterClientFactory.getOrCreateCommandResponseSender()).thenReturn(commandResponseSender); + + when(amqpAdapterClientFactory.createDeviceSpecificCommandConsumer(anyString(), any())) + .thenAnswer(invocation -> { + final Consumer msgHandler = invocation.getArgument(1); + setCommandHandler(msgHandler); + return AmqpAdapterClientCommandConsumer.create(connection, TestMqttProtocolGateway.TENANT_ID, + TestMqttProtocolGateway.DEVICE_ID, + (protonDelivery, message) -> msgHandler.accept(message)); + }); + + when(vertx.createNetServer(any())).thenReturn(netServer); + when(netServer.listen(anyInt(), anyString(), ProtocolGatewayTestHelper.anyHandler())).then(invocation -> { + final Handler> handler = invocation.getArgument(2); + handler.handle(Future.succeededFuture(netServer)); + return netServer; + }); + + doAnswer(invocation -> { + final Promise handler = invocation.getArgument(0); + handler.complete(); + return null; + }).when(netServer).close(ProtocolGatewayTestHelper.anyHandler()); + + } + + /** + * Verifies that the MqttServerOptions for the MQTT server are taken from the given the server configuration. + */ + @Test + public void testMqttServerConfigWithoutTls() { + final int port = 1111; + final String bindAddress = "127.0.0.127"; + + final MqttProtocolGatewayConfig config = new MqttProtocolGatewayConfig(); + config.setBindAddress(bindAddress); + config.setPort(port); + + // GIVEN a protocol gateway with properties configured + final TestMqttProtocolGateway gateway = createGateway(config); + + // WHEN the server options are created + final MqttServerOptions serverOptions = gateway.getMqttServerOptions(); + + // THEN the server options contain the configured properties... + assertThat(serverOptions.getHost()).isEqualTo(bindAddress); + assertThat(serverOptions.getPort()).isEqualTo(port); + + // ...AND TLS has not been enabled + assertThat(serverOptions.isSsl()).isFalse(); + assertThat(serverOptions.getKeyCertOptions()).isNull(); + assertThat(serverOptions.getTrustOptions()).isNull(); + assertThat(serverOptions.getClientAuth()).isEqualTo(ClientAuth.NONE); + + } + + /** + * Verifies that the MqttServerOptions for the MQTT server are configured correctly for the use of TLS when setting + * the corresponding properties in the server configuration. + */ + @Test + public void testMqttServerConfigWithTls() { + + final String keyStorePath = "src/test/resources/emptyKeyStoreFile.p12"; + final List enabledProtocols = Arrays.asList("TLSv1", "TLSv1.1", "TLSv1.2"); + + // GIVEN a protocol gateway with TLS configured + final MqttProtocolGatewayConfig config = new MqttProtocolGatewayConfig(); + config.setKeyStorePath(keyStorePath); // sets KeyCertOptions + config.setSecureProtocols(enabledProtocols); + config.setSni(true); + + final TestMqttProtocolGateway gateway = createGateway(config); + + // WHEN the server options are created + final MqttServerOptions serverOptions = gateway.getMqttServerOptions(); + + // THEN the TLS configuration is correct + assertThat(serverOptions.isSsl()).isTrue(); + assertThat(serverOptions.getKeyCertOptions()).isEqualTo(new PfxOptions().setPath(keyStorePath)); + + final LinkedHashSet expectedEnabledSecureProtocols = new LinkedHashSet<>(enabledProtocols); + assertThat(serverOptions.getEnabledSecureTransportProtocols()).isEqualTo(expectedEnabledSecureProtocols); + assertThat(serverOptions.isSni()).isTrue(); + + // and not trust options have been set + assertThat(serverOptions.getTrustOptions()).isNull(); + assertThat(serverOptions.getClientAuth()).isEqualTo(ClientAuth.NONE); + } + + /** + * Verifies that the MqttServerOptions for the MQTT server are configured correctly for the use of client + * certificate based authentication when setting the corresponding properties in the server configuration. + */ + @Test + public void testMqttServerConfigWithTlsAndClientAuth() { + + final String keyStorePath = "src/test/resources/emptyKeyStoreFile.p12"; + final String trustStorePath = "src/test/resources/emptyTrustStoreFile.pem"; + final List enabledProtocols = Arrays.asList("TLSv1", "TLSv1.1", "TLSv1.2"); + + // GIVEN a protocol gateway with client certificate based authentication (and TLS) configured + final MqttProtocolGatewayConfig config = new MqttProtocolGatewayConfig(); + config.setKeyStorePath(keyStorePath); // sets KeyCertOptions + config.setTrustStorePath(trustStorePath); // sets TrustOptions + config.setSecureProtocols(enabledProtocols); + config.setSni(true); + + final TestMqttProtocolGateway gateway = createGateway(config); + + // WHEN the server options are created + final MqttServerOptions serverOptions = gateway.getMqttServerOptions(); + + // THEN the trust options are set from the configuration and client certificate based authentication is enabled + assertThat(serverOptions.getTrustOptions()).isEqualTo(new PemTrustOptions().addCertPath(trustStorePath)); + assertThat(serverOptions.getClientAuth()).isEqualTo(ClientAuth.REQUEST); + + assertThat(serverOptions.isSsl()).isTrue(); + assertThat(serverOptions.getKeyCertOptions()).isEqualTo(new PfxOptions().setPath(keyStorePath)); + } + + /** + * Verifies that an MQTT server is bound to the configured port and address during startup and + * {@link AbstractMqttProtocolGateway#afterStartup(Promise)} is being invoked. + * + * @param ctx The helper to use for running async tests on vertx. + */ + @Test + public void testStartup(final VertxTestContext ctx) { + final int port = 1111; + final String bindAddress = "127.0.0.127"; + + // GIVEN a protocol gateway with port and address configured + final MqttProtocolGatewayConfig serverConfig = new MqttProtocolGatewayConfig(); + serverConfig.setPort(port); + serverConfig.setBindAddress(bindAddress); + + final TestMqttProtocolGateway gateway = createGateway(serverConfig); + + // WHEN starting the verticle + final Promise startupTracker = Promise.promise(); + gateway.start(startupTracker); + + // THEN the server starts to listen on the configured port and the start method completes + startupTracker.future().onComplete(ctx.succeeding(s -> { + + ctx.verify(() -> { + verify(netServer).listen(eq(port), eq(bindAddress), ProtocolGatewayTestHelper.anyHandler()); + assertThat(gateway.isStartupComplete()).isTrue(); + }); + ctx.completeNow(); + })); + + } + + /** + * Verifies that an MQTT server is bound to the configured port and address during startup and + * {@link AbstractMqttProtocolGateway#afterStartup(Promise)} is being invoked. + * + * @param ctx The helper to use for running async tests on vertx. + */ + @Test + public void testServerStopSucceeds(final VertxTestContext ctx) { + + // GIVEN a started protocol gateway + final TestMqttProtocolGateway gateway = createGateway(); + + final Promise startupTracker = Promise.promise(); + gateway.start(startupTracker); + + startupTracker.future().onComplete(ctx.succeeding(v -> { + + // WHEN stopping the verticle + final Promise stopTracker = Promise.promise(); + gateway.stop(stopTracker); + + stopTracker.future().onComplete(ctx.succeeding(ok -> { + + // THEN the MQTT server is closed and the shutdown completes + ctx.verify(() -> { + assertThat(gateway.isShutdownStarted()).isTrue(); + verify(netServer).close(ProtocolGatewayTestHelper.anyHandler()); + }); + ctx.completeNow(); + })); + })); + + } + + /** + * Verifies that the authentication with valid username and password succeeds. + */ + @Test + public void testConnectWithUsernamePasswordSucceeds() { + + // GIVEN a protocol gateway + final AbstractMqttProtocolGateway gateway = createGateway(); + + // WHEN connecting with known credentials + final MqttEndpoint mqttEndpoint = ProtocolGatewayTestHelper.connectMqttEndpoint(gateway, + TestMqttProtocolGateway.DEVICE_USERNAME, + TestMqttProtocolGateway.DEVICE_PASSWORD); + + // THEN the connection is accepted + verify(mqttEndpoint).accept(false); + + } + + /** + * Verifies that the authentication with invalid username fails. + */ + @Test + public void testAuthenticationWithWrongUsernameFails() { + + // GIVEN a protocol gateway + final TestMqttProtocolGateway gateway = createGateway(); + + // WHEN connecting with an unknown user + final MqttEndpoint mqttEndpoint = ProtocolGatewayTestHelper.connectMqttEndpoint(gateway, + "unknown-user", + TestMqttProtocolGateway.DEVICE_PASSWORD); + + // THEN the connection is rejected + verify(mqttEndpoint).reject(MqttConnectReturnCode.CONNECTION_REFUSED_NOT_AUTHORIZED); + + } + + /** + * Verifies that the authentication with invalid password fails. + */ + @Test + public void testAuthenticationWithWrongPasswordFails() { + + // GIVEN a protocol gateway + final TestMqttProtocolGateway gateway = createGateway(); + + // WHEN connecting with an invalid password + final MqttEndpoint mqttEndpoint = ProtocolGatewayTestHelper.connectMqttEndpoint(gateway, + TestMqttProtocolGateway.DEVICE_USERNAME, "wrong-password"); + + // THEN the connection is rejected + verify(mqttEndpoint).reject(MqttConnectReturnCode.CONNECTION_REFUSED_NOT_AUTHORIZED); + + } + + /** + * Verifies that the authentication with a valid client certificate succeeds. + */ + @Test + public void testConnectWithClientCertSucceeds() { + + final X509Certificate deviceCertificate = ProtocolGatewayTestHelper.createCertificate(); + + // GIVEN a protocol gateway configured with a trust anchor + final TestMqttProtocolGateway gateway = new TestMqttProtocolGateway(amqpClientConfig, + new MqttProtocolGatewayConfig(), vertx, amqpAdapterClientFactory) { + + @Override + protected Future> getTrustAnchors(final List certificates) { + // verification will always succeed because the client certificate is used as its own trust anchor + return Future.succeededFuture(Collections.singleton(new TrustAnchor(deviceCertificate, null))); + } + }; + + // WHEN connecting with a client certificate that can be validated by the trust anchor + final MqttEndpoint endpoint = ProtocolGatewayTestHelper.connectMqttEndpointWithClientCertificate(gateway, + deviceCertificate); + + // THEN the connection is accepted + verify(endpoint).accept(false); + } + + /** + * Verifies that the authentication with an invalid client certificate fails. + */ + @Test + public void testAuthenticationWithClientCertFailsIfTrustAnchorDoesNotMatch() { + + // GIVEN a protocol gateway configured with a trust anchor + final TestMqttProtocolGateway gateway = new TestMqttProtocolGateway(amqpClientConfig, + new MqttProtocolGatewayConfig(), vertx, amqpAdapterClientFactory) { + + @Override + protected Future> getTrustAnchors(final List certificates) { + // verification will fail because the certificate used for the trust anchor has nothing to do with the + // client certificate + final X509Certificate newCertificate = ProtocolGatewayTestHelper.createCertificate(); + return Future.succeededFuture(Collections.singleton(new TrustAnchor(newCertificate, null))); + } + }; + + // WHEN connecting with a client certificate that can NOT be validated by the trust anchor + final X509Certificate deviceCertificate = ProtocolGatewayTestHelper.createCertificate(); + final MqttEndpoint endpoint = ProtocolGatewayTestHelper + .connectMqttEndpointWithClientCertificate(gateway, deviceCertificate); + + // THEN the connection is rejected + verify(endpoint).reject(MqttConnectReturnCode.CONNECTION_REFUSED_NOT_AUTHORIZED); + } + + /** + * Verifies that the MQTT connection fails if the Hono instance is not available. + */ + @Test + public void testConnectFailsWhenGatewayCouldNotConnect() { + + // GIVEN a protocol gateway where establishing a connection to Hono's AMQP adapter fails + when(amqpAdapterClientFactory.connect()).thenReturn(Future.failedFuture("Connect failed")); + + final TestMqttProtocolGateway gateway = createGateway(); + + // WHEN a device connects + final MqttEndpoint endpoint = connectTestDevice(gateway); + + // THEN the connection is rejected + verify(endpoint).reject(MqttConnectReturnCode.CONNECTION_REFUSED_SERVER_UNAVAILABLE); + } + + /** + * Verifies that the credentials for the gateway provided by the implementation of + * {@link AbstractMqttProtocolGateway} are used to configure the connection to the AMQP adapter, if no credentials + * are provided in the client configuration. + */ + @Test + public void testConnectWithGatewayCredentialsResolvedDynamicallySucceeds() { + + // GIVEN a protocol gateway where the AMQP config does NOT contain credentials ... + // ... and where the gateway credentials are resolved by the implementation + final ClientConfigProperties configWithoutCredentials = new ClientConfigProperties(); + final AbstractMqttProtocolGateway gateway = new TestMqttProtocolGateway(configWithoutCredentials, + new MqttProtocolGatewayConfig(), vertx, amqpAdapterClientFactory) { + + @Override + AmqpAdapterClientFactory createTenantClientFactory(final String tenantId, + final ClientConfigProperties clientConfig) { + + // THEN the AMQP connection is authenticated with the provided credentials... + assertThat(clientConfig.getUsername()).isEqualTo(GW_USERNAME); + assertThat(clientConfig.getPassword()).isEqualTo(GW_PASSWORD); + + // ... and not with the credentials from the configuration + assertThat(clientConfig.getUsername()).isNotEqualTo(configWithoutCredentials.getUsername()); + assertThat(clientConfig.getPassword()).isNotEqualTo(configWithoutCredentials.getPassword()); + + return super.createTenantClientFactory(tenantId, clientConfig); + } + }; + + // WHEN the gateway connects + connectTestDevice(gateway); + + } + + /** + * Verifies that the credentials for the gateway provided by the client configuration are used to configure the + * connection to the AMQP adapter and take precedence over the ones provided by the implementation of + * {@link AbstractMqttProtocolGateway}. + */ + @Test + public void testConfiguredCredentialsTakePrecedenceOverImplementation() { + + final String username = "a-user"; + final String password = "a-password"; + final ClientConfigProperties configWithCredentials = new ClientConfigProperties(); + configWithCredentials.setUsername(username); + configWithCredentials.setPassword(password); + + // GIVEN a protocol gateway where the AMQP config does contains credentials + final AbstractMqttProtocolGateway gateway = new TestMqttProtocolGateway(configWithCredentials, + new MqttProtocolGatewayConfig(), vertx, amqpAdapterClientFactory) { + + @Override + AmqpAdapterClientFactory createTenantClientFactory(final String tenantId, + final ClientConfigProperties clientConfig) { + + // THEN the AMQP connection is authenticated with the configured credentials... + assertThat(clientConfig.getUsername()).isEqualTo(username); + assertThat(clientConfig.getPassword()).isEqualTo(password); + + // ... and not with the credentials from the implementation + assertThat(clientConfig.getUsername()).isNotEqualTo(GW_USERNAME); + assertThat(clientConfig.getPassword()).isNotEqualTo(GW_PASSWORD); + + return super.createTenantClientFactory(tenantId, clientConfig); + } + }; + + // WHEN the gateway connects + connectTestDevice(gateway); + + } + + /** + * Verifies that the downstream message constructed in + * {@link AbstractMqttProtocolGateway#onPublishedMessage(MqttDownstreamContext)} is set completely into the AMQP + * message sent downstream. + */ + @Test + public void testDownstreamMessage() { + + final String payload = "payload1"; + final String topic = "topic/1"; + + final ArgumentCaptor messageCaptor = ArgumentCaptor.forClass(Message.class); + + // GIVEN a protocol gateway with a MQTT endpoint connected + final TestMqttProtocolGateway gateway = createGateway(); + final MqttEndpoint mqttEndpoint = connectTestDevice(gateway); + + // WHEN sending a MQTT message + ProtocolGatewayTestHelper.sendMessage(mqttEndpoint, Buffer.buffer(payload), topic); + + // THEN the AMQP message contains the payload, application properties and content type + verify(protonSender).send(messageCaptor.capture(), any()); + + final Message amqpMessage = messageCaptor.getValue(); + + assertThat(MessageHelper.getPayloadAsString(amqpMessage)).isEqualTo(payload); + + assertThat(MessageHelper.getApplicationProperty(amqpMessage.getApplicationProperties(), + TestMqttProtocolGateway.KEY_APPLICATION_PROPERTY_TOPIC, String.class)).isEqualTo(topic); + assertThat(MessageHelper.getDeviceId(amqpMessage)).isEqualTo(TestMqttProtocolGateway.DEVICE_ID); + + assertThat(amqpMessage.getContentType()).isEqualTo(TestMqttProtocolGateway.CONTENT_TYPE); + } + + /** + * Verifies that an event message is being sent to the right address. + */ + @Test + public void testEventMessage() { + + final String payload = "payload1"; + final String topic = "topic/1"; + + final ArgumentCaptor messageCaptor = ArgumentCaptor.forClass(Message.class); + + // GIVEN a protocol gateway that sends every MQTT publish message as an event downstream and a connected MQTT + // endpoint + final TestMqttProtocolGateway gateway = createGateway(); + final MqttEndpoint mqttEndpoint = connectTestDevice(gateway); + + // WHEN sending a MQTT message + ProtocolGatewayTestHelper.sendMessage(mqttEndpoint, Buffer.buffer(payload), topic); + + // THEN the AMQP message contains the correct address + verify(protonSender).send(messageCaptor.capture(), any()); + + final String expectedAddress = "event/" + TestMqttProtocolGateway.TENANT_ID + "/" + + TestMqttProtocolGateway.DEVICE_ID; + assertThat(messageCaptor.getValue().getAddress()).isEqualTo(expectedAddress); + + } + + /** + * Verifies that a telemetry message is being sent to the right address. + */ + @Test + public void testTelemetryMessage() { + + final ArgumentCaptor messageCaptor = ArgumentCaptor.forClass(Message.class); + + // GIVEN a protocol gateway that sends every MQTT publish messages as telemetry messages downstream and a + // connected MQTT endpoint + final TestMqttProtocolGateway gateway = new TestMqttProtocolGateway(amqpClientConfig, + new MqttProtocolGatewayConfig(), vertx, amqpAdapterClientFactory) { + + @Override + protected Future onPublishedMessage(final MqttDownstreamContext ctx) { + return Future.succeededFuture(new TelemetryMessage(ctx.message().payload(), false)); + } + }; + + final MqttEndpoint mqttEndpoint = connectTestDevice(gateway); + + // WHEN sending a MQTT message + ProtocolGatewayTestHelper.sendMessage(mqttEndpoint, Buffer.buffer("payload"), "topic"); + + // THEN the AMQP message contains the correct address + verify(protonSender).send(messageCaptor.capture(), any()); + + final String expectedAddress = "telemetry/" + TestMqttProtocolGateway.TENANT_ID + "/" + + TestMqttProtocolGateway.DEVICE_ID; + assertThat(messageCaptor.getValue().getAddress()).isEqualTo(expectedAddress); + } + + /** + * Verifies that a command response message is constructed correctly and being sent to the right address. + */ + @Test + public void testCommandResponse() { + + final String payload = "payload1"; + final String correlationId = "the-correlation-id"; + final String replyId = "the-reply-id"; + final Integer status = 200; + + final ArgumentCaptor messageCaptor = ArgumentCaptor.forClass(Message.class); + + // GIVEN a protocol gateway that sends every MQTT publish messages as command response messages downstream and a + // connected MQTT endpoint + final TestMqttProtocolGateway gateway = new TestMqttProtocolGateway(amqpClientConfig, + new MqttProtocolGatewayConfig(), vertx, amqpAdapterClientFactory) { + + @Override + protected Future onPublishedMessage(final MqttDownstreamContext ctx) { + return Future.succeededFuture( + new CommandResponseMessage(replyId, correlationId, status.toString(), ctx.message().payload())); + } + }; + + final MqttEndpoint mqttEndpoint = connectTestDevice(gateway); + + // WHEN sending a MQTT message + ProtocolGatewayTestHelper.sendMessage(mqttEndpoint, Buffer.buffer(payload), "topic/123"); + + // THEN the AMQP message contains the required values and the correct address + verify(protonSender).send(messageCaptor.capture(), any()); + + final Message amqpMessage = messageCaptor.getValue(); + + assertThat(MessageHelper.getPayloadAsString(amqpMessage)).isEqualTo(payload); + assertThat(amqpMessage.getCorrelationId()).isEqualTo(correlationId); + assertThat(MessageHelper.getApplicationProperty(amqpMessage.getApplicationProperties(), + MessageHelper.APP_PROPERTY_STATUS, Integer.class)).isEqualTo(status); + + final String expectedAddress = "command_response/" + TestMqttProtocolGateway.TENANT_ID + "/" + + TestMqttProtocolGateway.DEVICE_ID + "/" + replyId; + assertThat(amqpMessage.getAddress()).isEqualTo(expectedAddress); + } + + /** + * Verifies that subscriptions are stored and acknowledged correctly. + */ + @Test + public void testCommandSubscription() { + + @SuppressWarnings("unchecked") + final ArgumentCaptor> subscribeAckCaptor = ArgumentCaptor.forClass(List.class); + + // GIVEN a protocol gateway and a connected MQTT endpoint + final TestMqttProtocolGateway gateway = createGateway(); + final MqttEndpoint mqttEndpoint = connectTestDevice(gateway); + + // WHEN sending a subscribe message with multiple topic filters + final int subscribeMsgId = ProtocolGatewayTestHelper.subscribe(mqttEndpoint, + new MqttTopicSubscription(TestMqttProtocolGateway.FILTER1, MqttQoS.AT_LEAST_ONCE), + new MqttTopicSubscription(TestMqttProtocolGateway.FILTER2, MqttQoS.AT_MOST_ONCE)); + + // THEN the subscriptions are acknowledged correctly... + verify(mqttEndpoint).subscribeAcknowledge(eq(subscribeMsgId), subscribeAckCaptor.capture()); + + assertThat(subscribeAckCaptor.getValue()).isEqualTo(Arrays.asList(MqttQoS.AT_LEAST_ONCE, MqttQoS.AT_MOST_ONCE)); + + // ... and the internal map is correct as well + final Map subscriptions = gateway.getCommandSubscriptionsManager() + .getSubscriptions(); + + assertThat(subscriptions.size()).isEqualTo(2); + assertThat(subscriptions.get(TestMqttProtocolGateway.FILTER1).getQos()).isEqualTo(MqttQoS.AT_LEAST_ONCE); + assertThat(subscriptions.get(TestMqttProtocolGateway.FILTER2).getQos()).isEqualTo(MqttQoS.AT_MOST_ONCE); + } + + /** + * Verifies that when a device tries to subscribe using the unsupported QoS 2, then it is only granted QoS 1. + */ + @Test + public void testCommandSubscriptionDowngradesQoS2() { + + @SuppressWarnings("unchecked") + final ArgumentCaptor> subscribeAckCaptor = ArgumentCaptor.forClass(List.class); + + // GIVEN a protocol gateway and a connected MQTT endpoint + final TestMqttProtocolGateway gateway = createGateway(); + final MqttEndpoint mqttEndpoint = connectTestDevice(gateway); + + // WHEN sending a subscribe message that requests QoS 2 + final int subscribeMsgId = ProtocolGatewayTestHelper.subscribe(mqttEndpoint, + new MqttTopicSubscription(TestMqttProtocolGateway.FILTER1, MqttQoS.EXACTLY_ONCE)); + + // THEN the QoS is downgraded to QoS 1 in the acknowledgement... + verify(mqttEndpoint).subscribeAcknowledge(eq(subscribeMsgId), subscribeAckCaptor.capture()); + + assertThat(subscribeAckCaptor.getValue()).isEqualTo(Collections.singletonList(MqttQoS.AT_LEAST_ONCE)); + + // ... and in the internal map as well + final Map subscriptions = gateway.getCommandSubscriptionsManager() + .getSubscriptions(); + + assertThat(subscriptions.get(TestMqttProtocolGateway.FILTER1).getQos()).isEqualTo(MqttQoS.AT_LEAST_ONCE); + } + + /** + * Verifies that no subscriptions are being accepted for unsupported topic filters. + */ + @Test + public void testCommandSubscriptionFailsForInvalidTopicFilter() { + + @SuppressWarnings("unchecked") + final ArgumentCaptor> subscribeAckCaptor = ArgumentCaptor.forClass(List.class); + + // GIVEN a protocol gateway and a connected MQTT endpoint + final TestMqttProtocolGateway gateway = createGateway(); + final MqttEndpoint mqttEndpoint = connectTestDevice(gateway); + + // WHEN sending a subscribe message with a topic filter that the gateway does not provide + final int subscribeMsgId = ProtocolGatewayTestHelper.subscribe(mqttEndpoint, + TestMqttProtocolGateway.FILTER_INVALID); + + // THEN the subscription is acknowledged correctly as a failure... + verify(mqttEndpoint).subscribeAcknowledge(eq(subscribeMsgId), subscribeAckCaptor.capture()); + + assertThat(subscribeAckCaptor.getValue()).isEqualTo(Collections.singletonList(MqttQoS.FAILURE)); + + // ... and it is not contained in the internal map + final Map subscriptions = gateway.getCommandSubscriptionsManager() + .getSubscriptions(); + + assertThat(subscriptions.isEmpty()).isTrue(); + } + + /** + * Verifies that when the protocol gateway receives a command for a subscribed device, then the command is published + * via MQTT to the device. + */ + @Test + public void testReceiveCommand() { + final String subject = "the/subject"; + final String replyTo = "the/reply/address"; + final String correlationId = "the-correlation-id"; + final String messageId = "the-message-id"; + + final Message commandMessage = new MessageImpl(); + MessageHelper.setJsonPayload(commandMessage, TestMqttProtocolGateway.PAYLOAD); + commandMessage.setSubject(subject); + commandMessage.setReplyTo(replyTo); + commandMessage.setCorrelationId(correlationId); + commandMessage.setMessageId(messageId); + + final JsonObject expected = new JsonObject() + .put(TestMqttProtocolGateway.KEY_SUBJECT, subject) + .put(TestMqttProtocolGateway.KEY_REPLY_TO, replyTo) + .put(TestMqttProtocolGateway.KEY_CORRELATION_ID, correlationId) + .put(TestMqttProtocolGateway.KEY_MESSAGE_ID, messageId) + .put(TestMqttProtocolGateway.KEY_COMMAND_PAYLOAD, TestMqttProtocolGateway.PAYLOAD) + .put(TestMqttProtocolGateway.KEY_CONTENT_TYPE, TestMqttProtocolGateway.CONTENT_TYPE); + + // GIVEN a protocol gateway and a connected MQTT endpoint with a command subscription + final TestMqttProtocolGateway gateway = createGateway(); + + final MqttEndpoint mqttEndpoint = connectTestDevice(gateway); + + ProtocolGatewayTestHelper.subscribe(mqttEndpoint, TestMqttProtocolGateway.FILTER1); + + // WHEN receiving the command + commandHandler.accept(commandMessage); + + // THEN the command is published to the MQTT endpoint + final ArgumentCaptor payloadCaptor = ArgumentCaptor.forClass(Buffer.class); + + verify(mqttEndpoint).publish(eq(TestMqttProtocolGateway.COMMAND_TOPIC), payloadCaptor.capture(), + eq(MqttQoS.AT_LEAST_ONCE), eq(false), eq(false), any()); + + assertThat(payloadCaptor.getValue().toJsonObject()).isEqualTo(expected); + + } + + /** + * Verifies that subscriptions are remove when unsubscribing. + */ + @Test + public void testUnsubscribe() { + + // GIVEN a protocol gateway and a connected MQTT endpoint with two subscriptions + final TestMqttProtocolGateway gateway = createGateway(); + final MqttEndpoint mqttEndpoint = connectTestDevice(gateway); + + ProtocolGatewayTestHelper.subscribe(mqttEndpoint, + new MqttTopicSubscription(TestMqttProtocolGateway.FILTER1, MqttQoS.AT_LEAST_ONCE), + new MqttTopicSubscription(TestMqttProtocolGateway.FILTER2, MqttQoS.AT_MOST_ONCE)); + + // WHEN sending an unsubscribe message containing one of the topic filters and a third onw + final int unsubscribeMsgId = ProtocolGatewayTestHelper.unsubscribe(mqttEndpoint, + TestMqttProtocolGateway.FILTER2, TestMqttProtocolGateway.FILTER_INVALID); + + // THEN the message is acknowledged + verify(mqttEndpoint).unsubscribeAcknowledge(eq(unsubscribeMsgId)); + + // ... and the internal map is correct as well + final Map subscriptions = gateway.getCommandSubscriptionsManager() + .getSubscriptions(); + assertThat(subscriptions.size()).isEqualTo(1); + assertThat(subscriptions.containsKey(TestMqttProtocolGateway.FILTER1)).isTrue(); + assertThat(subscriptions.containsKey(TestMqttProtocolGateway.FILTER2)).isFalse(); + + } + + /** + * Verifies that when the MQTT connections is being closed, the subscriptions are removed and + * {@link AbstractMqttProtocolGateway#onDeviceConnectionClose(MqttEndpoint)} is invoked. + */ + @Test + public void testConnectionClose() { + + // GIVEN a protocol gateway and a connected MQTT endpoint with subscriptions + final TestMqttProtocolGateway gateway = createGateway(); + final MqttEndpoint mqttEndpoint = connectTestDevice(gateway); + + ProtocolGatewayTestHelper.subscribe(mqttEndpoint, + new MqttTopicSubscription(TestMqttProtocolGateway.FILTER1, MqttQoS.AT_LEAST_ONCE), + new MqttTopicSubscription(TestMqttProtocolGateway.FILTER2, MqttQoS.AT_MOST_ONCE)); + + // WHEN the connection is closed + mqttEndpoint.close(); + + // THEN the subscriptions are removed ... + assertThat(gateway.getCommandSubscriptionsManager().getSubscriptions().isEmpty()).isTrue(); + + // ... and the callback onDeviceConnectionClose() has been invoked + assertThat(gateway.isConnectionClosed()).isTrue(); + } + + /** + * Creates a mocked Hono connection that returns a Noop Tracer. + * + * @param vertx The vert.x instance to use. + * @param props The client properties to use. + * @param protonSender The proton sender to use. + * @return The connection. + */ + private HonoConnection mockHonoConnection(final Vertx vertx, final ClientConfigProperties props, + final ProtonSender protonSender) { + + final Tracer tracer = NoopTracerFactory.create(); + final HonoConnection connection = mock(HonoConnection.class); + when(connection.getVertx()).thenReturn(vertx); + when(connection.getConfig()).thenReturn(props); + when(connection.getTracer()).thenReturn(tracer); + when(connection.isConnected(anyLong())).thenReturn(Future.succeededFuture()); + when(connection.executeOnContext(ProtocolGatewayTestHelper.anyHandler())).then(invocation -> { + final Promise result = Promise.promise(); + final Handler> handler = invocation.getArgument(0); + handler.handle(result.future()); + return result.future(); + }); + + when(connection.getTracer()).thenReturn(tracer); + when(connection.createSender(any(), any(), any())).thenReturn(Future.succeededFuture(protonSender)); + + final ProtonReceiver receiver = mockProtonReceiver(); + when(connection.createReceiver(anyString(), any(), any(), any())).thenReturn(Future.succeededFuture(receiver)); + + return connection; + } + + /** + * Creates a mocked Proton sender which always returns {@code true} when its isOpen method is called. + * + * @return The mocked sender. + */ + private ProtonSender mockProtonSender() { + + final ProtonSender sender = mock(ProtonSender.class); + when(sender.isOpen()).thenReturn(Boolean.TRUE); + when(sender.getQoS()).thenReturn(ProtonQoS.AT_LEAST_ONCE); + when(sender.getTarget()).thenReturn(mock(Target.class)); + + return sender; + } + + /** + * Creates a mocked Proton receiver which always returns {@code true} when its isOpen method is called. + * + * @return The mocked receiver. + */ + public ProtonReceiver mockProtonReceiver() { + + final ProtonReceiver receiver = mock(ProtonReceiver.class); + when(receiver.isOpen()).thenReturn(Boolean.TRUE); + when(receiver.getSource()).thenReturn(new Source()); + + return receiver; + } + + private void setCommandHandler(final Consumer msgHandler) { + commandHandler = msgHandler; + } + + private MqttEndpoint connectTestDevice(final AbstractMqttProtocolGateway gateway) { + return ProtocolGatewayTestHelper.connectMqttEndpoint(gateway, + TestMqttProtocolGateway.DEVICE_USERNAME, + TestMqttProtocolGateway.DEVICE_PASSWORD); + } + + private TestMqttProtocolGateway createGateway() { + return createGateway(new MqttProtocolGatewayConfig()); + } + + private TestMqttProtocolGateway createGateway(final MqttProtocolGatewayConfig gatewayServerConfig) { + return new TestMqttProtocolGateway(amqpClientConfig, gatewayServerConfig, vertx, amqpAdapterClientFactory); + } + +} diff --git a/protocol-gateway/mqtt-protocol-gateway-template/src/test/java/org/eclipse/hono/gateway/sdk/mqtt2amqp/ProtocolGatewayTestHelper.java b/protocol-gateway/mqtt-protocol-gateway-template/src/test/java/org/eclipse/hono/gateway/sdk/mqtt2amqp/ProtocolGatewayTestHelper.java new file mode 100644 index 00000000..1be0cda9 --- /dev/null +++ b/protocol-gateway/mqtt-protocol-gateway-template/src/test/java/org/eclipse/hono/gateway/sdk/mqtt2amqp/ProtocolGatewayTestHelper.java @@ -0,0 +1,262 @@ +/** + * Copyright (c) 2020 Contributors to the Eclipse Foundation + * + * See the NOTICE file(s) distributed with this work for additional + * information regarding copyright ownership. + * + * This program and the accompanying materials are made available under the + * terms of the Eclipse Public License 2.0 which is available at + * http://www.eclipse.org/legal/epl-2.0 + * + * SPDX-License-Identifier: EPL-2.0 + */ + +package org.eclipse.hono.gateway.sdk.mqtt2amqp; + +import static org.mockito.Mockito.doAnswer; +import static org.mockito.Mockito.mock; +import static org.mockito.Mockito.verify; +import static org.mockito.Mockito.when; + +import java.io.FileInputStream; +import java.io.FileNotFoundException; +import java.security.cert.Certificate; +import java.security.cert.CertificateException; +import java.security.cert.CertificateFactory; +import java.security.cert.X509Certificate; +import java.util.Arrays; +import java.util.concurrent.ThreadLocalRandom; + +import javax.net.ssl.SSLPeerUnverifiedException; +import javax.net.ssl.SSLSession; + +import org.mockito.ArgumentCaptor; +import org.mockito.ArgumentMatchers; + +import io.netty.handler.codec.mqtt.MqttQoS; +import io.netty.handler.codec.mqtt.MqttTopicSubscription; +import io.vertx.core.Handler; +import io.vertx.core.buffer.Buffer; +import io.vertx.core.net.SelfSignedCertificate; +import io.vertx.mqtt.MqttAuth; +import io.vertx.mqtt.MqttEndpoint; +import io.vertx.mqtt.messages.MqttPublishMessage; +import io.vertx.mqtt.messages.MqttSubscribeMessage; +import io.vertx.mqtt.messages.MqttUnsubscribeMessage; +import io.vertx.mqtt.messages.impl.MqttSubscribeMessageImpl; +import io.vertx.mqtt.messages.impl.MqttUnsubscribeMessageImpl; + +/** + * Support for mocking MQTT connections to a {@link AbstractMqttProtocolGateway}. + **/ +public final class ProtocolGatewayTestHelper { + + private ProtocolGatewayTestHelper() { + } + + /** + * Creates a mocked MQTT endpoint and connects it to the given protocol gateway by calling the endpoint handler. + * Authenticates with the given username and password. + * + * @param gateway The protocol gateway to connect to. + * @param username The username. + * @param password The password. + * @return The connected and authenticated endpoint mock. + */ + public static MqttEndpoint connectMqttEndpoint(final AbstractMqttProtocolGateway gateway, + final String username, + final String password) { + + final MqttEndpoint endpoint = createMqttEndpoint(); + + when(endpoint.auth()).thenReturn(new MqttAuth(username, password)); + + gateway.handleEndpointConnection(endpoint); + return endpoint; + } + + /** + * Creates a mocked MQTT endpoint and connects it to the given protocol gateway by calling the endpoint handler. + * Authenticates with the given client certificate. + * + * @param gateway The protocol gateway to connect to. + * @param deviceCertificate The X.509 client certificate. + * @return The connected and authenticated endpoint mock. + */ + public static MqttEndpoint connectMqttEndpointWithClientCertificate(final AbstractMqttProtocolGateway gateway, + final X509Certificate deviceCertificate) { + + final MqttEndpoint endpoint = createMqttEndpoint(); + + when(endpoint.isSsl()).thenReturn(true); + + final SSLSession sslSession = mock(SSLSession.class); + try { + when(sslSession.getPeerCertificates()).thenReturn(new Certificate[] { deviceCertificate }); + } catch (SSLPeerUnverifiedException e) { + throw new RuntimeException("this should not be possible", e); + } + when(endpoint.sslSession()).thenReturn(sslSession); + + gateway.handleEndpointConnection(endpoint); + return endpoint; + } + + /** + * Simulates sending a MQTT subscribe message by invoking the subscribe handler, that has been set on the given mock + * endpoint during the connection establishment in one of the "connect..." methods in this class. + *

+ * The given topics are all subscribed with QoS "AT_LEAST_ONCE". + * + * @param endpoint The connected endpoint mock. + * @param topicFilters The topic filters to subscribe for. + * @return A random message id. + * + * @see #connectMqttEndpoint(AbstractMqttProtocolGateway, String, String) + * @see #connectMqttEndpointWithClientCertificate(AbstractMqttProtocolGateway, X509Certificate) + */ + public static int subscribe(final MqttEndpoint endpoint, final String... topicFilters) { + + final MqttTopicSubscription[] mqttTopicSubscriptions = Arrays.stream(topicFilters) + .map(topic -> new MqttTopicSubscription(topic, MqttQoS.AT_LEAST_ONCE)) + .toArray(MqttTopicSubscription[]::new); + + return subscribe(endpoint, mqttTopicSubscriptions); + } + + /** + * Simulates sending a MQTT subscribe message by invoking the subscribe handler, that has been set on the given mock + * endpoint during the connection establishment in one of the "connect..." methods in this class. + * + * @param endpoint The connected endpoint mock. + * @param subscriptions The topic subscriptions to subscribe for. + * @return A random message id. + * + * @see #connectMqttEndpoint(AbstractMqttProtocolGateway, String, String) + * @see #connectMqttEndpointWithClientCertificate(AbstractMqttProtocolGateway, X509Certificate) + */ + public static int subscribe(final MqttEndpoint endpoint, final MqttTopicSubscription... subscriptions) { + + final ArgumentCaptor> captor = argumentCaptorHandler(); + verify(endpoint).subscribeHandler(captor.capture()); + + final int messageId = newRandomMessageId(); + captor.getValue().handle(new MqttSubscribeMessageImpl(messageId, Arrays.asList(subscriptions))); + + return messageId; + } + + /** + * Simulates sending a MQTT unsubscribe message by invoking the unsubscribe handler, that has been set on the given + * mock endpoint during the connection establishment in one of the "connect..." methods in this class. + * + * @param endpoint The connected endpoint mock. + * @param topics The topic filters to unsubscribe. + * @return A random message id. + * + * @see #connectMqttEndpoint(AbstractMqttProtocolGateway, String, String) + * @see #connectMqttEndpointWithClientCertificate(AbstractMqttProtocolGateway, X509Certificate) + */ + public static int unsubscribe(final MqttEndpoint endpoint, final String... topics) { + + final ArgumentCaptor> captor = argumentCaptorHandler(); + verify(endpoint).unsubscribeHandler(captor.capture()); + + final int messageId = newRandomMessageId(); + captor.getValue().handle(new MqttUnsubscribeMessageImpl(messageId, Arrays.asList(topics))); + + return messageId; + } + + /** + * Simulates sending a MQTT publish message by invoking the publish handler, that has been set on the given mock + * endpoint during the connection establishment in one of the "connect..." methods in this class. + * + * @param endpoint The connected endpoint mock. + * @param payload The payload of the message. + * @param topic The topic of the message. + * + * @see #connectMqttEndpoint(AbstractMqttProtocolGateway, String, String) + * @see #connectMqttEndpointWithClientCertificate(AbstractMqttProtocolGateway, X509Certificate) + */ + public static void sendMessage(final MqttEndpoint endpoint, final Buffer payload, final String topic) { + + final ArgumentCaptor> captor = argumentCaptorHandler(); + verify(endpoint).publishHandler(captor.capture()); + + final MqttPublishMessage mqttMessage = mock(MqttPublishMessage.class); + when(mqttMessage.payload()).thenReturn(payload); + when(mqttMessage.topicName()).thenReturn(topic); + + captor.getValue().handle(mqttMessage); + } + + /** + * Returns a self signed certificate. + * + * @return A new X.509 certificate. + */ + public static X509Certificate createCertificate() { + final SelfSignedCertificate selfSignedCert = SelfSignedCertificate.create("eclipse.org"); + try { + final CertificateFactory factory = CertificateFactory.getInstance("X.509"); + return (X509Certificate) factory.generateCertificate(new FileInputStream(selfSignedCert.certificatePath())); + } catch (CertificateException | FileNotFoundException e) { + throw new RuntimeException("Generating self signed cert failed", e); + } + } + + /** + * Matches any handler of given type, excluding nulls. + * + * @param The handler type. + * @return The value returned by {@link ArgumentMatchers#any(Class)}. + */ + public static Handler anyHandler() { + @SuppressWarnings("unchecked") + final Handler result = ArgumentMatchers.any(Handler.class); + return result; + } + + /** + * Argument captor for a handler. + * + * @param The handler type. + * @return The value returned by {@link ArgumentCaptor#forClass(Class)}. + */ + public static ArgumentCaptor> argumentCaptorHandler() { + @SuppressWarnings("unchecked") + final ArgumentCaptor> result = ArgumentCaptor.forClass(Handler.class); + return result; + } + + private static MqttEndpoint createMqttEndpoint() { + + final MqttEndpoint endpoint = mock(MqttEndpoint.class); + when(endpoint.isConnected()).thenReturn(true); + when(endpoint.clientIdentifier()).thenReturn("the-client-id"); + addCloseHandlerToEndpoint(endpoint); + + return endpoint; + } + + /** + * When the endpoint is closed, the close handler is invoked. + */ + private static void addCloseHandlerToEndpoint(final MqttEndpoint endpoint) { + when(endpoint.closeHandler(anyHandler())).then(invocation -> { + final Handler handler = invocation.getArgument(0); + doAnswer(s -> { + when(endpoint.isConnected()).thenReturn(false); + handler.handle(null); + return null; + }).when(endpoint).close(); + return null; + }); + } + + private static int newRandomMessageId() { + return ThreadLocalRandom.current().nextInt(); + } + +} diff --git a/protocol-gateway/mqtt-protocol-gateway-template/src/test/java/org/eclipse/hono/gateway/sdk/mqtt2amqp/TestMqttProtocolGateway.java b/protocol-gateway/mqtt-protocol-gateway-template/src/test/java/org/eclipse/hono/gateway/sdk/mqtt2amqp/TestMqttProtocolGateway.java new file mode 100644 index 00000000..76036cd4 --- /dev/null +++ b/protocol-gateway/mqtt-protocol-gateway-template/src/test/java/org/eclipse/hono/gateway/sdk/mqtt2amqp/TestMqttProtocolGateway.java @@ -0,0 +1,200 @@ +/** + * Copyright (c) 2020 Contributors to the Eclipse Foundation + * + * See the NOTICE file(s) distributed with this work for additional + * information regarding copyright ownership. + * + * This program and the accompanying materials are made available under the + * terms of the Eclipse Public License 2.0 which is available at + * http://www.eclipse.org/legal/epl-2.0 + * + * SPDX-License-Identifier: EPL-2.0 + */ + +package org.eclipse.hono.gateway.sdk.mqtt2amqp; + +import java.security.cert.X509Certificate; +import java.util.concurrent.atomic.AtomicBoolean; + +import org.eclipse.hono.auth.Device; +import org.eclipse.hono.client.device.amqp.AmqpAdapterClientFactory; +import org.eclipse.hono.config.ClientConfigProperties; +import org.eclipse.hono.gateway.sdk.mqtt2amqp.Command; +import org.eclipse.hono.gateway.sdk.mqtt2amqp.CommandSubscriptionsManager; +import org.eclipse.hono.gateway.sdk.mqtt2amqp.Credentials; +import org.eclipse.hono.gateway.sdk.mqtt2amqp.MqttCommandContext; +import org.eclipse.hono.gateway.sdk.mqtt2amqp.MqttDownstreamContext; +import org.eclipse.hono.gateway.sdk.mqtt2amqp.MqttProtocolGatewayConfig; +import org.eclipse.hono.gateway.sdk.mqtt2amqp.downstream.DownstreamMessage; +import org.eclipse.hono.gateway.sdk.mqtt2amqp.downstream.EventMessage; + +import io.vertx.core.Future; +import io.vertx.core.Promise; +import io.vertx.core.Vertx; +import io.vertx.core.json.JsonObject; +import io.vertx.mqtt.MqttEndpoint; + +/** + * A {@link AbstractMqttProtocolGateway} implementation for testing purposes. It handles only one device. + */ +class TestMqttProtocolGateway extends AbstractMqttProtocolGateway { + + public static final String DEVICE_USERNAME = "device-user"; + public static final String DEVICE_PASSWORD = "device-password"; + public static final String TENANT_ID = "the-tenant"; + public static final String DEVICE_ID = "the-device-id"; + public static final Device DEVICE = new Device(TENANT_ID, DEVICE_ID); + + public static final String GW_USERNAME = "gw@tenant2"; + public static final String GW_PASSWORD = "gw-secret"; + + public static final JsonObject PAYLOAD = new JsonObject("{\"the-key\": \"the-value\"}"); + public static final String CONTENT_TYPE = "application/json"; + public static final String COMMAND_TOPIC = "the/command/topic"; + public static final String FILTER1 = "topic/FILTER1/#"; + public static final String FILTER2 = "topic/FILTER2/#"; + public static final String FILTER_INVALID = "unknown/#"; + public static final String KEY_COMMAND_PAYLOAD = "command-payload"; + public static final String KEY_SUBJECT = "subject"; + public static final String KEY_REPLY_TO = "reply-to"; + public static final String KEY_CORRELATION_ID = "correlation-id"; + public static final String KEY_MESSAGE_ID = "message-id"; + public static final String KEY_CONTENT_TYPE = "content-type"; + public static final String KEY_APPLICATION_PROPERTIES = "application-properties"; + public static final String KEY_APPLICATION_PROPERTY_TOPIC = "topic"; + + private final AtomicBoolean startupComplete = new AtomicBoolean(); + private final AtomicBoolean shutdownStarted = new AtomicBoolean(); + private final AtomicBoolean connectionClosed = new AtomicBoolean(); + private final AmqpAdapterClientFactory amqpAdapterClientFactory; + + private CommandSubscriptionsManager commandSubscriptionsManager; + + TestMqttProtocolGateway(final ClientConfigProperties clientConfigProperties, + final MqttProtocolGatewayConfig mqttProtocolGatewayConfig, final Vertx vertx, + final AmqpAdapterClientFactory amqpAdapterClientFactory) { + super(clientConfigProperties, mqttProtocolGatewayConfig); + this.amqpAdapterClientFactory = amqpAdapterClientFactory; + super.vertx = vertx; + } + + /** + * Checks if the startup completed. + * + * @return {@code true} if {@link AbstractMqttProtocolGateway#afterStartup(Promise)} has been invoked. + */ + public boolean isStartupComplete() { + return startupComplete.get(); + } + + /** + * Checks if the shutdown has been initiated. + * + * @return {@code true} if {@link AbstractMqttProtocolGateway#beforeShutdown(Promise)} has been invoked. + */ + public boolean isShutdownStarted() { + return shutdownStarted.get(); + } + + /** + * Checks if the connection to a device has been closed. + * + * @return {@code true} if {@link AbstractMqttProtocolGateway#onDeviceConnectionClose(MqttEndpoint)} has been + * invoked. + */ + public boolean isConnectionClosed() { + return connectionClosed.get(); + } + + /** + * Return the command subscription manager for the test device. + * + * @return The command subscription manager that has been created during the establishment of the device connection. + */ + public CommandSubscriptionsManager getCommandSubscriptionsManager() { + return commandSubscriptionsManager; + } + + @Override + AmqpAdapterClientFactory createTenantClientFactory(final String tenantId, + final ClientConfigProperties clientConfig) { + return amqpAdapterClientFactory; + } + + @Override + protected Future authenticateDevice(final String username, final String password, + final String clientId) { + if (DEVICE_USERNAME.equals(username) && DEVICE_PASSWORD.equals(password)) { + return Future.succeededFuture(DEVICE); + } else { + return Future.failedFuture("auth failed"); + } + } + + @Override + protected boolean isTopicFilterValid(final String topicFilter, final String tenantId, final String deviceId, + final String clientId) { + return FILTER1.equals(topicFilter) || FILTER2.equals(topicFilter); + } + + @Override + protected Future onPublishedMessage(final MqttDownstreamContext ctx) { + final EventMessage message = new EventMessage(ctx.message().payload()); + message.addApplicationProperty(KEY_APPLICATION_PROPERTY_TOPIC, ctx.topic()); + message.setContentType(CONTENT_TYPE); + + return Future.succeededFuture(message); + } + + @Override + protected Command onCommandReceived(final MqttCommandContext ctx) { + final JsonObject payload = new JsonObject(); + payload.put(KEY_COMMAND_PAYLOAD, ctx.getPayload().toJson()); + payload.put(KEY_SUBJECT, ctx.getSubject()); + payload.put(KEY_REPLY_TO, ctx.getReplyTo()); + payload.put(KEY_CORRELATION_ID, ctx.getCorrelationId()); + payload.put(KEY_MESSAGE_ID, ctx.getMessageId()); + payload.put(KEY_CONTENT_TYPE, ctx.getContentType()); + + if (ctx.getApplicationProperties() != null) { + payload.put(KEY_APPLICATION_PROPERTIES, new JsonObject(ctx.getApplicationProperties().getValue())); + + } + + return new Command(COMMAND_TOPIC, FILTER1, payload.toBuffer()); + } + + @Override + protected Future authenticateClientCertificate(final X509Certificate deviceCertificate) { + return Future.succeededFuture(DEVICE); + } + + @Override + protected Future provideGatewayCredentials(final String tenantId) { + return Future.succeededFuture(new Credentials(GW_USERNAME, GW_PASSWORD)); + } + + @Override + protected void afterStartup(final Promise startPromise) { + startupComplete.compareAndSet(false, true); + super.afterStartup(startPromise); + } + + @Override + protected void beforeShutdown(final Promise stopPromise) { + shutdownStarted.compareAndSet(false, true); + super.beforeShutdown(stopPromise); + } + + @Override + CommandSubscriptionsManager createCommandHandler(final Vertx vertx) { + commandSubscriptionsManager = super.createCommandHandler(vertx); + return commandSubscriptionsManager; + } + + @Override + protected void onDeviceConnectionClose(final MqttEndpoint endpoint) { + connectionClosed.compareAndSet(false, true); + } + +} diff --git a/protocol-gateway/mqtt-protocol-gateway-template/src/test/resources/emptyKeyStoreFile.p12 b/protocol-gateway/mqtt-protocol-gateway-template/src/test/resources/emptyKeyStoreFile.p12 new file mode 100644 index 00000000..e69de29b diff --git a/protocol-gateway/mqtt-protocol-gateway-template/src/test/resources/emptyTrustStoreFile.pem b/protocol-gateway/mqtt-protocol-gateway-template/src/test/resources/emptyTrustStoreFile.pem new file mode 100644 index 00000000..e69de29b diff --git a/protocol-gateway/mqtt-protocol-gateway-template/src/test/resources/logback.xml b/protocol-gateway/mqtt-protocol-gateway-template/src/test/resources/logback.xml new file mode 100644 index 00000000..255aebc6 --- /dev/null +++ b/protocol-gateway/mqtt-protocol-gateway-template/src/test/resources/logback.xml @@ -0,0 +1,32 @@ + + + + + + + + + + + %d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n + + + + + + + + +