HiveMQ Enterprise Bridge Extension

The HiveMQ Enterprise Bridge Extension enables HiveMQ to bridge to one or more MQTT brokers for scalable, reliable, and bi-directional exchange of MQTT messages.

Features

Customizable bidirectional topic mapping that allows you to configure a set of topic filters that forward matching messages to specific destination topics at another MQTT broker.
Fine-grained configuration of the source topic filter and destination topic with the use of all MQTT-specific variables as well as configuration-specific templating.
Extensive logging capabilities and useful metrics for monitoring.

Requirements

An MQTT bridge broker that runs HiveMQ Professional or Enterprise Edition installation, version 4.4.0 or higher.
A remote MQTT message broker that is compliant with the MQTT 5 specification. The remote broker can be a HiveMQ broker or a different MQTT-5 compliant broker.

Installation

Place your HiveMQ Enterprise Bridge Extension license file (.elic) in the license folder of your HiveMQ installation.
Move the entire hivemq-bridge-extension folder into the extensions folder of your HiveMQ installation.
In the bridge-configuration.xml, configure the Enterprise Bridge Extension as desired. An example from the configuration-samples folder can be used to get started. For more information, see Configuration.
For HiveMQ to recognize and start the extension, the bridge-configuration.xml must be located in the hivemq-bridge-extension folder.
To use the bridge-configuration.xml to adjust the logging scope for your individual use case, copy the logback-example-adapter in the logback-example.xml file to the logback.xml of your HiveMQ installation.

└─ <HiveMQ folder>
    ├─ bin
    ├─ conf
    │   ├─ logback.xml
    ├─ data
    ├─ extensions
    │   ├─ hivemq-bridge-extension
    │   ├──── bridge-configuration.xml
    │   ├──── bridge-extension.jar
    │   ├──── hivemq-extension.xml
    │   └──── ...
    │   └─ ...
    ├─ license
    ├─ log
    └─ ...

If you want to bridge from an entire HiveMQ cluster, you must install identically configured Enterprise Bridge Extensions on every broker node in the HiveMQ cluster.

To verify the installation and configuration of the bridge extension, check your HiveMQ logs.

You can change the configuration of your HiveMQ Enterprise Bridge Extension during runtime:

Add a DISABLED file to the hivemq-bridge-extension folder.
Change the configuration of the extension in the bridge-configuration.xml file.
Remove the DISABLED file from the hivemq-bridge-extension folder.

Configuration

The minimum bridge extension configuration must contain the following items:

A bridge name that is unique within your local node structure.
Connection information for the remote broker.
A topic filter to forward incoming publishes from MQTT clients on the bridge broker to the remote broker.

To use the bridge extension, you must set up and store a configuration file named bridge-configuration.xml in the hivemq-bridge-extension folder that is located in the extensions directory of your HiveMQ installation.

Example minimum bridge extension configuration

<hivemq-bridge-extension>
    <bridges>
        <bridge>
            <name>your-unique-bridge-name</name>
            <remote-broker>
                <connection>
                    <static>
                        <host>the.hostname.of.remote.broker</host>
                    </static>
                </connection>
            </remote-broker>
            <topics>
                <topic>
                    <filter>a/topic/filter</filter>
                </topic>
            </topics>
        </bridge>
    </bridges>
</hivemq-bridge-extension>

A basic configuration example can be found in the configuration-samples folder of the Enterprise Bridge Extension.

Bridge Extension Configuration Options

The HiveMQ Enterprise Bridge Extension provides many flexible configuration options. You can adjust the settings of each bridge to meet your needs. At least one bridge must be configured in the bridge extension.

Configuration Required Description

bridge

The configuration settings for one bridge:

enabled: Defines whether the selected bridge is enabled.
The default setting is true.
name: The unique identifier of the bridge. The name must be unique per cluster node. The name can contain the following characters: [a-z] [A-Z] [0-9] - _.
The environment variable $ENV{HIVEMQ_BRIDGE_NAME} can be used to name the bridge.

remote-broker

Configuration of the remote broker to which bridge clients connect.

authentication: Optional setting that defines the type of authentication used for the MQTT connection.
By default, no values are set:
- username: The username required to establish the MQTT connection
- password: The password required to establish the MQTT connection
connection: Connection information for the MQTT listener of the remote broker:
- host: The hostname of the MQTT listener.
  The environment variable $ENV{REMOTE_BROKER_HOST} can be used to set the hostname.
- port: The port of the MQTT listener.
  The environment variable $ENV{REMOTE_BROKER_PORT} can be used to set the listener port. By default, the port is set on 1883.
- websocket: Optional WebSocket configuration for the MQTT connection:
  - enabled: Defines whether the WebSocket configuration is enabled. The default value is false.
  - server-path:
mqtt: Optional internal settings for establishment of MQTT connections for MQTT 5 clients.
- clean-start: Defines whether the client wants to start a new session upon reconnection (clean-start = true) or resume a previous session if present (clean-start = false). The default setting is false.

To prevent loss of messages when the remote broker is not available, best practice is to resume existing sessions with a clean-start setting of false.

session-expiry: The time in seconds until the bridge client sessions expire, when the connections are lost or closed. The default setting is 3600 seconds (one hour).
keep-alive: The time in seconds that the broker permits between when a client finishes sending one MQTT packet and starts to send the next. The broker must disconnect a client that does not send a a message or a PINGREQ packet in one and a half times the stated keep alive interval. The default setting is 60 seconds.
- tls:Optional TLS configuration for the bridge connection:
enabled: Defines whether the TLS cryptographic protocol is used to provide an encrypted connection to the bridge. The default setting is true.
keystore: When TLS is enabled, a keystore must be configured.
- path: Location of the P12 or JKS certificates that the keystore uses to store entries.
- password: The password to the associated keystore.
- private-key-password: The password that protects the private key of the keystore.
truststore: Configuration of an optional truststore:
- path: Location of the P12 or JKS certificates that the truststore uses to store entries.
- password: The password to the associated truststore.
cipher-suites: The sets of cryptographic algorithms the TLS protocol uses to create keys and encrypt information. By default, the bridge extension uses the default cipher suites of your system. The default cipher suites that are available on your system vary based on the Java Runtime Environment.
- cipher-suite: Optional setting to explicitly define cipher suites. If desired, you can define specific cipher suites to limit the number of suites that are enabled.
protocols: The versions of the TLS protocol that can be used to secure communication between the bridge extension and the specified remote broker. By default, the bridge extension uses the default TLS protocol version of your JDK (typically, TLSv1.2 and TLSv1.3).
- protocol: Optional setting to explicitly define the TLS protocol version for the bridge clients. If desired, you can define one or more specific TLS versions manually. For example, TLSv1.1, TLSv1.2, and TLSv1.3.

topics

Configuration of one or more topics that map MQTT messages for the bridge clients.
At least one topic must be configured.

topic: The configuration of one MQTT topic.
- filter: The UTF-8 string that the broker uses to filter messages for the bridge clients. The topic filter must conform to the standard syntax of MQTT topic filtering paths.
- mode: Optional setting that defines the method of message transfer. The following options are possible:
  - PUB: In PUB mode, the bridge extension transfers MQTT PUBLISH messages from HiveMQ to the remote broker. The default mode is PUB.
  - SUB: In SUB mode, the bridge extension transfers MQTT PUBLISH messages from the remote broker to HiveMQ.
- exclude: Optional setting that excludes specific topics from bridging. The default setting does not exclude any topics.
- destination: Optional setting that routes the message for a specific topic to a different destination topic. The default setting uses the original topic.

Example HiveMQ Enterprise Bridge Extension configuration options

<hivemq-bridge-extension>
    <!-- A list of bridges, at least one bridge must be configured  -->
    <bridges>
        <bridge>
            <enabled>true</enabled>
            <name>a-unique-bridge-name</name>
            <remote-broker>
                <authentication>
                    <mqtt-simple-authentication>
                        <username>a-username</username>
                        <password>a-user-password</password>
                    </mqtt-simple-authentication>
                </authentication>
                <connection>
                    <static>
                        <host>localhost</host>
                        <port>8000</port>
                    </static>
                    <websocket>
                        <enabled>true</enabled>
                        <server-path>mqtt</server-path>
                    </websocket>
                </connection>
                <mqtt>
                    <clean-start>false</clean-start>
                    <session-expiry>3600</session-expiry>
                    <keep-alive>60</keep-alive>
                </mqtt>
                <tls>
                    <enabled>true</enabled>
                    <keystore>
                        <path>path/to/keystore.jks</path>
                        <password>change-me</password>
                        <private-key-password>change-me-too</private-key-password>
                    </keystore>
                    <truststore>
                        <path>path/to/truststore.jks</path>
                        <password>change-it</password>
                    </truststore>
                    <cipher-suites>
                        <cipher-suite>TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384</cipher-suite>
                        <cipher-suite>TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256</cipher-suite>
                        <cipher-suite>TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256</cipher-suite>
                        <cipher-suite>TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384</cipher-suite>
                        <cipher-suite>TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA</cipher-suite>
                        <cipher-suite>TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA</cipher-suite>
                        <cipher-suite>TLS_RSA_WITH_AES_128_GCM_SHA256</cipher-suite>
                        <cipher-suite>TLS_RSA_WITH_AES_128_CBC_SHA</cipher-suite>
                        <cipher-suite>TLS_RSA_WITH_AES_256_CBC_SHA</cipher-suite>
                    </cipher-suites>
                    <protocols>
                        <protocol>TLSv1.2</protocol>
                    </protocols>
                </tls>
            </remote-broker>
            <topics>
                <topic>
                    <filter>+/+/machine/station</filter>
                    <mode>PUB</mode>
                    <exclude>test/+/machine/station</exclude>
                    <destination>mytest/{#}</destination>
                </topic>
            </topics>
        </bridge>
    </bridges>
</hivemq-bridge-extension>

To ensure proper support of shared subscriptions and consumers, all QoS 2 messages are downgraded to QoS 1 messages in the HiveMQ Enterprise Bridge Extension.

Overlapping Message Configurations in Topic Filters

The HiveMQ Enterprise Bridge Extension applies the first matching topic filter that you define in your configuration. When multiple topic filters are present, the max-qos setting in the first topic filter that matches a topic is applied.

In the following example, when a message with the topic sub/topic is incoming, the first topic filter is applied. max-qos of 1 is applied. The second topic filter is ineffectual.

If a message with the topic my-topic is incoming, the third topic filter is applied. max-qos of 0 is applied.

Example Overlapping SUB Topic Filters (no retained messages preserved)

<topic>
    <filter>sub/#</filter>
    <mode>SUB</mode>
    <max-qos>1</max-qos>
</topic>
<topic>
    <filter>sub/topic3</filter>
    <mode>SUB</mode>
    <max-qos>0</max-qos>
</topic>
<topic>
    <filter>my-topic</filter>
    <mode>SUB</mode>
    <max-qos>0</max-qos>
</topic>

Topic Mapping

Topic mapping filters the messaging flow from the source topics to the destination topics. Each topic can consist of one or more topic levels. Topic levels are separated by a forward slash (topic level separator).

Topic mapping functions are bidirectional:

The topic filter determines which messages the bridge extension processes.
The exclude tag of the topic filter defines a subset of topics from the topic filter that are not forwarded to the destination topic.
The destination tag modifies the selected topic to map to a different topic at the remote broker. This modification is executed before the message is published.

<topic>
    <filter>{incomingFilter}</filter>
    <exclude>{excludeFilter}</exclude>
    <destination>{filterDestination}</destination>
</topic>

Modifications of destination tag based on a topic filter:

Variables can be used inside the destination and exclude tags
Each topic level can be addressed by number, starting with 1
Static text can be added
A prefix and postfix can be used

Topic Filter Configuration Examples

1. Forward to a static remote topic.

Example forward to a static remote topic

<topic>
   <filter>+/status</filter>
   <destination>remote/Data</destination>
</topic>

2. Exclude specific topics.

Example exclusion of a topic

<topic>
   <filter>+/status</filter>
   <exclude>client2/status</exclude>
   <destination>remote/Data</destination>
</topic>

The bridge extension consumes all topics that match the filter and forwards messages other than messages to the excluded topics to the destination topic.

3. Add prefix and postfix to the incoming topics as destination.

<topic>
   <filter>+/status</filter>
   <destination>prefix/{#}/data</destination>
</topic>

The bridge extension consumes all topics that match the filter and uses the full MQTT topic from the client as the destination topic at the remote broker with a prefix and a postfix.

4. Use parts of the topic filter in the destination topic.

<topic>
   <filter>+/+/+/+</filter>
   <destination>prefix/{1-3}/allData</destination>
</topic>

This example uses a four-level incoming filter. The bridge extension forwards all publish topics that match the pattern.
For the destination topic at the remote broker, only the first three parts of the local publish topic are used and a prefix and a postfix are added.

5. Use prefix, postfix, and parts of the topic filter as the destination.

<topic>
   <filter>a/+/+/+</filter>
   <exclude>{1}/client2/{#}</exclude>
   <destination>prefix/{2-3}/allData</destination>
</topic>

This example uses a four-level incoming filter. The bridge extension forwards all publish topics that match the pattern.
For the destination topic at the remote broker, only the second and third parts of the local publish topic are used and a prefix (prefix) and postfix (allData) are added.
Messages that have the string client2 at the second level are not forwarded.

6. Use prefix, postfix, and parts of the topic filter as the destination topic.

<topic>
   <filter>+/+/+/+</filter>
   <destination>prefix/{2}/NEU/{4}/allData</destination>
</topic>

This example uses a four-level incoming filter. The bridge extension forwards all publish topics that match the pattern.
For the destination topic at the remote broker, only the second and fourth level from the incoming topic are used. The third level is replaced with a constant and a prefix is added.

7. Use variables and topic filter parts as the destination topic.

<topic>
    <filter>+/+/+/+</filter>
    <destination>prefix/{2}/${bridge.name}/{4}/allData</destination>
</topic>

This example uses a four-level incoming filter. The bridge extension forwards all publish topics that match the pattern.
For the destination topic at the remote broker, only the second and the fourth level of the incoming topic are used. The third level is replaced with a variable value that contains the bridge name and a prefix is added.
A set of 10 variables can be configured. For example, a variable can be configured to use the mqtt clientId, bridgeName, or userName.

Further examples of topic level addressing:

{2}: Topic filter level 2
{2 - #}: Topic filter level 2 and all following levels
{#}: Whole topic filter
prefix/{#}/postfix: Whole topic filter inside prefix and postfix
{1}/A/{2}/B: Level 1 + "/A" + level 2 + "/B"
{#}/{1-3): Whole topic filter + level 1, 2, and 3

Logging Configuration

The HiveMQ Enterprise Bridge Extension can log key activity between the bridge and the remote broker. It is possible to log the following events:

Bridge client connects to a remote broker
Bridge client disconnects from a remote broker
Bridge client sends a PUBLISH message to a remote broker
Bridge client sends a SUBSCRIBE message to a remote broker
Bridge client receives a message from a remote broker

The <message-log> tag defines the logging behavior for all configured bridges globally.

Table 2. Message Logging Configuration
Configuration	Default Value	Description
`enabled`	false	Defines whether the logging configuration is enabled. Possible settings are `false` and `true`.
`log-level`	-	Defines the type of logging information the extension provides. The following options are possible: `TRACE` `DEBUG` `INFO` `WARN` `ERROR`
`mqtt-packets`	-	Defines which operations of the bridge extension are included in the log. Multiple items can be selected: `CONNECT` `DISCONNECT` `INCOMING-PUBLISH` `SUBSCRIBE` `OUTGOING-PUBLISH`

Example message logging configuration

<hivemq-bridge-extension>
    <message-log>
        <enabled>true | false</enabled>
        <log-level>DEBUG | INFO | TRACE | ERROR | WARN</log-level>
        <mqtt-packets>CONNECT, DISCONNECT, INCOMING-PUBLISH, SUBSCRIBE, OUTGOING-PUBLISH</mqtt-packets>
    </message-log>
    <bridges>
        ...
    </bridges>
</hivemq-bridge-extension>

Bridge Metrics

The Bridge Extension exposes a dashboard of several useful metrics that you can use to monitor behavior:

com.hivemq.bridge-extension.{bridgeName}.total.connect.count
com.hivemq.bridge-extension.{bridgeName}.total.connect-failed.count
com.hivemq.bridge-extension.{bridgeName}.total.disconnect.count
com.hivemq.bridge-extension.{bridgeName}.total.publish.count
com.hivemq.bridge-extension.{bridgeName}.total.publish-failed.count
com.hivemq.bridge-extension.{bridgeName}.total.subscribe.count
com.hivemq.bridge-extension.{bridgeName}.total.subscribe-failed.count

com.hivemq.bridge-extension.{bridgeName}.{clientId}.total.connect.count
com.hivemq.bridge-extension.{bridgeName}.{clientId}.total.connect-failed.count
com.hivemq.bridge-extension.{bridgeName}.{clientId}.total.disconnect.count
com.hivemq.bridge-extension.{bridgeName}.{clientId}.total.publish.count
com.hivemq.bridge-extension.{bridgeName}.{clientId}.total.publish-failed.count
com.hivemq.bridge-extension.{bridgeName}.{clientId}.total.subscribe.count
com.hivemq.bridge-extension.{bridgeName}.{clientId}.total.subscribe-failed.count

The {bridgeName} is replaced with the configured bridge name. The {clientId} is replaced with the MQTT client ID that is used internally for bridging.

Error Handling

During the start of the extension, the bridge configuration is validated.
If parts of the bridge are incorrectly configured, the extension does not start and a DISABLED flag is set in the extension folder.
Setting up the connection to the remote broker is an important part of the configuration.
- If a connection problem occurs during the extension startup or during runtime, the bridge extension attempts to reconnect internally an unlimited number of times.
  The period between connection retries starts at one second and increases exponentially to a maximum of two minutes.
The bridge extension logs errors in the HiveMQ log file.