Shared Subscriptions

Shared subscriptions allow the MQTT broker to balance the load of a single subscription across multiple MQTT clients. HiveMQ Edge supports shared subscriptions for MQTT 3 and MQTT 5 clients. Until the release of MQTT 5, shared subscriptions were a unique feature of HiveMQ. Now, shared subscriptions are a standard feature of MQTT 5.

In a regular MQTT subscription, each client that subscribes to a topic receives a copy of every message that is published on the topic.

In an MQTT shared subscription, each client that shares the same subscription receives messages that are published with the shared topic in an alternating fashion.

Shared subscription is sometimes called client load balancing since the message load of a single topic is distributed across all the clients that subscribe to that topic.

Clients can subscribe to a shared subscription with standard MQTT mechanisms. The topic structure for shared subscriptions is the following:

$share/GROUPID/TOPIC

The shared subscription consists of 3 parts:

  • A static shared subscription identifier (“$share”).

  • A group identifier (share name).

  • The specific topic subscriptions (can include wildcards).

HiveMQ Edge supports a shared subscription syntax that separates the shared subscription segments with a slash ( / ).
Here is an example shared subscriber:

$share/my-shared-subscribers/myhome/groundfloor/+/temperature

Only one subscriber per group identifier receives a particular message. When multiple MQTT clients with the same group identifier subscribe to the same topic, HiveMQ Edge distributes messages on the topic among each of the MQTT clients in an alternating fashion.
Subscriptions that have the same group identifier (share name) but different topics are handled as separate shared subscriptions.

Use Cases

There are many use cases for shared subscriptions which excel most in high-scalability scenarios. Here are some popular shared-subscription use cases:

  • Client load balancing for MQTT clients which cannot handle the load on subscribed topics individually.

  • Worker (backend) applications that ingest MQTT streams and must be able to scale horizontally.

  • Intra-cluster node traffic needs to be relieved by optimizing subscriber node-locality for incoming publishes.

  • QoS 1 and 2 are used for their delivery semantics but ordered topic guarantees are not required.

  • Popular topics that have a higher message rate than other topics on the system create a scalability bottleneck.

Shared Subscriptions Explained

With standard publish / subscribe mechanisms every subscriber gets its own copy of every message which matches the subscribed topic. When using shared subscriptions, each subscription group, which can be conceptually imagined as a virtual client, acts as proxy for multiple real subscribers at once. HiveMQ then selects one subscriber of the group and delivers the message. The following picture demonstrates the principle:

Shared Subscriptions

There can be an arbitrary number of Shared Subscription Groups in a HiveMQ deployment. So for example the following scenario would be possible:

Shared Subscription Groups

In this example, there are two different groups with two subscribing clients in each shared subscription group. Both groups have the same subscription but have different group identifiers. When a publisher sends a message with a matching topic, one (and only one) client of each group receives the message.

It is possible for different clients to have different subscriptions for the same group identifier. In this case, HiveMQ Edge filters by matching subscribers per group and then distributes the message to one of the found clients. While technically possible, this can cause lots of confusion in understanding the message flow in your system. Our recommendation is to use identical client subscriptions per shared subscription group.

Shared Subscription distribution

In HiveMQ Edge single node deployments, the distribution mode of messages for a shared subscription group is round-robin. This guarantees that the load is distributed evenly across all active subscribers in the same shared subscription group.

Shared subscriptions + QoS Levels

HiveMQ Edge automatically downgrades shared subscriptions with QoS 2 to QoS 1.
It is currently not possible to guarantee a QoS 2 in shared subscriptions because assumptions about client state would be required.

To avoid complex situations that are difficult to debug, we highly recommended that all shared subscribers for a group subscribe with the same Quality of Service level.

When members of a shared subscription group subscribe with different QoS levels, the shared-subscription algorithm honors the QoS level of each individual subscribing client.

Best practice is to always use the same QoS level for a all members of a shared subscription group.