2.3.x to 3.0.x Migration Guide

This is a major HiveMQ upgrade. HiveMQ 3.0 comes with many new features and improvements. HiveMQ 3.0 is not compatible with HiveMQ 2.x and is not a drop in replacement.

Manual steps are required for upgrading the configuration file of HiveMQ 2.3 to HiveMQ 3.0.

HiveMQ Upgrade

Create a backup of the entire HiveMQ 2 installation folder
Install a recent Java 7 version (or newer)
Install HiveMQ 3 as described in the HiveMQ Installation Guide
Migrate the configuration file contents of your old HiveMQ installation
Delete all contents in the data folder. This will delete all persistent data of HiveMQ 2.x.

Configuration File Upgrade

HiveMQ 3.0 comes with a new, single, XML based configuration file. HiveMQ 2.x uses two plain properties files, so the migration from HiveMQ 2.3 needs to be done manually.

Removed Configurations

The following table shows all removed configurations which are not available in HiveMQ 3 anymore. Configurations ending with a star (*) mean, that all configurations which start with that name were removed.

Table 1. Removed HiveMQ 2.3 configurations
Properties name	Deprecation reason	Alternative
plugin.performanceMonitoring	Plugin monitoring is done automatically by HiveMQ 3, so this configuration is not necessary anymore.	None
graphite.*	Graphite integration is not in the HiveMQ core anymore.	A more powerful Graphite plugin is available for free.
measureResourceConsumption.*	Resource consumption logfile reporting was too inaccurate for real monitoring and was too resource intensive.	The usage of a dedicated monitoring tool like collectl, collectd, or any other monitoring tool of your choice is recommended
mqtt.lwt.onEveryDisconnect	This configuration wasn’t very popular and caused a lot of confusion.	None
plugin.LowLevelCallbacksEnabled	In HiveMQ 3 low level callbacks are always enabled, without performance loss.	None
ssl.hybridMode	HiveMQ 3 allows multiple listeners, so this setting isn’t needed anymore.	Use different listeners simultaneously.
statistics.*	All HiveMQ statistics and metrics are enabled by default and are available instantly.	None, statistics are always enabled
sys.*	The $SYS Topics were removed from the core and are now available as plugin, delivered with HiveMQ.	Use the SYS-Topic plugin (also delivered with HiveMQ)

Upgrading Configurations

HiveMQ 3 uses a XML based file format, so manual migration of configurations is necessary. In case you’re using a default HiveMQ configuration, no configuration changes are needed, just use the HiveMQ 3 default config.xml.

HiveMQ 3 comes with various popular example configurations which are located in the conf/examples folder.

The following instructions show how to upgrade manually. It’s not necessary to follow all steps, you can pick the steps you need for your concrete configuration.

Listener Configuration

HiveMQ 3 uses a new listener concept, which allows to add as many listeners to different network interfaces and ports as needed. This also deprecates the HiveMQ 2 Hybrid Mode.

The following listener types are available:

TCP Listener
TLS-TCP Listener
Websocket Listener
TLS Websocket Listener

The following guide shows how to use the new listener concept: Listener Documentation

MQTT Configuration

Most of the MQTT configurations are now organized under the mqtt XML element.

The following table shows the equivalent of the old properties to the new XML elements.

Table 2. MQTT Config Migration
Old properties name	New XML element	Migration notes
global.retryInterval	<retry-interval>	None
global.maxClientIdLength	<max-client-id-length>	None
messages.maxInflight	None	Max In-flight window is always at most 1 per topic subscription per client in order to give Ordered Topic guarantees.
messages.maxQueued	<max-queued-messages>	None

Throttling Configuration

All throttling configurations are now organized under the throttling XML element.

The following table shows the equivalent of the old properties to the new XML elements.

HiveMQ 3 does not support reloading of throttling configuration via the XML file at runtime. You can still use custom plugins for runtime throttling.

Table 3. Throttling Config Migration
Old properties name	New XML element	Migration notes
global.maxConnections	<max-connections>	None
global.maxMessageSize	<max-message-size>	None
global.outgoingLimit	<outgoing-limit>	The old value was in KB/s, the new value is in B/s.
global.ingoingLimit	<incoming-limit>	The old value was in KB/s, the new value is in B/s.

Plugin System Upgrade

The plugin system significantly improved in HiveMQ 3.0. The goal with HiveMQ 3 was to be as backwards compatible as possible while providing the foundation for even better plugin integration for the future. Most of the plugin services are now cluster-aware, which means information from the cluster can be gathered at runtime by plugins.

New Features

The HiveMQ 3 plugin systems has a lot of new features and improvements, the most noteworthy are the following:

A PluginExecutorService, managed by HiveMQ, was added, so plugins don’t need to bring their own Executors for asynchronous processing
A completely new RESTService is now included in HiveMQ. This allows to hook in your own Java Servlets or JAX-RS resources for creating HTTP APIs inside of HiveMQ.
Most of the configuration can be changed at runtime, thanks to the new Configuration Services.
Additional informal annotations (like @NotNull,@Nullable, @Threadsafe) are now available.
Annotations for creating automated Metrics for your plugins are now available (@Metered, @Counted, @ExceptionMetered).

Removed Functionality

Removed MqttMessageEstimationUtil
Removed HiveMQStatistics (these got replaced by the more powerful HiveMQMetrics)
Removed OnStatisticsUpdate Callback (the HiveMQ statistics are now up to date all the time)
Removed Apache Commons Configuration. Plugins are now responsible for providing their information themselves.
PathUtils was deprecated, the SystemInformation interface can be injected instead for retrieving relevant information.

Upgrading your plugins

In order to upgrade your plugin, execute the following steps:

Adapt your pom.xml file

Change your HiveMQ SPI version to 3.0.0 in your pom.xml file:

<dependency>
    <groupId>com.hivemq</groupId>
    <artifactId>hivemq-spi</artifactId>
    <version>3.0.0</version>
</dependency>

You also need to upgrade your HiveMQ Maven plugin to the newest version in order to develop against HiveMQ 3.0:

<profile>
    <id>RunWithHiveMQ</id>
    <build>
        <plugins>
            <plugin>
                <groupId>com.hivemq</groupId>
                <artifactId>hivemq-maven-plugin</artifactId>
                <version>3.0.1</version>
                <executions>
                    <execution>
                        <id>hivemq</id>
                        <phase>package</phase>
                        <goals>
                            <goal>hivemq</goal>
                        </goals>
                        <configuration>
                            <hiveMQDir>INSERT_HERE</hiveMQDir>
                            <debugMode>SERVER</debugMode>
                        </configuration>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>
</profile>

Change the HiveMQ namespace

The namespace of all SPI classes was changed from com.dcsquare.hivemq to com.hivemq.

An easy way to fix this is to use your favourite text editor or IDE and bulk replace com.dcsquare. with com..

Upgrading the META-INF/services file

Rename the com.dcsquare.hivemq.spi.HiveMQPluginModule file to com.hivemq.spi.HiveMQPluginModule.

Upgrading MQTT Messages

The MQTT message classes received a general clean-up to represent the MQTT specification better and to avoid misunderstandings.

The following changes may affect your plugins:

All messages: duplicateDelivery, retain and QoS flags are now only supported for PUBLISH messages.
CONNECT: the getPassword() method now returns a byte[] instead of a String to conform the MQTT specification. For convenience there’s now a getPasswordAsUTF8String().
CONNECT: The getWillMessage() and setWillMessage methods now also return/take a byte[] instead of a String to conform the MQTT specification.

Upgrade Subscription Store API

The Subscription Store now has methods for receiving cluster wide state of subscriptions. Methods with the old semantics (get subscription data from local node) are still available, with a slightly different name.

The Subscription Stores main API change is, that methods now return a ListenableFuture instead of the result directly, so plugin developers can decide when to wait for the results or if a callback-based approach should be used.

The following table shows the new name of the 'old' methods:

Table 4. Subscription Store API Migration
Old method	New Method
getSubscriptions()	getLocalSubscriptions()
getSubscribers(String topic)	getLocalSubscribers(String topic)
getTopics(String clientID)	getLocalTopics(String clientID)

Upgrade ClientService API

The ClientService APIs are now capable of collecting cluster-wide data. Methods with the old semantics (get data only from the current HiveMQ node) are still available with another name.

The ClientServices main API change is, that methods now return a ListenableFuture instead of the result directly, so plugin developers can decide when to wait for the results or if a callback-based approach should be used.

The following table shows the new name of the 'old' methods:

Table 5. ClientService API Migration
Old method	New Method
getConnectedClients()	getLocalConnectedClients()
getDisconnectedClients()	getLocalDisconnectedClients()
isClientConnected(String clientId)	isClientConnectedLocal(String clientId)
getClientDataForClientId(String clientId)	getLocalClientDataForClientId(String clientId)

Upgrading Authorization and Authentication

While HiveMQ 2.x only allowed blacklists, HiveMQ 3 now allows a white and black list authorization approach.

Each OnAuthorizationCallback needs to override the getDefaultBehaviour() method.

MqttTopicPermission has been improved and can be used to allow or deny a specific actions on a topic.

In order to upgrade the MqttTopicPermissions to the new API with semantics from HiveMQ 2, use ALLOW as Permission Type.

The following additional authentication and authorization changes are included in HiveMQ 3:

RestrictionsAfterLoginCallback is now an asynchronous callback, not a synchronous one.
ClientCredentialsData now offers a getPasswordBytes() method which allows to get the raw byte array of a password instead of a UTF-8 representation.
Added an additional SslClientCertificate wrapper for client certificates with many utility methods for dealing with X.509 client certificates. It’s also possible to directly obtain the java.security.cert.Certificate object via this wrapper.
ClientData now also allows to get the IP Address of a client via getInetAddress().

Executor Services

HiveMQ 3 offers a HiveMQ-managed PluginExecutorService for all asynchronous needs. All plugins share this managed executor service.

In the past some HiveMQ plugin developers used custom CachedThreadPools which drained all system resources and HiveMQ got unresponsive due to excessive thread spawning in the plugins. To avoid this problem the PluginExecutorService was introduced in HiveMQ 3.0.

Upgrading is easy: Remove all your executor services and inject PluginExecutorService.

The PluginExecutorService can be used to schedule your Runnables and returns ListenableFutures, so you can use callback-based, non-blocking programming paradigms.

REST Service

The old experimental REST Service was removed and a new, more sophisticated REST Service was introduced. The Plugin Developer Guide shows how to use the new REST Service.

Configurations

HiveMQ 3 does not support the old Apache Commons Configuration approach anymore. In the past this allowed to break classloader isolation since properties were shared between plugins.

HiveMQ 3 requires plugin developers to bring their own configuration mechanism.

The following configuration libraries are known to work with HiveMQ:

If you want to use (reloadable) standard Java Properties files for configuration, you can take a look at this example plugin: Graphite Example Plugin.