HiveMQ Enterprise Security Extension (ESE) Start Up Guide

Installation

If you do not provide a valid license, HiveMQ uses a free evaluation license automatically. The evaluation license for the HiveMQ Enterprise Security Extension is valid for 5 hours. For more license information or to request an extended evaluation period, contact HiveMQ sales.

  1. Place your HiveMQ Enterprise Security Extension license file (.elic) in the license folder of your HiveMQ installation. (Skip this step if you are using a trial version of the extension. For information on evaluation limits, see HiveMQ Marketplace.)

  2. Since HiveMQ 4.4, the HiveMQ Enterprise Extensions come preinstalled in your HiveMQ release bundle and disabled.
    To enable the HiveMQ Enterprise Security Extension, locate the hivemq-enterprise-security-extension folder in the extensions directory of your HiveMQ instatllation and remove the DISABLED file.

  3. To configure the extension, open the hivemq-enterprise-security-extension/conf subfolder of the extension and configure the enterprise-security-extension.xml file as desired.
    The enterprise-security-extension.xsd file outlines the schema and elements that can be used in the XML configuration. For more information, see Configuration.

  4. To verify that the extension is installed and configured properly, check your HiveMQ log files.

For HiveMQ to recognize and start the extension, the enterprise-security-extension.xml must be located in the hivemq-enterprise-security-extension folder. 
└─ <HiveMQ folder>
    ├─ bin
    ├─ config
    ├─ data
    ├─ extensions
    │   ├─ hivemq-enterprise-security-extension
    │   └─ ...
    ├─ license
    ├─ log
    └─ ...
To secure an entire HiveMQ cluster, you must install the HiveMQ Enterprise Security Extension on every HiveMQ broker node in the cluster.
If you want to change the configuration of the ESE, you can add a DISABLED file to the ESE folder to disable the extension. To enable ESE once more, remove the DISABLED file that you added.

Configuration

The Enterprise Security Extension needs to be configured to use your particular authentication and authorization sources and meet your individual requirements. Enter your configuration preferences in the enterprise-security-extension.xml file that is located in the hivemq-enterprise-security-extension/conf folder.

The following snippet shows the overall structure of the configuration file:

Configuration file structure
<enterprise-security-extension
        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xsi:noNamespaceSchemaLocation="enterprise-security-extension.xsd"
        version="1">

    <access-log>
        <!-- optional access log configuration -->
    </access-log>
    <realms>
        <!-- add one or more realm configurations -->
    </realms>
    <pipelines>
        <!-- add one or more pipeline configurations -->
    </pipelines>
</enterprise-security-extension>

The root tag in the configuration file is called <enterprise-security-extension>. This tag encloses the configurations for your access log, realms, and pipelines. For more information, refer to the individual sections.

To facilitate configuration and validation, an enterprise-security-extension.xsd schema file is included next to the configuration file.

All HiveMQ Enterprise Extensions support the use of environment variables. The ${ENV:VARIABLE_NAME} pattern allows you to add placeholders to your configuration xml file that are replaced with the value of an environment variable at runtime. Environment variables cannot be hot reloaded. You must set your environment variables before HiveMQ startup. For more information, see Environment Variables.

The following snippet shows the minimal configuration example:

Configuration file example
<?xml version="1.0" encoding="UTF-8" ?>
<enterprise-security-extension
        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xsi:noNamespaceSchemaLocation="enterprise-security-extension.xsd"
        version="1">
    <realms>
        <!-- a postgresql db-->
        <sql-realm>
            <name>postgres-backend</name>
            <enabled>true</enabled>
            <configuration>
                <db-type>POSTGRES</db-type>
                <db-name>hivemq</db-name>
                <db-host>hostname</db-host>
                <db-port>5432</db-port>
                <db-username>hivemq</db-username>
                <db-password>password</db-password>
            </configuration>
        </sql-realm>
    </realms>
    <pipelines>
        <!-- secure access to the mqtt broker -->
        <listener-pipeline listener="ALL">
            <!-- authenticate over a sql db -->
            <sql-authentication-manager>
                <realm>postgres-backend</realm>
            </sql-authentication-manager>
            <!-- authorize over a sql db -->
            <sql-authorization-manager>
                <realm>postgres-backend</realm>
                <use-authorization-key>false</use-authorization-key>
                <use-authorization-role-key>true</use-authorization-role-key>
            </sql-authorization-manager>
        </listener-pipeline>
        <!-- secure access to the control center -->
        <control-center-pipeline>
            <!-- authenticate over a sql db -->
            <sql-authentication-manager>
                <realm>postgres-backend</realm>
            </sql-authentication-manager>
            <!-- authorize over a sql db -->
            <sql-authorization-manager>
                <realm>postgres-backend</realm>
                <use-authorization-key>true</use-authorization-key>
                <use-authorization-role-key>true</use-authorization-role-key>
            </sql-authorization-manager>
        </control-center-pipeline>
    </pipelines>
</enterprise-security-extension>

Extension Priority

HiveMQ stores extension metadata in the himemq-extension.xml file of each extension. This information includes optional extension priority settings for the extension.

The priority setting in the hivemq-extension.xml file determines the order in which HiveMQ executes extension callbacks. The extension with the lowest priority executes last. Null (0) is the lowest possible priority setting. By default, the priority setting of the Enterprise Security Extension is 1000.

The start-priority setting determines the order in which HiveMQ starts the extension. HiveMQ starts the extension with the lowest priority last. Null (0) is the lowest possible start-priority setting. The default start-priority for the Enterprise Security Extension is 1000.

The start-priority in the hivemq-extension.xml file is only used if it is not possible to determine the temporal resolution between two extension start events. This usually happens at the start of HiveMQ if several extensions are already present in the extensions folder, or during a simultaneous hot reload of multiple extensions. Otherwise, the first extension that HiveMQ detects loads first. HiveMQ does not guarantee the order in which extensions stop.
Default hivemq-extension.xml file
<?xml version="1.0" encoding="UTF-8" ?>
<hivemq-extension>
    <id>hivemq-enterprise-security-extension</id>
    <version>4.9.0</version>
    <name>HiveMQ Enterprise Security Extension</name>
    <author>HiveMQ</author>
    <priority>1000</priority>
    <start-priority>1000</start-priority>
</hivemq-extension>
When you use custom authorizers or authenticators with the Enterprise Security Extension, you can configure the priority settings in the hivemq-extension.xml file as needed to fulfil your particular use case.


Start Up Guide with SQL Databases

This start up guide uses the Postgres docker image. You can also download the Postgres application from the official download page and follow the installation instructions according to your operating system.

  1. To get your PostgreSQL database, open the command line and pull the official docker image for Postgres:

    docker pull postgres

  2. Start the docker container:

    docker run -d --name hivemq-ese-database -p 5432:5432 postgres

  3. Open your favourite database GUI. For example, pgAdmin.

  4. Connect the docker container.

    1. Click Add New Server.

      Connect postgresql docker container to pgAdmin.

    2. Insert a server name that identifies your server, e.g. hivemq-ese-db.

      Insert postgresql docker server name.

    3. Insert the connection information to the server.

      Insert postgresql docker connection information.

  5. To create a new database:

    1. Use right click > create > Database…​

      Create a new database in pgAdmin.

    2. Insert the name of your database. For example, hivemq-ese-db.

      Insert the name of the new database in pgAdmin.

  6. Create your database tables with the predefined sql script:

    1. Open the Query Editor.

      Click the button with the flash to open the Query Editor.

      Open the Query Editor in pgAdmin.

    2. Open the file search assistant

      Click the button with the folder to open the file search assistant.

      Open File Search Assistant in pgAdmin.

    3. Navigate to the sql subfolder of your HiveMQ Enterprise Security Extension folder.

    4. Select the postgresql_create.sql file.

      Select the postgresql_create.sql file.

      1. Select the postgresql_create.sql file.

      2. Click the select button.

      Alternatively, you can copy and paste following sql script to the query editor:

      SQL Script to create PostgreSQL database tables
      create table users
(
  id serial not null
    constraint users_pkey
    primary key,
  username text not null
    constraint users_username_unique unique,
  password text,
  password_iterations integer not null,
  password_salt text not null,
  algorithm text not null
);

create unique index users_id_uindex
  on users (id);

create unique index users_username_uindex
  on users (username);

comment on column users.password is 'Base64 encoded raw byte array';

comment on column users.password_salt is 'Base64 encoded raw byte array';

create table roles
(
  id serial not null
    constraint roles_pkey
    primary key,
  name text not null
    constraint roles_name_unique unique,
  description text
);

create unique index roles_id_uindex
  on roles (id);

create unique index roles_name_uindex
  on roles (name);

create table user_roles
(
  user_id integer not null
    constraint user_roles_users_id_fk
    references users,
  role_id integer not null
    constraint user_roles_roles_id_fk
    references roles,
  constraint user_roles_user_role_pk
  primary key (user_id, role_id)
);

create table permissions
(
  id serial not null
    constraint permissions_pkey
    primary key,
  topic text not null,
  publish_allowed boolean default false not null,
  subscribe_allowed boolean default false not null,
  qos_0_allowed boolean default false not null,
  qos_1_allowed boolean default false not null,
  qos_2_allowed boolean default false not null,
  retained_msgs_allowed boolean default false not null,
  shared_sub_allowed boolean default false not null,
  shared_group text
);

create index permissions_topic_index
  on permissions (topic);

comment on table permissions is 'All permissions are whitelist permissions';

create table role_permissions
(
  role integer not null
    constraint role_permissions_roles_id_fk
    references roles,
  permission integer not null
    constraint role_permissions_permissions_id_fk
    references permissions,
  constraint role_permissions_role_permission_pk
  primary key (role, permission)
);

create table user_permissions
(
  user_id integer not null
    constraint user_permissions_users_id_fk
    references users,
  permission integer not null
    constraint user_permissions_permissions_id_fk
    references permissions,
  constraint user_permissions_user_permission_pk
  primary key (user_id, permission)
);

    5. Execute the sql statements. In pgAdmin, you should see the created tables in the table section of your side navigation.

      Execute the sql statement in pgAdmin.

      Created Tables result in the side navigation.

  7. Download the Postgres JDBC driver jar file and move it into the drivers/jdbc subfolder of your ESE setup.

    └─ <HiveMQ folder>
    ├─ bin
    ├─ config
    ├─ data
    ├─ extensions
    │   ├─ hivemq-enterprise-security-extension
    │   │   ├─ drivers
    │   │   │   ├─ jdbc
    │   │   │   │   ├─ postgresql-<version>.jar
    │   │   │   ├─ ...
    │   │   ├─ ...
    │   └─ ...
    ├─ license
    ├─ log
    └─ ...

  8. Fill data into the prepared database.

    For your convenience, we provide the ESE Helper Tool to help you insert username and password values for your MQTT clients. Use the ESE Helper Tool to create SQL INSERT statements that you can execute directly in your database.

    For example, to create an MQTT client with the username username, the password password and the algorithm SHA-512 and 100 iterations.

    The hivemq-ese-helper-jar file is located in the extensions | hivemq-enterprise-security-extension | helper | jvm folder of your HiveMQ installation. For more information, see ESE Helper folder.

    Command-line command to create an sql insert statement with the ESE Helper Tool
    java -jar hivemq-ese-helper.jar db create-insert-statement username password

    The output is similar to the following example:

    Output of the ESE Helper Tool
    insert into users (username, password, password_iterations, password_salt, algorithm) values ('username', 'yHfGSuyVwq5/hInsBO0T5Xq1Dl25l5iigYHGailLU/5RqnroFW0YKhj4Hr85hnpbqUEcaUZ9bxDOLetP/Vknhg==', 100, 'jkxQqLohaFB5cilFqprocQ==', 'SHA512');
    Because the ESE Helper Tool creates the salt, that is prepended to the password randomly, the actual output varies.

    To get started quickly, execute the following sql script in the query editor. This script fills your database with users, roles and permissions for following three clients:

    • Backendservice: Allows subscription to the topic/+/status topic filter and reception of all messages that match that topic filter. Publication of messages is not permitted.

      Enables connection with the username = backendservice and password = backendpassword.

    • Frontendclient: Allows publication of messages to the topic/${mqtt-clientid}/status topic. Subscriptions to the topic/${mqtt-clientid}/status topic is not permitted.

      Enables connection with the username = frontendclient and password = clientpassword.

    • Superuser: Allows everything. This user has no restrictions.

      Enables connection with the username = superuser and password = supersecurepassword.

      This is just an example. Create more meaningful roles and permissions before you decide to deploy your HiveMQ with HiveMQ ESE to Production.
    SQL script to fill the PostgreSQL database with values
    insert into public.users
  (username, password, password_iterations, password_salt, algorithm)
  values
    ('backendservice', 'wtUo2dri+ttHGHRpngg9uG21piWLiKSX7IaNSnU/BfN9pt+ZOLQByG/3JlPPQ7t/pl8S3tjR2+Um/DPBdAQULg==', 100, 'Nv6NU9XY7tvHdSGaKmNTOw==', 'SHA512'),
    ('frontendclient', 'ZHg/rNJel1BHOYMEvc40ekCRUE5vVLcsPF6mk9GPDcdEmX3stm50MplaqjGb8Lxhy6rNFQZSQRSbOxmFZ8ps1Q==', 100, 'JhpW27QU9WfIaG6FJT5MkQ==', 'SHA512'),
    ('superuser', 'nOgr9xVnkt51Lr68KS/rAKm/LqxAt8oEki7vCerRod3qDbyMFfDBGT8obnkw+AGygxCQDWdaA2sQnXXoAbVK6Q==', 100, 'wxw+3diCV4bWXQHb6LLniA==', 'SHA512');

insert into public.permissions
  (id, topic, publish_allowed, subscribe_allowed, qos_0_allowed, qos_1_allowed, qos_2_allowed, retained_msgs_allowed, shared_sub_allowed, shared_group)
  values
    (1, 'topic/+/status', false, true, true, true, true, false, false, ''),
    (2, 'topic/${mqtt-clientid}/status', true, false, true, true, true, true, false, ''),
    (3, '#', true, true, true, true, true, true, true, '');

insert into public.roles
  (id, name, description)
  values
    (1, 'backendservice', 'only allowed to subscribe to topics'),
    (2, 'frontendclients', 'only allowed to publish to topics'),
    (3, 'superuser', 'is allowed to do everything');

insert into public.user_roles
  (user_id, role_id)
  values
    (1, 1),
    (2, 2),
    (3, 3);

insert into public.role_permissions
  (role, permission)
  values
    (1, 1),
    (2, 2),
    (3, 3);

  9. You configure your HiveMQ Enterprise Security Extension with the enterprise-security-extension.xml file that is located in the ESE extension folder.

    Configuration file example for use of a PostgreSQL database
    <?xml version="1.0" encoding="UTF-8" ?>
<enterprise-security-extension
        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xsi:noNamespaceSchemaLocation="enterprise-security-extension.xsd"
        version="1">
    <realms>
        <!-- a postgresql db-->
        <sql-realm>
            <name>postgres-backend</name> (1)
            <enabled>true</enabled>
            <configuration>
                <db-type>POSTGRES</db-type>
                <db-name>hivemq</db-name> (2)
                <db-host>hostname</db-host> (3)
                <db-port>5432</db-port>
                <db-username>hivemq</db-username> (4)
                <db-password>password</db-password> (5)
            </configuration>
        </sql-realm>
    </realms>
    <pipelines>
        <listener-pipeline listener="ALL"> (6)
            <!-- authenticate over a sql db -->
            <sql-authentication-manager>
                <realm>postgres-backend</realm> (7)
            </sql-authentication-manager>
            <!-- authorize over a sql db -->
            <sql-authorization-manager>
                <realm>postgres-backend</realm>
                <use-authorization-key>false</use-authorization-key>
                <use-authorization-role-key>true</use-authorization-role-key>
            </sql-authorization-manager>
        </listener-pipeline>
    </pipelines>
</enterprise-security-extension>
1 Give the PostgreSQL realm a unique name that can be used as a reference.
2 Replace 'hivemq' with the name of the database that you use.
3 Replace 'hostname' with the name or the IP address of the host on which the PostgreSQL database runs.
4 Replace 'hivemq' with your database username.
5 Replace 'password' with your database password.
6 Create a listener pipeline and assign it to the listeners that want to have active on the pipeline.
7 In the authentication manager and the authorization manager, use the unique realm name that you assigned to the PostgreSQL realm to reference the realm that is used.

That completes your basic setup. Now, you can start HiveMQ with the HiveMQ Enterprise Security Extension.

To get started with ESE on AWS using AWS Aurora, have a look at our detailed and informative Blogpost.