Concepts and Components of the HiveMQ Enterprise Security Extension

Concept of the HiveMQ Enterprise Security Extension.

Realms

Realms are abstract representations of external sources of authentication or authorization information. The ESE uses realms to authenticate connecting clients and authorize the future actions of the connecting clients. Specific types of realms correspond to the different types of authentication and authorization sources that are available. For example, the source behind an SQL realm is an SQL database.

Each type of realm has a specific configuration.

This example realm-configuration structure uses an sql-realm (1):

Example SQL realm configuration structure
<enterprise-security-extension>
    ...

    <realms>
        <sql-realm> (1)
            <name>realm-name</name>
            <enabled>true</enabled>
            <configuration>
                <!-- enter specific configuration options, for example, database url and credentials -->
            </configuration>
        </sql-realm>

        <!-- configure additional realms -->
    </realms>

    ...
</enterprise-security-extension>

In general, realms need to establish connections to external services. You must ensure that these external services are available for connection establishment when you start the extension.

This example realm-configuration structure uses a jwt-realm (1):

Example JWT realm configuration structure
<jwt-realm>
    <name>jwt-backend</name> <!-- mandatory -->
    <enabled>true</enabled> <!-- mandatory -->
    <configuration>
        <jwks-endpoint>https://some.uri</jwks-endpoint> <!-- mandatory -->
        <jwks-cache-duration>1800</jwks-cache-duration> <!-- optional, default is 1800 -->
        <ssl-trust-store password="pass" type="jks">/file/path</ssl-trust-store>  <!-- optional -->
        <introspection-endpoint>https://another.uri</introspection-endpoint>  <!-- optional -->
        <simple-auth>  <!-- optional -->
            <user-name>clientName</user-name>
            <password>client-secret</password>
        </simple-auth>
    </configuration>
</jwt-realm>

Realms are configured in the <realms> tag of the ESE configuration file. Each realm needs a unique name so it can be accurately referenced in your pipelines. Since establishing a realm connection can be an expensive operation, all realms can be disabled for testing purposes.

Available Realm Types

Table 1. Available Realm Types

Realm

MQTT Client

Control-Center

REST API

Authentication

Authorization

Authentication

Authorization

Authentication

Authorization

SQL Realm

JWT Realm

LDAP Realm

File Realm

OAuth Realm

OIDC Realm

Pipelines in Enterprise Security Extension

When a client tries to connect to HiveMQ, the client runs through a pipeline of the Enterprise Security Extension before the connection is accepted. This pipeline contains the individual steps that are required to authenticate and authorize the client. It is possible to configure multiple pipelines to authenticate or authorize different types of clients with different mechanisms.

Pipelines are highly configurable. However, a pipeline usually has four stages:

  • Authentication preprocessing

  • Authentication

  • Authorization preprocessing

  • Authorization

MQTT client CONNECT sequence

Authentication Preprocessing

Preprocessing is done with ESE variables. When a client connects at the start of a pipeline, only the MQTT variables (mqtt-clientid, mqtt-username, mqtt-password) are filled with the values the client provides in its CONNECT packet. The purpose of authentication preprocessing is to fill the authentication variables that the next stage of the pipeline requires to actually authenticate the client. The way variables are preprocessed and transformed is explained in the Preprocessors section.

Authentication preprocessing is optional. By default, the MQTT username and password are filled in as the authentication key and authentication secret.

Authentication

Authentication is performed by an authentication manager. For example, an SQL authentication manager that uses a database realm. The authentication manager uses the authentication variables to verify whether a client is authenticated to connect to HiveMQ. Additionally, the authentication manager can provide values for some authorization variables. For example, provide a role key that can be used for authorization.

Authorization Preprocessing

Preprocessing for authorization is similar to authentication preprocessing. The only difference is that authorization preprocessing fills or transforms the authorization variables.

Authorization

Authorization is performed by an authorization manager. For example, an SQL authorization manager that uses a database realm. The authorization manager uses authorization variables to query permissions for a client. These permissions define which operations the client is allowed to perform.

Example Pipeline configuration structure
<enterprise-security-extension>
    ...

    <pipelines>
        <listener-pipeline listener="ALL">
            <authentication-preprocessors>
                ...
            </authentication-preprocessors>

            <sql-authentication-manager>
                ...
            </sql-authentication-manager>

            <authorization-preprocessors>
                ...
            </authorization-preprocessors>

            <sql-authorization-manager>
                ...
            </sql-authorization-manager>
        </listener-pipeline>

        <!-- configurations for additional pipelines -->
    </pipelines>

</enterprise-security-extension>

Listener Pipelines

When a client connects to HiveMQ, the Enterprise Security Extension selects the appropriate pipeline for the client.

It is possible to configure multiple pipelines that each map to different HiveMQ listeners. This enables you to authenticate and authorize backend applications and devices that connect over the internet differently.

Example Listener pipelines configuration
<enterprise-security-extension>
    ...

    <pipelines>
        <listener-pipeline listener="listener-internet"> (1)
            ...
        </listener-pipeline>

        <listener-pipeline listener="listener-backend">
            ...
        </listener-pipeline>

        <listener-pipeline listener="ALL"> (2)
            ...
        </listener-pipeline>
    </pipelines>

</enterprise-security-extension>
1 The listener attribute takes the name of the listener that is defined in the HiveMQ config.xml.
2 ALL is a keyword for a special pipeline. This pipeline is used if a client connects to a listener whose name does not match another pipeline.

If a client connects to a listener whose name does not match any pipeline and no ALL pipeline is configured, the ESE rejects the connection attempt.

Pipeline selection process

ESE Variables

The Enterprise Security Extension assigns a set of ESE variables to every connecting client. To give you maximum authentication and authorization flexibility, you can configure how the ESE variables are manipulated in the preprocessing stages of a pipeline.

There are two data types available for ESE variables:

  • String: ESE variables of this data type store a sequence of characters of arbitrary length or null

  • Byte: ESE variables of this data type store a sequence of bytes of arbitrary length or null

Additionally, ESE variables can be constant. Constant ESE variables contain read-only values that the ESE runtime system provides. Specific preprocessors can only use specific types of ESE variables.

Some ESE variables have predefined meanings, others can be used as general purpose variables. ESE variables are grouped into four categories:

MQTT Variables

MQTT variables are initialized by the ESE runtime system. The value of an MQTT variable is derived directly from the MQTT CONNECT packet of the connecting client.

Table 2. MQTT variables
Name Data type Description

mqtt-clientid

Constant string

Set to the client ID of the connecting client.

mqtt-username

Constant string

Set to the username field of the connecting client.

mqtt-password

Constant byte

Set to the password field of the connecting client.

General Purpose Variables

General purpose variables do not have a predefined meaning. These variables can be used during pipeline processing as desired. At the start of a pipeline, the initial state of a general purpose variable is always null.

Table 3. General purpose variables
Pattern Data type

byte-[a-z0-9\-]+

Byte

string-[a-z0-9\-]+

String

Authentication Variables

Authentication variables are used by the authentication managers to verify the identity of a connecting client. These variables are usually manipulated in the authentication-preprocessing step of a pipeline.

Table 4. Authentication variables
Name Data type Description

authentication-key

String

The authentication-key is generally used by the authentication manager to identify the connecting client.

authentication-byte-secret

Byte

The authentication-byte-secret is generally used by the authentication manager to ensure the claims of the connecting client.

Authorization Variables

Authorization variables are used by the authorization managers to specify access rights to MQTT topics. Based on the state of the authorization variables, the authorization manager assigns permissions to a connecting client. Authorization variables are usually manipulated in the authorization-preprocessing step of a pipeline.

Table 5. Authorization variables
Name Data type Description

authorization-key

String

The authorization-key is generally used by the authorization manager to assign client-specific permissions.

authorization-role-key

String

The authorization-role-key is generally used by the authentication manager to assign permissions that are inherent to a conceptional group of clients.

Permission Placeholders

Permission placeholders facilitate the creation of permissions for groups of clients that require similar, but client-specific permissions. The use of permission placeholders can eliminate the need to create individual permissions for each client.

The placeholders can be used inside the topic filter and shared group of a permission. All ESE variables of the string data type can be used as placeholders. At the end of the authorization step, the placeholders are dynamically replaced with the values of the referenced ESE variables.

To create a permission with a placeholder, insert the name of the ESE variable framed with ${ and } at the desired position in the permission.

Example for permission placeholders:

  • The authorization manager grants subscription to the topic filter topic/${mqtt-clientid}/${string-1}/subtopic.

  • If the value of mqtt-clientid is my-special-client and string-1 is last, then the connecting client is only allowed to subscribe with the topic filter topic/my-special-client/last/subtopic.

Only variables of the string data type can be substituted. Byte variables can not be replaced.

Preprocessors

In a pipeline, you can add preprocessing steps before authentication and before authorization. The preprocessing is done through the configuration of one or more preprocessors. Preprocessors are lightweight pipeline steps that work with the current state of the ESE variables and have no external dependencies.

Available Preprocessor Types

Table 6. Available Preprocessor Types

Preprocessor

MQTT Client

Control-Center

REST API

Authentication

Authorization

Authentication

Authorization

Authentication

Authorization

Set

Copy

Plain (deprecated)

Regex

Concatenation

Split

Letter Case

X.509

JWT

Logging

Session Attribute

Connection Attribute

HTTP Headers

Set Preprocessor

A set preprocessor offers a way to set the state of ESE variables to a predefined value of your choice. The Enterprise Security Extension offers two types of set preprocessors:

Set String Preprocessor

Example set string preprocessor configuration
<set-string-preprocessor>
    <variable>string-variable</variable>
    <value>value</value>
</set-string-preprocessor>
Table 7. Set string preprocessor parameters
Parameter Type Description

variable

ESE variable

Defines to which modifiable string ESE variable the configured value is set.

value

String

The predefined value.

Set Byte Preprocessor

Example set byte preprocessor configuration
<set-byte-preprocessor>
    <variable>byte-variable</variable>
    <value encoding="UTF8">value</value>
</set-byte-preprocessor>
Table 8. Set byte preprocessor parameters
Parameter Type Description

variable

ESE variable

Defines to which modifiable byte ESE variable the decoded configured value is set.

value

String

The predefined value.

value/@encoding

Enum

The encoding of the value. Possible values are UTF8, Base64, and Hex.

Copy Preprocessor

A copy preprocessor offers a way to copy the state from one ESE variable to another. There are multiple types:

Copy String-to-String Preprocessor

Example copy string-to-string preprocessor configuration
<copy-string-to-string-preprocessor>
    <from>string-from</from>
    <to>string-to</to>
</copy-string-to-string-preprocessor>
Table 9. Copy string-to-string preprocessor parameters
Parameter Type Description

from

ESE variable

Defines the source string ESE variable.

to

ESE variable

Defines to which modifiable string ESE variable the source value is copied.

Copy Byte-to-Byte Preprocessor

Example copy byte-to-byte preprocessor configuration
<copy-byte-to-byte-preprocessor>
    <from>byte-from</from>
    <to>byte-to</to>
</copy-byte-to-byte-preprocessor>
Table 10. Copy byte-to-byte preprocessor parameters
Parameter Type Description

from

ESE variable

Defines the source byte ESE variable.

to

ESE variable

Defines to which modifiable byte ESE variable the source value is copied.

Copy String-to-Byte Preprocessor

Example copy string-to-byte preprocessor configuration
<copy-string-to-byte-preprocessor>
    <from encoding="UTF8">string-from</from>
    <to>byte-to</to>
</copy-string-to-byte-preprocessor>
Table 11. Copy string-to-byte preprocessor parameters
Parameter Type Description

from

ESE variable

Defines the source string ESE variable.

from/@encoding

Enum

The encoding of the value stored in from. Possible values are UTF8, Base64, and Hex.

to

ESE variable

Defines to which modifiable byte ESE variable the decoded value is copied.

Copy Byte-to-String Preprocessor

Example copy byte-to-string preprocessor configuration
<copy-byte-to-string-preprocessor>
    <from>byte-from</from>
    <to encoding="UTF8">string-to</to>
</copy-byte-to-string-preprocessor>
Table 12. Copy byte-to-string preprocessor parameters
Parameter Type Description

from

ESE variable

Defines the source byte ESE variable.

to

ESE variable

Defines to which modifiable byte ESE variable the encoded value is copied.

to/@encoding

Enum

The encoding of the value stored in to. Possible values are UTF8, Base64, and Hex.

Plain Preprocessor (deprecated)

The plain preprocessor is deprecated. Please use one of the copy preprocessors instead. Plain preprocessor support will be removed in a future release.

The plain preprocessor offers a way to transfer the state of ESE variables. It contains a list of one or more transformations. During runtime, the transformations are processed in the order in which they appear in the configuration file.

Example Plain preprocessor configuration
<plain-preprocessor>
    <transformations>
        <transformation>...</transformation>
        <transformation>...</transformation>
        <transformation>...</transformation>
        <!-- add  more transformations-->
    </transformations>
</plain-preprocessor>

A transformation copies values from one ESE variable to another ESE variable. These variables are annotated as from and to variables. After the transformation process, the value in the from variable is stored in the to variable as well as in the from variable. Constant ESE variables are not permitted in a to variable.

If the data type of the from and to variables differ, an encoding must be specified.

Example Transformation configuration
<transformation encoding="UTF8">
    <from>byte-1</from>
    <to>string-1</to>
</transformation>
Table 13. Transformation parameters
Parameter Type Description

encoding

Encoding

Defines how the ESE variable referenced in the from tag is interpreted. This parameter is only valid if the from and to variables have different data types and require a type conversion.

from

String

References the ESE variable in which the incoming data of the transformation is stored.

to

String

References the ESE variable in which the outgoing data of the transformation is stored.
Table 14. Transformation encodings
Encoding Description

UTF8

Uses the UTF-8 encoding as specified in RFC 3629.

Base64

Uses the Base64 encoding as specified in RFC 4648. Some transformations produce or require padding of up to two = characters.

Regex Preprocessor

The regex preprocessor allows you to pull substrings from a string ESE variable with the help of regular expressions. The preprocessor takes the value of the selected string variable, matches a pattern, and writes the resulting matches back into the string variable.

The regex preprocessor is useful when just a part of an ESE variable must be used for authentication or authorization purposes. Here are some examples:

  • All client passwords start with the phrase password followed by the real secret.

  • A part of the client ID needs to be inserted into the topic permissions to specify the client.

  • Only the first three characters of the username are relevant.

The regex preprocessor contains a regular expression pattern, an optional group number, and a list of ESE variables to which the preprocessor is applied.

Example Regex preprocessor configuration
<regex-preprocessor>
    <pattern>mqtt-client=((.{3})-(.{3}))</pattern>
    <group>1</group>
    <variable>authentication-key</variable>
</regex-preprocessor>
Table 15. Regex preprocessor parameters
Parameter Type Description

pattern

String

The regular expression that is used

group

Positive integer or 0

The group in the regular expression that is extracted. Groups are framed with parenthesis ( and ). The 0 group is always the whole pattern. The default is 0.

variable

String

References the ESE variable that is processed. This tag can be present multiple times.

Regular Expression Syntax

The regex preprocessor expands the regular expressions syntax that the HiveMQ Control Center uses to include grouping logic.

To add a group to a pattern, include a pair of matching parenthesis (). Groups are ordered by their opening parenthesis ( from left to right. Regardless of the parenthesis, the 0 group is always present and contains the whole matched pattern.

For example, a variable that contains the string "SOMETHINGmqtt-client=abc-defSOMETHING" is processed with the pattern mqtt-client=((.{3})-(.{3})) that contains four groups. Based on the matched group, the value of the string variable after preprocessing is as follows:

Table 16. Example Regular expression group
Group Number New Value

0

mqtt-client=abc-def

1

abc-def

2

abc

3

def

Concatenation Preprocessor

The Concatenation Preprocessor allows you to join the values of multiple string ESE variables and place the concatenated result into a single ESE variable.

The Concatenation Preprocessor contains one or more input string ESE variables and a single output string ESE variable. If desired, you can define a prefix and postfix to enclose the values of every input string ESE variable that is given.

Example Concatenation preprocessor configuration
<concatenation-preprocessor prefix="{{" postfix="}}">
    <from>string-1</from>
    <from>string-2</from>
    <to>string-3</to>
</concatenation-preprocessor>
Table 17. Concatenation preprocessor parameters
Parameter Type Description

prefix

String (XML attribute)

Specifies the characters that are used before a value to separate multiple values in a field (optional). Default: {{

postfix

String (XML attribute)

Specifies the characters that are used after a value to separate multiple values in a field (optional). Default: }}

from

String

References the string ESE variables that are concatenated. This tag can be present multiple times.

to

String

References the string ESE variable where the string that is the result of the concatenation is placed.

Concatenation Preprocessor Example

If you use the Concatenation Preprocessor example configuration, you can expect the following behavior:

Example 1. Example basic concatenation of variables

This example shows how the values of the string-1 and string-2 ESE variables are concatenated. The result of the concatenation is placed in the string-3 ESE variable.

The string-1 ESE variable contains the value "Lorem" and the string-2 ESE variable contains the value "Ipsum". After processing, the string-3 ESE variable contains the value "{{Lorem}}{{Ipsum}}".

Split Preprocessor

The Split Preprocessor allows you to divide the value of a string ESE variables into multiple tokens and place the resulting tokens into one or more ESE variables.

The Split Preprocessor contains one input string ESE variable and one or more output string ESE variables. You can also define a prefix and a postfix, that enclose the individual substrings in the input string ESE variable.

Example Split preprocessor configuration
<split-preprocessor prefix="{{" postfix="}}">
    <from>string-1</from>
    <to>string-2</to>
    <to>string-3</to>
</split-preprocessor>
Table 18. Split preprocessor parameters
Parameter Type Description

prefix

String (XML attribute)

The characters used before an extracted value to separate multiple values in a field. Default: {{ (must not be an empty string)

postfix

String (XML attribute)

The characters used after an extracted value to separate multiple values in a field. Default: }} (must not be an empty string)

from

String

References the string ESE variable that is divided into separate tokens.

to

String

References the string ESE variables where the tokens are placed. This tag can be present multiple times.
If more tokens are present than destination variables, the surplus tokens are not placed in a string ESE variable.

Split Preprocessor Example Usage

When you use the Split Preprocessor example configuration, you can expect the following behavior:

Example 2. Example basic splitting of a string ESE variable

This example shows how the value of the string-1 ESE variable is split into 2 strings (tokens) that are placed in the string-2 and string-3 ESE variables.
The string-1 ESE variable contains the value "{{Lorem}}{{Ipsum}}".
After processing, the string-2 ESE variable contains the value "Lorem" and the string-3 ESE variable contains the value "Ipsum".

Example 3. Example splitting of a string ESE variable into more tokens than destination ESE variables

This example shows how the value of the string-1 ESE variable is divided into tokens when only 2 destination ESE variables are given. As a result, the third token is ignored.
The string-1 ESE variable contains the value "{{Lorem}}{{Ipsum}}{{Dolor}}".
After processing, the string-2 ESE variable contains the value "Lorem" and the string-3 ESE variable contains the value "Ipsum".
The "Dolor" value is not placed in any ESE variable.

Letter Case Preprocessor

The Letter Case Preprocessor allows you to change the case of alphabetic characters in a string ESE Variable to uppercase or lowercase letters. The Letter Case Preprocessor only converts the alphabetic characters in the ESE Variable string. The preprocessor does not affect numbers and symbols and is for variables of the string data type only.

The Letter Case Preprocessor makes it possible to use a case-sensitive ESE variable with a case-insensitive (all uppercase or lowercase) system.

Constant ESE variables cannot be transformed. The constant variables contain read-only values that the ESE runtime system provides. For example, the MQTT variables mqtt-clientid and mqtt-username are constant variables that cannot be converted. ESE derives the value of these MQTT variables strings directly from the MQTT CONNECT packet of the connecting client. For more information, see ESE Variables.
Example Letter case preprocessor configuration
<letter-case-preprocessor>
    <transformations>
        <transformation>
            <variable>string-1</variable>
            <case>UPPER</case>
         </transformation>
         <transformation>
            <variable>string-3</variable>
            <case>LOWER</case>
         </transformation>
    </transformations>
</letter-case-preprocessor>
Table 19. Letter case preprocessor parameters
Parameter Type Description

variable

String

The string ESE variable to which the letter case conversion is applied.

case

String

The case to which the alphabetic characters in the defined string ESE variable are converted.

  • UPPER: Converts all alphabetic characters in the string ESE variable to uppercase letters.

  • LOWER: Converts all alphabetic characters in the string ESE variable to lowercase letters.
To minimize inconsistency, we recommend that you use an ASCII character set when you implement the Letter Case Preprocessor.

X.509 Preprocessor

The X.509 preprocessor allows you to use information that is provided in the X.509 certificate of a connecting MQTT client to authenticate and authorize the client in HiveMQ.

You can use the X.509 preprocessor to extract values from specific fields in the X.509 certificate of the client and copy the values to an ESE variable.

If desired, you can add one or more regex preprocessors to automatically transform the values you extract.

X.509 Preprocessor Configuration

The X.509 preprocessor contains a list of X.509 extractions and a configurable prefix/postfix (for fields that allow multiple entries).

Example X.509 preprocessor configuration
<enterprise-security-extension>
    <pipelines>
        <listener-pipeline listener="listener">
            <authentication-preprocessors>
                <x509-preprocessor prefix="{{" postfix="}}">
                    <x509-extractions>
                        <x509-extraction>
                            <x509-field>subject-common-name</x509-field>
                            <ese-variable>authentication-key</ese-variable>
                        </x509-extraction>
                    </x509-extractions>
                </x509-preprocessor>
            </authentication-preprocessors>
        </listener-pipeline>
    </pipelines>
</enterprise-security-extension>
Table 20. X.509 preprocessor parameters
Parameter Type Description

prefix

String (XML attribute)

The characters used before an extracted value to separate multiple values in a field. Default: {{

postfix

String (XML attribute)

The characters used after an extracted value to separate multiple values in a field. Default: }}

x509-extractions

x509-extraction

A list of x509-extractions that specify how the values that are stored in an X.509 certificate are transformed into ESE variables.
Table 21. X.509 extraction parameters
Parameter Type Description

x509-field

XML enum

The field in the X.509 certificate that is extracted into the ESE variable.

ese-variable

XML enum

The ESE variable that stores the extracted information. This ESE variable must be a non-constant string variable.
Table 22. Extractable X.509 fields
Attribute Type Description

issuer-common-name

Single

The common name (CN, OID 2.5.4.3) of the Certificate Authority (CA) that issued the certificate.

subject-common-name

Single

The common name (CN, 2.5.4.3) of the entity to which the certificate is issued.

issuer-alternative-common-names

Multi

The optional additional common names (CN, 2.5.4.3) for the certificate issuer extension (IAN, 2.5.29.18). The issuer alternative common names must be provided using the dirName type, which supports a fully structured distinguished name (DN).

subject-alternative-common-names

Multi

The optional additional common names (CN, 2.5.4.3) for the subject alternative names extension (SAN, 2.5.29.17). The subject alternative common names must be provided using the dirName type, which supports a fully structured distinguished name (DN).

issuer-organization

Single

The organization (O, 2.5.4.10) of the issuer.

subject-organization

Single

The organization (O, 2.5.4.10) of the subject.

issuer-organizational-unit

Single

The organizational unit (OU, 2.5.4.11) of the issuer.

subject-organizational-unit

Single

The organizational unit (OU, 2.5.4.11) of the subject.

issuer-country

Single

The country (C, 2.5.4.6) of the issuer.

subject-country

Single

The country (C, 2.5.4.6) of the subject.

serial-number

Single

The serial number of the certificate.

fingerprint-sha256

Single

The colon separated HEX SHA-256 fingerprint of the certificate. Example: 09:55:B0:B4:74:9D:48:CD:77:4F:6F:7C:C2:55:54:9E:A8:99:79:A2:49:A3:E9:FE:3D:38:DA:AC:48:1D:92:60

fingerprint-sha1

Single

The colon separated HEX SHA-1 fingerprint of the certificate. Example: FA:49:CE:EE:E4:48:E0:85:17:89:4D:79:5E:0C:DA:F1:96:DA:9E:92

X.509 Certificate Generation

The following example shows the basic steps needed to generate an X.509 certificate for an MQTT client.

  1. Prepare an OpenSSL openssl.cnf configuration file.

    Example OpenSSL configuration file
    [req]
distinguished_name = req_distinguished_name
req_extensions = v3_req
prompt = no

[req_distinguished_name]
C = DE
ST = Bayern
L = Landshut
O = HiveMQ GmbH
CN = client

[v3_req]
subjectAltName = dirName:dir_sect

[dir_sect]
C = DE
O = HiveMQ GmbH
OU = HiveMQ Support
CN = client

    The openssl.cnf file configures a certificate signing request (CSR) for the MQTT client.

    Table 23. OpenSSL configuration file
    File Section Description

    [req]

    Defines how the certificate signing request (CSR) is created including which distinguished name (DN) to prompt for, which extensions to include, and how the CSR must be formatted.

    [req_distinguished_name]

    Specifies the distinguished name (DN)of the client.

    [v3_req]

    Specifies that the certificate signing request (CSR) includes the subject alternate name (SAN) extension with a directory named dir_sect.

    [dir_sect]

    Specifies the values for the subject alternate name (SAN) extension.

  2. Create the certificate signing request (CSR) and key file:

    openssl req \
  -new \
  -out 'client.csr' \
  -newkey rsa:2048 \
  -nodes \
  -sha256 \
  -keyout 'client-key.temp' \
  -config 'openssl.cnf'

  3. Verify the certificate signing request (CSR):

    openssl req -text -noout -verify -in 'client.csr'

  4. Extract the RSA key from the key file:

    openssl rsa -in 'client-key.temp' -out 'client-key.pem'

  5. Generate the client certificate using the certificate signing request (CSR) and the OpenSSL configuration file:

    openssl x509 -req -in 'client.csr' \
  -signkey 'client-key.pem' \
  -out 'client-cert.pem' \
  -days 365 \
  -extensions 'v3_req' \
  -extfile 'openssl.cnf'

  6. View the generated client-cert.pem client certificate file:

    openssl x509 -in client-cert.pem -text -noout

    The client certificate contains the subject alternative name (SAN) with type dirName as shown in the following example:

    Example X.509 MQTT Client Certificate Extension
    X509v3 Extensions:
    X509v3 Issuer Alternative Name:
        DirName:/CN=client/OU=HiveMQ Support/O=HiveMQ GmbH/C=DE
    X509v3 Subject Alternative Name:
        DirName:/CN=client/OU=HiveMQ Support/O=HiveMQ GmbH/C=DE

X.509 Certificate Field Concatenation

In an X.509 certificate, some fields can contain more than one entry. If a field contains multiple entries, the entries of the field are wrapped in the configured prefix and postfix and concatenated before they are written into the ESE variable.

X.509 Certificate Field Concatenation Example

In this example, an MQTT client of a car-pool vehicle provides a certificate that contains the following two subject-alternative common names:

  • id-1234567 (the identification number of the car)

  • car-pool-south-east (the car-pool group to which the car belongs)

After extraction into an ESE variable with the default prefix/postfix, the ESE variable contains the following concatenated string:

{{id-1234567}}{{car-pool-south-east}}

Next, a regex preprocessor with the pattern {{.}}{{(.)}} and group 1 is applied.
The regex preprocessor creates an ESE variable that contains the following string:

car-pool-south-east

To complete the example, a preconfigured authorization manager uses the resulting ESE variable to assign role-based permissions to the MQTT client of the car-pool vehicle.

JWT Preprocessor

The JWT preprocessor makes it possible to extract information that is provided in the JSON Web Token (JWT) of a connecting client for authentication of the client in HiveMQ.

The HiveMQ Enterprise Security Extension retrieves information from the payload of the JWT. The JWT payload contains Base64URL encoded information about the bearer of the token. Each statement in the JWT payload is called a claim.

You can use the JWT preprocessor to extract values from specific claims in the JWT that the client presents and copy the value to an ESE variable.

If desired, add one or more regex preprocessors to transform the information that you extract automatically.

JWT Preprocessor Configuration

The JWT preprocessor contains a list of extractions and a configurable prefix/postfix (for fields that allow multiple entries).

Example JWT preprocessor configuration
<jwt-preprocessor prefix="{{" postfix="}}">
    <source>authentication-byte-secret</source>
    <jwt-extractions>
        <jwt-extraction>
            <jwt-claim>azp</jwt-claim>
            <ese-variable>string-1</ese-variable>
        </jwt-extraction>
    </jwt-extractions>
</jwt-preprocessor>
Table 24. JWT preprocessor parameters
Parameter Type Description

prefix

String (XML attribute)

The characters used before an extracted value to separate multiple values in a field. Default: {{

postfix

String (XML attribute)

The characters used after an extracted value to separate multiple values in a field. Default: }}

source

ESE Variable

The byte variable that contains the JWT.

jwt-extractions

jwt-extraction

A list of jwt-extractions that specify how the values that are stored in a JWT are transformed into ESE variables.
Table 25. JWT extraction parameters
Parameter Type Description

claim

String

The field in the JWT that is extracted into the ESE variable. For example, reserved-claim values such as aud, sub, iss, and scope. Possible custom-claim values are based on your individual configuration.

ese-variable

ESE variables

The ESE variable that stores the extracted information. This ESE variable must not be constant.

In a JWT, some claims can contain more than one entry. If a claim contains multiple entries, the entries of the field are wrapped in the configured prefix and postfix and concatenated before they are written into the ESE variable.

Logging Preprocessor

The logging preprocessor offers a way to debug the state of ESE variables. At the pipeline stage where the preprocessor is placed, the content of the variables is output to a logger. The loggers are placed in the pipeline in same the order in which they appear in the configuration file.

Logging preprocessors impact performance. Only use logging preprocessors for debugging purposes.
Example Logging preprocessor configuration
<logging-preprocessor>
    <message>The content of the string-1 ESE-Variable: ${string-1}</message>
    <level>debug</level>
    <name>com.example.logger</name>
</logging-preprocessor>
Table 26. Logging preprocessor parameters
Parameter Type Default Description

message (required)

String

Defines the message that is output to the logger. Use the following syntax to input the ESE variables: ${ESE_VARRIABLE_NAME}.

level

logging level

debug

The logging level of the message.

name (required)

String

The name of the logger.

codec

encoding

Base64

The encoding of byte ESE variables.
Table 27. Transformation encodings
Level Description

trace

Trace Logging Level

debug

Debug Logging Level

info

Info Logging Level

warn

Warn Logging Level

error

Error Logging Level

Log Level

By default, the root level that is set in the HiveMQ logback.xml is used. If you wish to set the log level for a specific logging preprocessor, you can define the level in the HiveMQ logback.xml.

Example to set a specific logging preprocessor to the debug level
<configuration scan="true" scanPeriod="60 seconds">

    ...

    <logger name="<LOGGING_PREPROCESSOR_NAME>" level="debug"/>

    ...

</configuration>
Example to set a specific logging preprocessor to the info level
<configuration scan="true" scanPeriod="60 seconds">

    ...

    <logger name="<LOGGING_PREPROCESSOR_NAME>" level="info"/>

    ...

</configuration>
Example to set a specific logging preprocessor to the trace level
<configuration scan="true" scanPeriod="60 seconds">

    ...

    <logger name="<LOGGING_PREPROCESSOR_NAME>" level="trace"/>

    ...

</configuration>

Session Attribute Preprocessor

The Session Attribute preprocessor offers a way to store the state of ESE variables as session attributes in the Session Attribute Store.

Writing session attributes impacts performance and storage. Only write necessary data.
Example Session Attribute preprocessor configuration
<session-attribute-preprocessor>
    <from-ese-variable>string-1</from-ese-variable>
    <to-session-attribute>my-session-attribute</to-session-attribute>
</session-attribute-preprocessor>
Table 28. Session Attribute preprocessor parameters
Parameter Type Default Description

from-ese-variable (required)

String

References the ESE variable from which the preprocessor reads data. ESE variables of data type String are decoded to UTF-8 bytes.

to-session-attribute (required)

String

References the session attribute to which the preprocessor writes the data.

Connection Attribute Preprocessor

The Connection Attribute preprocessor offers a way to store the state of ESE variables as connection attributes in the Connection Attribute Store.

Connection attributes enlarge the memory footprint per connection. Only write necessary data.
Example Connection Attribute preprocessor configuration
<connection-attribute-preprocessor>
    <from-ese-variable>string-1</from-ese-variable>
    <to-connection-attribute>my-connection-attribute</to-connection-attribute>
</connection-attribute-preprocessor>
Table 29. Connection Attribute preprocessor parameters
Parameter Type Default Description

from-ese-variable (required)

String

References the ESE variable from which the preprocessor reads data. ESE variables of data type String are decoded to UTF-8 bytes.

to-connection-attribute (required)

String

References the connection attribute to which the preprocessor writes the data.

HTTP Headers Preprocessor

The HTTP Headers preprocessor makes it possible to extract information from the HTTP header of an incoming REST API client request for authentication and authorization in HiveMQ.

The preprocessor supports different RFC7235 HTTP authentication schemes to extract authentication credentials.

When a JWT authentication manager is used, an extraction for the RFC6750 Bearer Token authentication scheme is required. For other authentication managers the RFC7617 Basic authentication scheme is needed.

An HTTP Headers preprocessor is required to configure a REST API pipeline.

RFC7617 Basic Authentication Credentials

The obtained username/password pair is concatenated with a single colon and base64 encoded. In the header value field of the Authorization header, enter the word Basic followed by an empty space and the base64 encoded value of your username:password.

For example, to send the username Aladdin and the password OpenSesame, the HTTP header field in the REST API client request would use the following entry:

Authorization: Basic QWxhZGRpbjpPcGVuU2VzYW1lCgo=

The extracted information is used for the authentication-key and authentication-byte-secret.

HTTP Headers Basic Authentication Preprocessor Configuration
<enterprise-security-extension>
    <pipelines>
        <rest-api-pipeline listener="ALL">
            <authentication-preprocessors>
                <http-headers-preprocessor>
                    <basic-auth-extraction/>
                </http-headers-preprocessor>
            </authentication-preprocessors>
            <sql-authentication-manager>
                <realm>mysql-backend</realm>
            </sql-authentication-manager>
            <sql-authorization-manager>
                <realm>mysql-backend</realm>
            </sql-authorization-manager>
        </rest-api-pipeline>
    </pipelines>
</enterprise-security-extension>

RFC6750 Bearer Authentication Credentials

In the header value field of the Authorization header, enter the word Bearer followed by the token used for authentication.

The HTTP header field in the REST API client request will look similar to the following example:

Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9…​

To be used for authentication, the token must comply with the JSON Web Token (JWT) specification.

The extracted information is used for the authentication-byte-secret.

HTTP Headers Bearer Authentication Preprocessor Configuration
<enterprise-security-extension>
    <pipelines>
        <rest-api-pipeline listener="ALL">
            <authentication-preprocessors>
                <http-headers-preprocessor>
                    <bearer-auth-extraction/>
                </http-headers-preprocessor>
            </authentication-preprocessors>
            <jwt-authentication-manager>
                <realm>jwt-realm</realm>
            </jwt-authentication-manager>
            <sql-authorization-manager>
                <realm>mysql-backend</realm>
            </sql-authorization-manager>
        </rest-api-pipeline>
    </pipelines>
</enterprise-security-extension>

Cryptographic Operations

The ESE uses cryptography to compare securely-stored secrets with the secrets that the MQTT clients provide for authentication. To make the comparison, the ESE currently employs four different password hashing schemata.

All schemata can be scaled in computational difficulty and all schemata allow for the use of an optional salt. Difficulty scaling is done with a difficulty parameter (if the one-way function that is used supports this parameter) or by using the output of the function as input in additional rounds of hashing. The Salt is used in the way that is specified by the one-way function or it is prepended once before the first round of hashing is done.

The ESE expects all byte strings such as hashed passwords to be encoded with the Base64 encoding scheme.

Once the cryptographic processing of the client-provided secret is done, the ESE compares the result with the configured trust for bit equality.

MD5 Message-Digest Algorithm

The ESE can use the MD5 message digest.

  • The salt is prepended to the secret before the first round of hashing.

  • The difficulty is scaled by using the output of MD5 again as input for iteration-times rounds.

  • The resulting hash has a length of 16 bytes or 24 characters in Base64 encoding.

MD5 is proven to be insufficiently strong for modern security standards and is only made available for testing and legacy purposes.

SHA512

The ESE can use the Secure Hashing Algorithm 2 with an output bit-length of 512 (SHA512).

  • The salt is prepended to the secret before the first round of hashing.

  • The difficulty is scaled by using the output of SHA512 again as input for iteration-times rounds.

  • The resulting hash has a length of 64 bytes or 87 characters in Base64 encoding.

BCRYPT

The ESE can use the Blowfish cipher-based password hashing scheme bcrypt.

  • The salt is hashed with MD5 to resize it before use. Because bcrypt requires the use of a salt, an empty string is hashed and used if no salt is configured.

  • The secret is hashed with SHA512 to resize it before use.

  • The iteration parameter is directly used as the difficulty parameter of bcrypt. Only values between 4 and including 31 are allowed.

  • The resulting hash has a length of 24 bytes or 32 characters in Base64 encoding.

PKCS5S2

The ESE can use the PKCS 5 V2.0 Scheme 2 as described in RFC 2898.

  • The underlying one-way function is SHA512.

  • The difficulty and the salt are directly passed on to the scheme.

  • The resulting hash has a length of 32 bytes or 44 characters in Base64 encoding.

PLAIN Schema in HiveMQ Enterprise Security Extension

The PLAIN schema (available in ESE version 1.1) is not a cryptographic operation. This schema simply prompts the ESE to do a plain-text secret comparison (without any transformation of the provided secret beforehand).

  • No one-way function is used.

  • Any difficulty parameter or salt is ignored.

  • The length of the resulting byte string or Base64 encoding is highly dependent on the secret that is used.

Only use the PLAIN schema if you have a specific use case that requires this schema. Because the performance of the PLAIN schema is very different from real hash functions, the schema is unsuitable even for testing purposes.

Authentication and Authorization in HiveMQ Enterprise Security Extension

The key tasks of the Enterprise Security Extension are authentication and authorization of MQTT clients. For a detailed explanation of these concepts in the context of HiveMQ, see Authorization Options for Your HiveMQ Extension.

Authentication Managers

Authentication verifies whether a person, device, or application is who they say they are.

In the ESE, authentication is done by authentication managers. An authentication manager is defined in a pipeline and handles the authentication processes of connecting MQTT clients by using the content of the authentication variables.

Authorization Managers

Authorization defines which actions a specific person, device, or application is allowed to perform. Authorization can be split up into the concepts of roles and permissions.

A permission contains a set of actions. Users who are granted a specific permission are allowed to do the actions that are defined in the permission.

Permissions can also be assigned to a role. This role can then be assigned to a set of users. Users who have a certain role receive all of the permissions that are assigned to the role.

In the ESE, authorization is done by authorization managers. An authorization manager is defined in a pipeline and handles the authorization processes of connecting MQTT clients by using the content of the authorization variables.

We highly recommend the use of roles to assign permissions rather than the assignment of user-specific permissions. The use of roles to manage permissions creates a clear authority structure on your system and can improve overall performance of the ESE.

Allow-all Authentication and Authorization Managers

The ESE provides two special authentication and authorization managers to facilitate testing. These managers allow all clients to connect to HiveMQ without checking the credentials or permissions of the connecting client.

The allow-all managers are especially useful when you want to test your authentication or authorization workflow individually. For example, to test whether all clients can connect to your SQL authentication manager, you can use the allow-all authorization manager to exclude failed authentications that authorization causes.

The allow-all managers can be used for testing in the HiveMQ listener pipelines. Based on your use case, you can use one or both allow-all managers in a listener pipeline.

You can also use allow-all authorization managers in your control center and REST API pipelines. In this case, the user is assigned the HIVEMQ_SUPER_ADMIN permission level.

Allow All Authentication Manager Configuration

Example Allow-All Authentication Manager configuration
<pipelines>
    <listener-pipeline listener="ALL">
        <allow-all-authentication-manager/>

        <!-- add authorization preprocessors if necessary -->
        <!-- add an authorization manager -->
    </listener-pipeline>
</pipelines>

Allow All Authorization Manager Configuration

Example Allow-All Authorization Manager configuration
<pipelines>
    <listener-pipeline listener="ALL">
        <!-- add authentication preprocessors if necessary -->
        <!-- add an authentication manager -->

        <allow-all-authorization-manager/>
    </listener-pipeline>
    <control-center-pipeline>
        <!-- add authentication preprocessors if necessary -->
        <!-- add an authentication manager -->

        <allow-all-authorization-manager/>
    </control-center-pipeline>
    <rest-api-pipeline listener="ALL">
        <!-- add authentication preprocessors if necessary -->
        <!-- add an authentication manager -->

        <allow-all-authorization-manager/>
    </rest-api-pipeline>
</pipelines>
The allow-all authentication and authorization managers are intended for testing purposes only. We do not recommend the use of these managers in a production environment.

Chain Authentication and Authorization Managers

The HiveMQ Enterprise Security Extension provides authentication and authorization chain managers that allow you to combine multiple authentication or authorization managers within a selected pipeline. Chain managers facilitate the implementation of advanced authentication and authorization scenarios.

The chain manager iterates through a list of authentication or authorization managers and supports several strategies for handling the authentication or authorization results.

The selection of authorization or authentication managers you can combine in the chain manager configuration varies based on the enclosing pipeline type.

You can implement chain managers for MQTT listeners, REST API pipelines, and the control center.

Authentication and authorization chain managers support a maximum of one nesting level. Deeper nesting of chain managers is not allowed.

Chain Authentication Manager Configuration

Example chain authentication manager configuration in an MQTT listener pipeline
<pipelines>
    <listener-pipeline listener="ALL">
        <!-- add authentication preprocessors if necessary -->
        <chain-authentication-manager>
            <strategy>
                <check-next-on-unknown-authentication-key-or-wrong-secret/>
            </strategy>
            <chain>
                <sql-authentication-manager>
                    <realm>sql-realm</realm>
                </sql-authentication-manager>
                <file-authentication-manager>
                    <realm>file-realm</realm>
                </file-authentication-manager>
            </chain>
        </chain-authentication-manager>

        <!-- add authorization preprocessors if necessary -->
        <!-- add an authorization manager -->
    </listener-pipeline>
</pipelines>
Parameter Type Mandatory Description

strategy

Choice

Defines how the chain manager handles the authentication results from the authentication managers listed in the chain. The following strategies are available:

  • check-next-on-unknown-authentication-key: Skips to the next manager in the chain when one of the following conditions is met:

    • An error prevents a successful retrieval.

    • The authentication-key is unknown to the manager.

  • check-next-on-unknown-authentication-key-or-wrong-secret: Skips to the next manager in the chain when one of the following conditions is met:

    • An error prevents a successful retrieval.

    • The authentication-key is unknown to the manager.

    • The supplied secret is incorrect.

chain

List

The list of authentication managers the chain manager checks. The list is checked in the defined order and must contain at least two entries. The type of the enclosing pipeline determines which authentication managers can be included in the list:

  • MQTT listener pipeline:

    • sql-authentication-manager

    • file-authentication-manager

    • allow-all-authentication-manager

    • ldap-authentication-manager

    • chain-authentication-manager

  • Control center pipeline:

    • sql-authentication-manager

    • file-authentication-manager

    • ldap-authentication-manager

    • chain-authentication-manager

  • REST API pipeline:

    • sql-authentication-manager

    • file-authentication-manager

    • chain-authentication-manager

Chain Authorization Manager Configuration

Example chain authorization manager configuration in an MQTT listener pipeline
<pipelines>
    <listener-pipeline listener="ALL">
        <!-- add authentication preprocessors if necessary -->
        <!-- add an authentication manager -->

        <!-- add authorization preprocessors if necessary -->
        <chain-authorization-manager>
            <strategy>
                <check-next-on-unknown-authorization-keys-or-empty-permissions/>
            </strategy>
            <chain>
                <sql-authorization-manager>
                    <realm>sql-realm</realm>
                </sql-authorization-manager>
                <file-authorization-manager>
                    <realm>file-realm</realm>
                </file-authorization-manager>
            </chain>
        </chain-authorization-manager>
    </listener-pipeline>
</pipelines>
Parameter Type Mandatory Description

strategy

Choice

Defines how the chain manager handles results from the authorization managers listed in the chain. The following strategies are available:

  • check-next-on-unknown-authorization-keys: Skips to the next manager in the chain when the authorization-key and all roles of the authorization-role-key met one of the following conditions:

    • An error prevents a successful retrieval.

    • The user/role is unknown to the manager.

    • The user/role retrieval is disabled by the manager.

  • check-next-on-unknown-authorization-keys-or-empty-permissions: Skips to the next manager in the chain when the authorization-key and all roles of the authorization-role-key met one of the following conditions:

    • An error prevents a successful retrieval.

    • The user/role is unknown to the manager.

    • The user/role retrieval is disabled by the manager.

    • The resulting permission list is empty.

  • merge-all-permissions: Iterates through all managers in the chain and merges all permissions into one list.

chain

List

The list of authorization managers the chain manager checks. The list is checked in the defined order and must contain at least two entries. The type of the enclosing pipeline determines which authorization managers can be included in the list:

  • MQTT listener pipeline:

    • sql-authorization-manager

    • file-authorization-manager

    • allow-all-authorization-manager

    • ldap-authorization-manager

    • chain-authorization-manager

  • Control center pipeline:

    • sql-authorization-manager

    • file-authorization-manager

    • allow-all-authorization-manager

    • ldap-authorization-manager

    • chain-authorization-manager

  • REST API pipeline:

    • sql-authorization-manager

    • file-authorization-manager

    • allow-all-authorization-manager

    • chain-authorization-manager

SQL Databases

The HiveMQ Enterprise Security Extension supports different SQL databases as external sources of authentication and authorization data. In the ESE configuration file, you can use the abstract representation of SQL realms to define the connection to one or more databases.
SQL realms can be referenced in SQL authentication managers and/or SQL authorization managers.
To enable the ESE to work with the stored information, the databases must adhere to the SQL structure that is described below.
In the ESE, SQL realms and the associated managers support access control for the HiveMQ Control Center. To use the access-control feature for your control center, you need to set up additional SQL tables.

The ESE supports the following SQL databases:

See the full list of supported database versions.

SQL Realm

To connect the HiveMQ Enterprise Security Extension to an SQL database, simply configure an SQL realm in the <realms> section of your ESE configuration. The SQL realm must contain the database connection information and have a unique name that the authentication and authorization managers can reference.

Example SQL realm configuration
<enterprise-security-extension>
    ...
    <realms>
        <sql-realm>
            <name>sql-realm-name</name>
            <enabled>true</enabled>
            <configuration>
                <db-type>POSTGRES</db-type>
                <db-name>hivemq</db-name>
                <db-host>hostname</db-host>
                <db-port>5432</db-port>
                <db-username>hivemq</db-username>
                <db-password>password</db-password>
            </configuration>
        </sql-realm>
    </realms>
    ...
</enterprise-security-extension>
Table 30. SQL realm database configuration parameters
Parameter Type Mandatory Description

db-type

SQL type

Specifies the database technology

db-name

String

Specifies the database within the DBMS

db-host

String

Specifies the remote address of the database

db-port

Integer

Specifies the remote port of the database

db-username

String

Specifies the username that the ESE uses to connect to the database

db-password

String

Specifies the password that the ESE uses to authenticate itself

db-schema

String

Overrides the default PostgreSQL schema search path. This setting is only supported for the SQL types POSTGRES and AURORA_POSTGRES.

Supported Database Technologies

Since the ESE supports various types of SQL databases, you must specify the type of database for each realm. Use the following predefined strings to specify the SQL database type:

Table 31. SQL types for SQL realm configuration
SQL Type Database Technology

POSTGRES

Use this type to connect to a PostgreSQL database

MYSQL

Use this type to connect to a MySQL or MariaDB database

MSSQL

Use this type to connect to an MSSQL database

AURORA_POSTGRES

Use this type to connect to an Amazon Aurora database, using the PostgreSQL API.

AURORA_MYSQL

Use this type to connect to an Amazon Aurora database, using the MySQL API.

AZURE_SQL

Use this type to connect to an Azure SQL database
Amazon Aurora offers the possibility to use a PostgreSQL or a MySQL API. You can specify the preferred type at database creation. If you do not know which database type you are using, you can check the RDS overview in the AWS management console.

TLS Connection to SQL databases (optional)

Transport Layer Security (TLS) is a cryptographic security protocol that secures data transfer between systems. The HiveMQ Enterprise Security Extension gives you the option to configure TLS encryption for the connection to your SQL database.

The ESE supports secure TLS connection to the following SQL database types:

  • PostgreSQL (POSTGRES)

  • MySQL (see note)

  • MariaDB (see note)

  • MSSQL (see note)

  • AURORA PostgreSQL

  • AURORA MySQL

  • Azure SQL (see note)

Example TLS configuration for an SQL realm
<enterprise-security-extension>
    ...
    <realms>
        <sql-realm>
            ...
            <configuration>
                ...
                <tls>
                    <enabled>true</enabled>
                    <protocols>
                        <protocol>TLSv1.3</protocol>
                        <protocol>TLSv1.2</protocol>
                    </protocols>
                    <cipher-suites>
                        <cipher-suite>MY_CIPHER_SUITE_1</cipher-suite>
                        <cipher-suite>MY_CIPHER_SUITE_2</cipher-suite>
                    </cipher-suites>
                    <keystore>
                        <path>/path/to/keystore</path>
                        <private-key-password>private-key-password</private-key-password>
                        <password>keystore-password</password>
                    </keystore>
                    <truststore>
                        <path>/path/to/truststore</path>
                        <password>trust-store-password</password>
                    </truststore>
                    <verifyHostname>true</verifyHostname>
                </tls>
            </configuration>
        </sql-realm>
    </realms>
    ...
</enterprise-security-extension>
Table 32. TLS configuration parameters for SQL realm
Parameter Type Description

enabled

Boolean

Specifies whether the TLS configuration is used to encrypt communication between the ESE and the defined SQL database. The default setting is true. To disable your TLS configuration, set the enabled parameter to false.

protocols

String

Specifies the versions of the TLS protocol that can be used to secure communication between ESE and the specified SQL database. By default, the ESE enables TLSv1.3 and TLSv1.2.

  • protocol: Optional setting to explicitly define TLS protocols. If desired, you can define one or more specific TLS versions manually.

cipher-suites

String

The sets of cryptographic algorithms the TLS protocol uses to create keys and encrypt information. By default, the ESE uses the default cipher suites of your system. The default cipher suites that are available on your system vary based on the Java Runtime Environment and the JDBC Driver that you use.

  • cipher suite: Optional setting to explicitly define cipher suites. If desired, you can define specific cipher suites to limit the number of suites that are enabled.

keystore

Integer

Specifies the set of security credentials that are used to provide proof of identity. By default, the ESE uses the keystore that is configured for your system.

  • path: The path to the keystore where client certificates are stored.

  • private-key-password: The password for the private key.

  • password: The password for the keystore.

truststore

String

Specifies the credentials that are used to verify the identity claim of a communication partner. By default, the ESE uses the truststore that is configured for your system.

  • path: The path to the truststore where trusted certificates are stored.

  • password: The password for the truststore.

verifyHostname

String

Specifies whether the ESE checks the identity of the remote server.
When you configure TLS communication for a MySQL database with the optional MySQL Connector/J driver, some special considerations apply.

MySQL/MariaDB-specific considerations:

  • MySQL and MariaDB have only one field for keystorePassword/privateKeyPassword. MySQL and MariaDB databases use this single value to unlock the keystore and to decrypt the private key password.
    To accommodate this behavior, the ESE uses only the configured keystore password value for MySQL databases.

  • If you configure TLS for a MySQL or MariaDB database in the ESE, you must enter the same value in the keystore password field and the keystore private-key-password field.

When you configure TLS communication for an MSSQL or Azure SQL database in the HiveMQ Enterprise Security Extension, some special considerations apply.

MSSQL/Azure SQL-specific considerations:

  • MSSQL and Azure SQL do not support client certificates. If your TLS configuration specifies a keystore for an MSSQL or Azure SQL database, the ESE ignores the setting and logs a warning.

  • MSSQL and Azure SQL require hostname verification. If you disable hostname verification in your TLS configuration for an MSSQL or Azure SQL database, the ESE ignores the setting, verifies the hostname, and logs a warning.

  • MSSQL and Azure SQL only support the specification of a single TLS protocol. If your TLS configuration for an MSSQL or Azure SQL database lists multiple TLS protocol versions, the ESE uses the first TLS version in the list and logs a warning.

  • MSSQL and Azure SQL do not support specification of cipher suites. If you list specific cipher suites in your TLS configuration for an MSSQL or Azure SQL database, the ESE ignores the setting and logs a warning.

Caching

The querying of SQL databases incurs significant overhead in execution time. To remedy this, ESE caches information gained from connected databases.

The amount of time each type of data is cached before a time-based eviction can occur is predefined:

As cache size is limited, evictions can occur before the caching time expires.
Authentication failures caused within the database, such as due to the user not existing, are also cached for one minute.

These caching rules also apply to the corresponding REST API data.

SQL Authentication Manager

To use an SQL database as an external source for authentication, configure an SQL authentication manager in the desired pipeline and reference the corresponding SQL realm name in the authentication manager.

The ESE uses the authentication variables to look up authentication information in the database. By default, the authentication variables are filled with the MQTT variables mqtt-username and mqtt-password. If you want to adapt the values of the authentication variables, you can use a preprocessor before the SQL authentication manager.

To ensure that the ESE can find the authentication information in the database, the layout of the database must be in accordance with the database table 'users' of the SQL database structure. If the authentication manager gets also authorization information from the database, it sets the authorization variables authorization-key and authorization-role-key too.

Example SQL Authentication Manager configuration
<enterprise-security-extension>

    <!-- add realms configuration -->

    <pipelines>
        <listener-pipeline listener="ALL">
            <!-- add authentication preprocessors if necessary -->

            <sql-authentication-manager>
                <realm>sql-realm-name</realm>
            </sql-authentication-manager>

            <!-- add authorization preprocessors if necessary -->
            <!-- add an authorization manager -->
        </listener-pipeline>
    </pipelines>

</enterprise-security-extension>
Table 33. SQL authentication manager parameters
Parameter Type Mandatory Description

realm

String

References the SQL realm from which the authentication data is drawn.

SQL Authorization Manager

To use an SQL database as an external source for authorization, configure an SQL authorization manager in the desired pipeline and reference the corresponding SQL realm name in the authorization manager.

The authorization manager also lets you define whether you want to use roles and/or user-specific permissions. Based on the configuration, the ESE uses different authorization variables. These variables can be set and manipulated with a preprocessor before the authorization manager.

Table 34. SQL authorization manager parameters
Parameter Type Mandatory / Default Description

realm

String

References the SQL realm from which the authentication data is drawn.

use-authorization-key

Boolean

false

If this parameter is set to true, the ESE uses user-specific permissions.

use-authorization-role-key

Boolean

true

If this parameter is set to true, the ESE uses role permissions.

Role Permissions

If you want to use role permissions, set <use-authorization-role-key> to true. The variable authorization-role-key must be set, so that the ESE can query the corresponding role and its permissions from the database.

Example SQL Authorization Manager configuration using role permissions
<enterprise-security-extension>

    <!-- add realms configuration -->

    <pipelines>
        <listener-pipeline listener="ALL">
            <!-- add authentication preprocessors if necessary -->
            <!-- add an authentication manager -->
            <!-- add authorization preprocessors if necessary -->

            <sql-authorization-manager>
                <realm>sql-realm-name</realm>
                <use-authorization-key>false</use-authorization-key>
                <use-authorization-role-key>true</use-authorization-role-key>
            </sql-authorization-manager>

        </listener-pipeline>
    </pipelines>

</enterprise-security-extension>

To ensure that the ESE finds the authorization information in the database, the layout of the database must be in accordance with the following database tables from the SQL database structure:

We recommend that you use role permissions rather than of user-specific permissions.

User-specific Permissions

If you want to use user-specific permissions, set <use-authorization-key> to true. The variable authorization-key must be set, so that the ESE can query the corresponding user-specific permissions from the database.

Example SQL Authorization Manager configuration using user-specific permissions
<enterprise-security-extension>

    <!-- add realms configuration -->

    <pipelines>
        <listener-pipeline listener="ALL">
            <!-- add  authentication preprocessors if necessary -->
            <!-- add  authentication manager -->
            <!-- add  authorization preprocessors if necessary -->

            <sql-authorization-manager>
                <realm>example-backend</realm>
                <use-authorization-key>true</use-authorization-key>
                <use-authorization-role-key>false</use-authorization-role-key>
            </sql-authorization-manager>

        </listener-pipeline>
    </pipelines>

</enterprise-security-extension>

To ensure that the ESE finds the authorization information in the database, the layout of the database must be in accordance with the following database tables from the SQL database structure:

SQL Database Setup

SQL Driver Installation (optional)

Each SQL realm must create a JDBC database connection to the configured database instance. To create the connection, a database-specific JDBC driver is used.

ESE provides the following JDBC drivers:

If there are special requirements regarding the used JDBC driver, it is possible to override the included drivers.

To override a specific JDBC driver, drop the corresponding jar file into the <ESE-Home>/drivers/jdbc directory, so that the driver can be loaded at extension start.

The included MariaDB java connector driver can only be overwritten by the MySQL Connector/J driver. Files that contain drivers other than PostgreSQL JDBC Driver, Microsoft JDBC Driver for SQL Server and MySQL Connector/J are ignored.

The following table offers links to drivers and additional information:

Table 35. JDBC drivers
Database Download Link Minimum Version

PostgreSQL and Amazon Aurora

Download link

JDBC 4.2

MySQL, MariaDB, and Amazon Aurora

Download link

Connector/J 8.0

MSSQL and Azure SQL

Download link

JDB Driver 7.2

SQL Database Setup

For the ESE to work properly, the SQL database has to have a certain schema. This structure is described in detail below.

To facilitate database setup, the ESE provides an SQL setup snippet for every supported SQL database. This snippet creates a database with all the required tables and columns. The SQL setup snippet can be found in the <ESE-Home>/sql/<ESE-Version> folder as a text file. By copying the content and inserting it into your database management system or using it directly as SQL script it generates the needed structure automatically.

Once the necessary tables are created, use the usual SQL commands to fill the tables with the data of your clients as desired.

To help you store the passwords in an appropriate format, the ESE provides a CLI tool. For example, use the CLI tool to create SQL INSERT statements for the 'users' table and ensure that all passwords are stored in a way that the ESE can process. Of course, you can process passwords and store them in the database without the provided CLI tool. The cryptography section explains how the ESE processes passwords. Always ensure that the passwords are stored in a way so that the ESE can validate them.

If you already have a supported SQL database, simply verify that your data is stored in a way that the ESE can work with. If you do not have a matching SQL structure, this SQL view documentation shows a way to create a database view that is in accordance with the required database structure of the ESE.

SQL Database Structure

Entity Relation Diagram - HiveMQ Enterprise Security Extension that Supports SQL Databases

If you use the provided sql scripts to set up your database in version 1.2 or later, two additional columns are added to each table: created_at and updated_at. These columns are of the appropriate database-specific data type for storing a uniquely identifiable timestamp. The entry created_at is always set to the time the database record was created. The entry updated_at is always set to the time of the last change to the database record. There is no need for you to interact with either created_at or updated_at manually.
Table 36. Table "users"
Column Data Type Description

username

Text

Primary Key, Unique, Not Null

password

Text

Base64 Encoded raw byte array

password_iterations

Integer

Not Null

password_salt

Text

Not Null

algorithm

Text

Not Null
Table 37. Table "roles"
Column Data Type Description

id

Serial

Primary Key, Unique, Not Null

name

Text

Unique, Not Null

description

Text
Table 38. Table "user_roles"
Column Data Type Description

user_id

Integer

Primary Key

role_id

Integer

Primary Key
Table 39. Table "permissions"
Column Data Type Description

id

Serial

Primary Key

topic

Text

Not Null

publish_allowed

Boolean

Default false, Not Null

subscribe_allowed

Boolean

Default false, Not Null

qos_0_allowed

Boolean

Default false, Not Null

qos_1_allowed

Boolean

Default false, Not Null

qos_2_allowed

Boolean

Default false, Not Null

retained_msgs_allowed

Boolean

Default false, Not Null

shared_sub_allowed

Boolean

Default false, Not Null

shared_group

Text
Table 40. Table "role_permissions"
Column Data type Description

role

Integer

Primary Key, Not Null

permission

Integer

Primary Key, Not Null
Table 41. Table "user_permissions"
Column Data Type Description

user_id

Integer

Primary Key, Not Null

permission

Integer

Primary Key, Not Null

SQL Database Structure for Control Center Access Control

The ESE-access control feature for your control center requires additional tables in your SQL database. If you only want to authenticate or authorize MQTT clients, no additional tables are required.

Entity Relation Diagram - HiveMQ Enterprise Security Extension that Supports SQL Databases

If you use the provided sql scripts to set up your database in version 1.2 or later, two additional columns are added to each table: created_at and updated_at. These columns are of the appropriate database-specific data type for storing a uniquely-identifiable timestamp. The entry created_at is always set to the time that the database record was created. The entry updated_at is always set to the time of the last change to the database record. There is no need for you to interact with either created_at or updated_at manually.
Table 42. Table "cc_users"
Column Data Type Description

username

Text

Primary Key, Unique, Not Null

password

Text

Base64 Encoded raw byte array

password_iterations

Integer

Not Null

password_salt

Text

Not Null

algorithm

Text

Not Null
Table 43. Table "cc_roles"
Column Data Type Description

id

Serial

Primary Key, Unique, Not Null

name

Text

Unique, Not Null

description

Text
Table 44. Table "cc_user_roles"
Column Data Type Description

user_id

Integer

Primary Key

role_id

Integer

Primary Key
Table 45. Table "cc_permissions"
Column Data Type Description

id

Serial

Primary Key

topic_string

Text

Not Null

description

Text
Table 46. Table "cc_role_permissions"
Column Data type Description

role

Integer

Primary Key, Not Null

permission

Integer

Primary Key, Not Null
Table 47. Table "cc_user_permissions"
Column Data Type Description

user_id

Integer

Primary Key, Not Null

permission

Integer

Primary Key, Not Null

A script to insert all supported HiveMQ permissions is provided in the sql folder or you can copy and paste the provided SQL insert script.

SQL Database Structure for REST API Access Control

The ESE-access control feature for your REST API requires additional tables in your SQL database. If you only want to authenticate or authorize MQTT clients, no additional tables are required.

Entity Relation Diagram - HiveMQ Enterprise Security Extension that Supports SQL Databases

If you use the provided sql scripts to set up your database in version 1.2 or later, two additional columns are added to each table: created_at and updated_at. These columns are of the appropriate database-specific data type for storing a uniquely-identifiable timestamp. The entry created_at is always set to the time that the database record was created. The entry updated_at is always set to the time of the last change to the database record. There is no need for you to interact with either created_at or updated_at manually.
Table 48. Table "rest_api_users"
Column Data Type Description

username

Text

Primary Key, Unique, Not Null

password

Text

Base64 Encoded raw byte array

password_iterations

Integer

Not Null

password_salt

Text

Not Null

algorithm

Text

Not Null
Table 49. Table "rest_api_roles"
Column Data Type Description

id

Serial

Primary Key, Unique, Not Null

name

Text

Unique, Not Null

description

Text
Table 50. Table "rest_api_user_roles"
Column Data Type Description

user_id

Integer

Primary Key

role_id

Integer

Primary Key
Table 51. Table "rest_api_permissions"
Column Data Type Description

id

Serial

Primary Key

topic_string

Text

Not Null

description

Text
Table 52. Table "rest_api_role_permissions"
Column Data type Description

role

Integer

Primary Key, Not Null

permission

Integer

Primary Key, Not Null
Table 53. Table "rest_api_user_permissions"
Column Data Type Description

user_id

Integer

Primary Key, Not Null

permission

Integer

Primary Key, Not Null

A script to insert all supported HiveMQ permissions is provided in the sql folder or you can copy and paste the provided SQL insert script.

SQL INSERT HiveMQ Control-Center and REST API Permissions
-- insert the default permissions strings that are supported by HiveMQ into the cc_permissions
insert into cc_permissions (permission_string, description)
values ('HIVEMQ_SUPER_ADMIN', 'special cc_permission, that allows access to everything'),
       ('HIVEMQ_VIEW_PAGE_CLIENT_LIST', 'allowed to view client list'),
       ('HIVEMQ_VIEW_PAGE_CLIENT_DETAIL', 'allowed to view client detail'),
       ('HIVEMQ_VIEW_PAGE_LICENSE', 'allowed to view license page'),
       ('HIVEMQ_VIEW_DATA_ACTIVE_LICENSE', 'allowed to view active license data'),
       ('HIVEMQ_VIEW_PAGE_TRACE_RECORDINGS', 'allowed to view trace recording page'),
       ('HIVEMQ_VIEW_PAGE_DROPPED_MESSAGES', 'allowed to dropped message page'),
       ('HIVEMQ_VIEW_PAGE_BACKUP', 'allowed to view backup page'),
       ('HIVEMQ_VIEW_PAGE_SUPPORT', 'allowed to view support page'),
       ('HIVEMQ_VIEW_DATA_CLIENT_ID', 'allowed to see client identifiers'),
       ('HIVEMQ_VIEW_DATA_IP', 'allowed to see client''s IPs'),
       ('HIVEMQ_VIEW_DATA_PAYLOAD', 'allowed to see message payloads'),
       ('HIVEMQ_VIEW_DATA_PASSWORD', 'allowed to see client''s passwords'),
       ('HIVEMQ_VIEW_DATA_USERNAME', 'allowed to see client''s usernames'),
       ('HIVEMQ_VIEW_DATA_WILL_MESSAGE', 'allowed to see client''s LWT'),
       ('HIVEMQ_VIEW_DATA_TOPIC', 'allowed to see client''s topics'),
       ('HIVEMQ_VIEW_DATA_SUBSCRIPTION', 'allowed to see client''s subscriptions'),
       ('HIVEMQ_VIEW_DATA_PROXY', 'allowed to see client''s proxy information'),
       ('HIVEMQ_VIEW_DATA_TLS', 'allowed to see client''s TLS information'),
       ('HIVEMQ_VIEW_PAGE_KAFKA_DASHBOARD', 'allowed to view the HiveMQ Extension for Kafka page'),
       ('HIVEMQ_VIEW_PAGE_RETAINED_MESSAGE_LIST', 'allowed to view retained message list'),
       ('HIVEMQ_VIEW_PAGE_RETAINED_MESSAGE_DETAIL', 'allowed to view retained message details'),
       ('HIVEMQ_VIEW_DATA_USER_PROPERTIES', 'allowed to see user properties of messages'),
       ('HIVEMQ_VIEW_DATA_SESSION_ATTRIBUTES', 'allowed to see session attributes of clients'),
       ('HIVEMQ_VIEW_DATA_CLIENT_EVENT_HISTORY', 'allowed to see the event history of clients'),
       ('HIVEMQ_VIEW_DATA_CLUSTER_NODES', 'allowed to see granular node information'),
       ('HIVEMQ_VIEW_PAGE_SHARED_SUBSCRIPTION_LIST', 'allowed to view shared subscription list'),
       ('HIVEMQ_VIEW_PAGE_SHARED_SUBSCRIPTION_DETAIL', 'allowed to view shared subscription details'),
       ('HIVEMQ_VIEW_PAGE_EXTENSION_CONSUMER_LIST', 'allowed to view extension consumer list'),
       ('HIVEMQ_VIEW_PAGE_SCHEMAS_LIST', 'allowed to view schema list'),
       ('HIVEMQ_VIEW_PAGE_SCHEMA_DETAIL', 'allowed to view schema detail'),
       ('HIVEMQ_EDIT_DATA_SCHEMA', 'allowed to create/edit schemas'),
       ('HIVEMQ_VIEW_PAGE_DATA_POLICIES_LIST', 'allowed to view data policy list'),
       ('HIVEMQ_VIEW_PAGE_BEHAVIOR_POLICIES_LIST', 'allowed to view behavior policy list'),
       ('HIVEMQ_VIEW_PAGE_DATA_POLICY_DETAIL', 'allowed to view data policy detail'),
       ('HIVEMQ_VIEW_PAGE_BEHAVIOR_POLICY_DETAIL', 'allowed to view behavior policy detail'),
       ('HIVEMQ_VIEW_PAGE_DATA_HUB_CHARTS', 'allowed to view data hub charts'),
       ('HIVEMQ_EDIT_DATA_DATA_POLICY', 'allowed to create/edit data policies'),
       ('HIVEMQ_EDIT_DATA_BEHAVIOR_POLICY', 'allowed to create/edit behavior policies'),
       ('HIVEMQ_VIEW_PAGE_SCRIPTS_LIST', 'allowed to view script list'),
       ('HIVEMQ_VIEW_PAGE_SCRIPT_DETAIL', 'allowed to view script detail'),
       ('HIVEMQ_EDIT_DATA_SCRIPT', 'allowed to create/edit scripts')
;
-- insert the default permissions strings that are supported by HiveMQ into the rest_api_permissions
insert into rest_api_permissions (permission_string, description)
values ('HIVEMQ_SUPER_ADMIN', 'special rest_api_permission, that allows access to everything'),
       ('HIVEMQ_MANAGEMENT_BACKUPS_GET', 'allows GET on /api/v1/management/backups'),
       ('HIVEMQ_MANAGEMENT_BACKUPS_POST', 'allows POST on /api/v1/management/backups'),
       ('HIVEMQ_MANAGEMENT_BACKUPS_BACKUPID_GET', 'allows GET on /api/v1/management/backups/{backupId}'),
       ('HIVEMQ_MANAGEMENT_BACKUPS_BACKUPID_POST', 'allows POST on /api/v1/management/backups/{backupId}'),
       ('HIVEMQ_MANAGEMENT_TRACE_RECORDINGS_GET', 'allows GET on /api/v1/management/trace-recordings'),
       ('HIVEMQ_MANAGEMENT_TRACE_RECORDINGS_POST', 'allows POST on /api/v1/management/trace-recordings'),
       ('HIVEMQ_MANAGEMENT_TRACE_RECORDINGS_TRACERECORDINGID_PATCH', 'allows PATCH on /api/v1/management/trace-recordings/{traceRecordingId}'),
       ('HIVEMQ_MANAGEMENT_TRACE_RECORDINGS_TRACERECORDINGID_DELETE', 'allows DELETE on /api/v1/management/trace-recordings/{traceRecordingId}'),
       ('HIVEMQ_MANAGEMENT_FILES_TRACE_RECORDINGS_TRACERECORDINGID_GET', 'allows GET on /api/v1/management/files/trace-recordings/{traceRecordingId}'),
       ('HIVEMQ_MANAGEMENT_FILES_BACKUPS_BACKUPID_GET', 'allows GET on /api/v1/management/files/backups/{backupId}'),
       ('HIVEMQ_MANAGEMENT_DIAGNOSTIC_ARCHIVES_POST', 'allows POST on /api/v1/management/diagnostic-archives'),
       ('HIVEMQ_MQTT_CLIENTS_CLIENTID_SUBSCRIPTIONS_GET', 'allows GET on /api/v1/mqtt/clients/{clientId}/subscriptions'),
       ('HIVEMQ_MQTT_CLIENTS_GET', 'allows GET on /api/v1/mqtt/clients'),
       ('HIVEMQ_MQTT_CLIENTS_CLIENTID_GET', 'allows GET on /api/v1/mqtt/clients/{clientId}'),
       ('HIVEMQ_MQTT_CLIENTS_CLIENTID_DELETE', 'allows DELETE on /api/v1/mqtt/clients/{clientId}'),
       ('HIVEMQ_MQTT_CLIENTS_CLIENTID_CONNECTION_GET', 'allows GET on /api/v1/mqtt/clients/{clientId}/connection'),
       ('HIVEMQ_MQTT_CLIENTS_CLIENTID_CONNECTION_DELETE', 'allows DELETE on /api/v1/mqtt/clients/{clientId}/connection'),

       ('HIVEMQ_DATA_HUB_SCHEMAS_POST', 'allows POST on /api/v1/data-hub/schemas'),
       ('HIVEMQ_DATA_HUB_SCHEMAS_SCHEMAID_GET', 'allows GET on /api/v1/data-hub/schemas/{schemaId}'),
       ('HIVEMQ_DATA_HUB_SCHEMAS_SCHEMAID_DELETE', 'allows DELETE on /api/v1/data-hub/schemas/{schemaId}'),
       ('HIVEMQ_DATA_HUB_SCHEMAS_GET', 'allows GET on /api/v1/data-hub/schemas'),
       ('HIVEMQ_DATA_HUB_SCRIPTS_POST', 'allows POST on /api/v1/data-hub/scripts'),
       ('HIVEMQ_DATA_HUB_SCRIPTS_SCRIPTID_GET', 'allows GET on /api/v1/data-hub/scripts/{scriptId}'),
       ('HIVEMQ_DATA_HUB_SCRIPTS_SCRIPTID_DELETE', 'allows DELETE on /api/v1/data-hub/scripts/{scriptId}'),
       ('HIVEMQ_DATA_HUB_SCRIPTS_GET', 'allows GET on /api/v1/data-hub/scripts'),
       ('HIVEMQ_DATA_HUB_DATA_VALIDATION_POLICIES_POST', 'allows POST on /api/v1/data-hub/data-validation/policies'),
       ('HIVEMQ_DATA_HUB_DATA_VALIDATION_POLICIES_POLICYID_GET', 'allows GET on /api/v1/data-hub/data-validation/policies/{policyId}'),
       ('HIVEMQ_DATA_HUB_DATA_VALIDATION_POLICIES_POLICYID_DELETE', 'allows DELETE on /api/v1/data-hub/data-validation/policies/{policyId}'),
       ('HIVEMQ_DATA_HUB_DATA_VALIDATION_POLICIES_GET', 'allows GET on /api/v1/data-hub/data-validation/policies'),
       ('HIVEMQ_DATA_HUB_DATA_VALIDATION_POLICIES_POLICYID_PUT', 'allows PUT on /api/v1/data-hub/data-validation/policies/{policyId}'),
       ('HIVEMQ_DATA_HUB_BEHAVIOR_VALIDATION_POLICIES_POST', 'allows POST on /api/v1/data-hub/behavior-validation/policies'),
       ('HIVEMQ_DATA_HUB_BEHAVIOR_VALIDATION_POLICIES_POLICYID_GET', 'allows GET on /api/v1/data-hub/behavior-validation/policies/{policyId}'),
       ('HIVEMQ_DATA_HUB_BEHAVIOR_VALIDATION_POLICIES_POLICYID_DELETE', 'allows DELETE on /api/v1/data-hub/behavior-validation/policies/{policyId}'),
       ('HIVEMQ_DATA_HUB_BEHAVIOR_VALIDATION_POLICIES_GET', 'allows GET on /api/v1/data-hub/behavior-validation/policies'),
       ('HIVEMQ_DATA_HUB_BEHAVIOR_VALIDATION_POLICIES_POLICYID_PUT', 'allows PUT on /api/v1/data-hub/behavior-validation/policies/{policyId}'),
       ('HIVEMQ_DATA_HUB_BEHAVIOR_VALIDATION_STATES_CLIENTID_GET', 'allows GET on /api/v1/data-hub/behavior-validation/states/{clientId}'),
       ('HIVEMQ_DATA_HUB_MANAGEMENT_START_TRIAL_POST', 'allows POST on /api/v1/data-hub/management/start-trial')
;
Supported SQL database versions
Table 54. Supported SQL database versions
SQL Database Supported Versions

MySQL

  • 5.7

  • 8.0

MSSQL

  • 2017-CU12

  • 2019-CTP3.1

PostgreSQL

  • 10

  • 11

  • 12

  • 13

  • 14

MariaDB

  • 10.3

  • 10.4

  • 10.5

  • 10.6

  • 10.7

  • 10.8

Amazon Aurora

  • Amazon Aurora with MySQL compatibility (see our supported versions for MySQL)

  • Amazon Aurora with PostgreSQL compatibility (see our supported versions for PostgreSQL)

Azure SQL

  • See our supported versions for MSSQL

File-based

The HiveMQ Enterprise Security Extension supports files as external sources of authentication and authorization data.

In the ESE configuration file, you can use the abstract representation of File realms to reference these files. They need to be present on each HiveMQ node or utilized from a centralized location such as an NFS mount.

File realms can be referenced in File authentication managers and/or File authorization managers.

To enable the ESE to work with the stored information, the file must follow the described structure.

File Realm

To add file-based content to an ESE authentication/authorization pipeline, configure a File realm in the <realms> section of your ESE configuration.

You can utilize ESE File realms by referencing them by the name in file authentication managers and/or file authorization managers.

Example File realm configuration
<file-realm>
    <name>file-backend-one</name>
    <enabled>true</enabled>
    <configuration>
        <file-path>/my/dir/my_hivemq_conf.xml</file-path>
    </configuration>
</file-realm>
Table 55. File realm parameters
Parameter Type Description

name

String

Specifies the name of the File realm

enabled

Boolean

Specifies whether the File realm is enabled

file-path

String

Specifies the path to the file
The path to the file can be absolute or relative. Relative paths are relative to the ESE home directory.

File Authentication Manager

To use a file as an external source for authentication, configure a File authentication manager in the desired pipeline and reference the corresponding File realm name in the authentication manager.

To ensure that the ESE can find the authentication information in the file, the layout of the file must be in accordance with the file structure.

The authentication manager sets the authorization variables authorization-key and authorization-role-key too if authorization information is present in the file.

Example File Authentication Manager configuration
<enterprise-security-extension>

    <!-- add realms configuration -->

    <pipelines>
        <listener-pipeline listener="ALL">
            <!-- add authentication preprocessors if necessary -->

            <file-authentication-manager>
                <realm>file-realm-name</realm>
            </file-authentication-manager>

            <!-- add authorization preprocessors if necessary -->
            <!-- add an authorization manager -->
        </listener-pipeline>
    </pipelines>

</enterprise-security-extension>
Table 56. File authentication manager parameters
Parameter Type Mandatory Description

realm

String

References the File realm from which the authentication data is drawn.

File Authorization Manager

The ESE File Authorization Manager gives you the ability to assign permissions for specific users and roles.

To use file-based authorization, configure a File Authorization Manager in the desired pipeline and reference the name of the corresponding File realm in the configuration of the authorization manager.

File Authorization Manager Configuration Options

The configuration of your authorization manager defines whether you use roles and/or user-specific permissions.
Based on the configuration, the ESE uses different authorization variables. These variables can be set and manipulated with a preprocessor before the authorization manager.

Table 57. File authorization manager parameters
Parameter Type Mandatory / Default Description

realm

String

References the File realm from which the authentication data is drawn.

use-authorization-key

Boolean

true

If this parameter is set to true, the ESE uses user-specific permissions.

use-authorization-role-key

Boolean

true

If this parameter is set to true, the ESE uses role permissions.

File Role Permissions

To use role-based permissions in a File Authorization Manager, set the <use-authorization-role-key> parameter to true. The variable authorization-role-key must be set, so that the ESE can query the corresponding role and its permissions from your configuration file.

Example File Authorization Manager configuration for role-based permissions
<enterprise-security-extension>

    <!-- add realms configuration -->

    <pipelines>
        <listener-pipeline listener="ALL">
            <!-- add authentication preprocessors if necessary -->
            <!-- add an authentication manager -->
            <!-- add authorization preprocessors if necessary -->
            <file-authorization-manager>
                <realm>file-backend-one</realm>
                <use-authorization-key>false</use-authorization-key>
                <use-authorization-role-key>true</use-authorization-role-key>
            </file-authorization-manager>

        </listener-pipeline>
    </pipelines>

</enterprise-security-extension>
For ease of maintenance, we recommend that you use role-based permissions rather than user-specific permissions.

File User-specific Permissions

To use user-specific permissions in a File Authorization Manager, set the <use-authorization-key> parameter to true. The variable authorization-key must be set, so that the ESE can query the corresponding user-specific permissions from your configuration file.

Example File Authorization Manager configuration for user-specific permissions
<enterprise-security-extension>

    <!-- add realms configuration -->

    <pipelines>
        <listener-pipeline listener="ALL">
            <!-- add  authentication preprocessors if necessary -->
            <!-- add  authentication manager -->
            <!-- add  authorization preprocessors if necessary -->

            <file-authorization-manager>
                <realm>example-backend</realm>
                <use-authorization-key>true</use-authorization-key>
                <use-authorization-role-key>false</use-authorization-role-key>
            </file-authorization-manager>

        </listener-pipeline>
    </pipelines>

</enterprise-security-extension>

File Realm File

The ESE File realm file contains a list of users and roles for authentication and authorization. The following use-cases can be configured in any combination:

  • For authentication with a file authentication manager you can define users with password and optionally assigned roles.

  • For authorization with a file authorization manager and user specific permissions you can define users with assigned permissions.

  • For authorization with a file authorization manager and role specific permissions you can define roles with assigned permissions.

To have a complete file-based setup, you can specify users with password and the matching roles with assigned permissions in the same file.

The file is divided into several sections:

  • The <mqtt> section defines the users, roles, and permissions for your MQTT clients. The <mqtt> section is relevant when you use the File Authentication Manager and/or the File Authorization Manager in a listener-pipeline of your ESE configuration.

  • The <control-center> section defines the users, roles, and permissions for your HiveMQ control center. The <control-center> section is relevant when you use the File Authentication Manager and/or the File Authorization Manager in a control center pipeline. For more information, see Control Center Access Control.

  • The <rest-api> section defines the users, roles, and permissions for your HiveMQ REST API. The <rest-api> section is relevant when you use the File Authentication Manager and/or the File Authorization Manager in a REST API pipeline. For more information, see REST API Access Control.

Users with a configured password will be used for authentication and can be optionally assigned to roles for later authorization.

When you assign a role to a user, the user receives all permissions that are associated with the role. The use of role-based permissions can simplify access management.

Alternatively, you can assign permissions for authorization to a user directly.

If duplicate users or roles are present in the file, ESE uses the first instance of the user or role, ignores subsequent duplicates, and prints a warning to your HiveMQ log file (DEBUG level).

The Enterprise Security Extension monitors your file for configuration changes and reloads the file during runtime when changes are detected (hot-reload). If the new configuration is invalid, ESE continues to use the most recent valid configuration and prints the following error message to your HiveMQ log file: FileRealm {} failed to reload auth file {}. Using previous configuration.

Changes that you make in your configuration file do not affect clients that are already connected. The changes only apply to clients that connect after ESE loads the changes.
Variable Substitution

In your file, you can use ESE Variables to substitute string values in your permissions. The syntax for the permission placeholders is ${var}. For more information, see ESE Variables.

Example configuration
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<ese-file-realm>
    <mqtt>
        <users>
            <user>
                <name>mqtt-user-1</name>
                <permissions>
                    <permission>
                        <topic>data/${mqtt-clientid}/#</topic>
                        <qos>ALL</qos>
                        <activity>ALL</activity>
                    </permission>
                </permissions>
            </user>
            <user>
                <name>mqtt-user-2</name>
                <password encoding="Base64">iYwiemNkYxaa5mVEMl36hRjBG5IeXuy652uehvL9lJM=</password>
                <algorithm>PKCS5S2</algorithm>
                <iterations>10</iterations>
                <salt>my-salt</salt>
                <roles>
                    <role>mqtt-role-1</role>
                </roles>
            </user>
        </users>
        <roles>
            <role>
                <id>mqtt-role-1</id>
                <permissions>
                    <permission>
                        <topic>incoming/${authorization-key}/actions</topic>
                        <activity>SUBSCRIBE</activity>
                        <qos>ZERO_ONE</qos>
                        <retain>NOT_RETAINED</retain>
                        <shared-subscription>SHARED</shared-subscription>
                        <shared-group>group1</shared-group>
                    </permission>
                </permissions>
            </role>
        </roles>
    </mqtt>
    <control-center>
        <users>
            <user>
                <name>cc-user-1</name>
                <permissions>
                    <permission>HIVEMQ_VIEW_DATA_TOPIC</permission>
                </permissions>
            </user>
            <user>
                <name>cc-user-2</name>
                <password encoding="Base64">iYwiemNkYxaa5mVEMl36hRjBG5IeXuy652uehvL9lJM=</password>
                <algorithm>PKCS5S2</algorithm>
                <iterations>10</iterations>
                <salt>my-salt</salt>
                <roles>
                    <role>cc-role-1</role>
                </roles>
            </user>
        </users>
        <roles>
            <role>
                <id>cc-role-1</id>
                <permissions>
                    <permission>HIVEMQ_VIEW_DATA_SUBSCRIPTION</permission>
                </permissions>
            </role>
        </roles>
    </control-center>
    <rest-api>
        <users>
            <user>
                <name>rest-api-user</name>
                <password>rest-api-user-password</password>
                <roles>
                    <role>subscriptions</role>
                </roles>
            </user>
        </users>
        <roles>
            <role>
                <id>subscriptions</id>
                <permissions>
                    <permission>HIVEMQ_MQTT_CLIENTS_CLIENTID_SUBSCRIPTIONS_GET</permission>
                </permissions>
            </role>
        </roles>
    </rest-api>
</ese-file-realm>

The file provides the users, roles, and permission information that the file authentication manager and the file authorization manager utilize.

MQTT Client

Table 58. ESE File realm MQTT client user configuration options
Element Type Mandatory/Default AuthN AuthZ Description

mqtt/users/user/name

String

The username.

mqtt/users/user/password

String

The password.

mqtt/users/user/password/@encoding

XML enum

UTF8

The password encoding. Possible values are UTF8, Base64, and Hex.

mqtt/users/user/iterations

Integer

0

The number of times the password hashing algorithm is applied to the password.

mqtt/users/user/salt

String

The unique, randomly generated characters that are added to the password before the password is hashed.

mqtt/users/user/salt/@encoding

XML enum

UTF8

The encoding used for the password hashing salt. Possible values are UTF8, Base64, and Hex.

mqtt/users/user/algorithm

XML enum

PLAIN

The password hashing algorithm. Possible values are PLAIN, MD5, SHA512, BCRYPT, and PKCS5S2.

mqtt/users/user/roles/role

String

The referenced role.

mqtt/users/user/permissions/permission

Complex

The permission that is assigned to the user.
Table 59. ESE File realm MQTT client authorization role configuration options
Element Type Mandatory/Default Description

mqtt/roles/role/id

String

The ID of the role.

mqtt/roles/role/permissions/permission

Complex

The permission that is assigned to the role.
Table 60. ESE File realm MQTT client authorization permission configuration options
Parameter Type Mandatory/Default Description

topic

String

Defines the topic to which the permission applies. The topic can contain any valid MQTT topic filter, including wildcards (+,#).

qos

String

Defines the MQTT QoS level with which the client can publish and subscribe on the topic. Possible values are: ZERO, ONE, TWO, ZERO_ONE, ONE_TWO, ZERO_TWO, or ALL.

activity

String

Defines which actions the client is permitted to do on the topic. Possible values are: PUBLISH, SUBSCRIBE, or ALL.

retain

String

NOT_RETAINED

Defines whether PUBLISH messages published on the topic are retained. Possible values are NOT_RETAINED, RETAINED, or ALL.

shared-subscription

String

NOT_SHARED

Defines whether subscriptions on the topic are shared subscriptions. Possible values are: SHARED, NOT_SHARED, or ALL. This setting is only relevant for SUBSCRIBE messages.

shared-group

String

“” (empty string)

Defines to which shared subscription group the permission applies. Possible values are: any string or # for all groups. This setting is only relevant for SUBSCRIBE messages that include a shared subscription.

Control Center

Table 61. ESE File realm Control Center user configuration options
Element Type Mandatory/Default AuthN AuthZ Description

control-center/users/user/name

String

The username.

control-center/users/user/password

String

The password.

control-center/users/user/password/@encoding

XML enum

UTF8

The password encoding. Possible values are: UTF8, Base64, and Hex.

control-center/users/user/iterations

Integer

0

The number of times the password hashing algorithm is applied to the password.

control-center/users/user/salt

String

The unique, randomly generated characters that are added to the password before the password is hashed.

control-center/users/user/salt/@encoding

XML enum

UTF8

The encoding used for the password hashing salt. Possible values are UTF8, Base64, and Hex.

control-center/users/user/algorithm

XML enum

PLAIN

The password hashing algorithm. Possible values are PLAIN, MD5, SHA512, BCRYPT, and PKCS5S2.

control-center/users/user/roles/role

String

The referenced role.

control-center/users/user/permissions/permission

String

The permission that is assigned to the user.
Table 62. ESE File realm Control Center authorization role configuration options
Element Type Mandatory/Default Description

control-center/roles/role/id

String

The ID of the role.

control-center/roles/role/permissions/permission

String

The permission that is assigned to the role.

REST API

Table 63. ESE File realm REST API user configuration options
Element Type Mandatory/Default AuthN AuthZ Description

rest-api/users/user/name

String

The username.

rest-api/users/user/password

String

The password.

rest-api/users/user/password/@encoding

XML enum

UTF8

The password encoding. Possible values are: UTF8, Base64, and Hex.

rest-api/users/user/iterations

Integer

0

The number of times the password hashing algorithm is applied to the password.

rest-api/users/user/salt

String

The unique, randomly generated characters that are added to the password before the password is hashed.

rest-api/users/user/salt/@encoding

XML enum

UTF8

The encoding used for the password hashing salt. Possible values are UTF8, Base64, and Hex.

rest-api/users/user/algorithm

XML enum

PLAIN

The password hashing algorithm. Possible values are PLAIN, MD5, SHA512, BCRYPT, and PKCS5S2.

rest-api/users/user/roles/role

String

The referenced role.

rest-api/users/user/permissions/permission

String

The permission that is assigned to the user.
Table 64. ESE File realm REST API authorization role configuration options
Element Type Mandatory/Default Description

rest-api/roles/role/id

String

The ID of the role.

rest-api/roles/role/permissions/permission

String

The permission that is assigned to the role.

OAuth 2.0 / JWT for MQTT and REST API Clients

The HiveMQ Enterprise Security Extension enables clients to authenticate via provided JSON Web Tokens (JWT). These tokens are frequently used for authentication and authorization in HTTP-based web applications. The HiveMQ Enterprise Security Extension makes it possible to integrate JWTs into your infrastructure.

JWTs are self-contained, digitally signed collections of JSON key-value pairs (claims). Claims are provided in the payload of the JWT and are used to assert information about the bearer of the token.

The JWT can contain two types of claims. You can configure the ESE authentication manager to validate one or more claims as needed to fulfill your use case:

  • Reserved claims: Statement name and value pairs with predefined names.

  • Custom claims: Statement name and value pairs that can be defined as desired.

Custom claims require specialized solutions to map or read and are more expensive to deserialize and validate.
Table 65. Reserved Claims
Reserved Claim Name Description

aud

The intended recipient of the JWT. This entry is the audience of the JWT.

sub

The name of the entity to which the JWT refers. This entry is the subject of the JWT.

iss

The name of the authority that provides the JWT. This entry is the issuer of the JWT.

scope

The actions for which the JWT is intended. The name of this entry can be adjusted with the alt attribute.
Example JWT payload
{
  "iss": "jwt-service",
  "sub": "my-mqtt-client",
  "aud": "hivemq.com",
  "iat": 1573567292,
  "exp": 1573653692,
  "scope": [
    "publish",
    "subscribe"
  ],
  "service": "mqtt-broker"
}

To add a trusted JWT provider, configure a <jwt-realm> in the <realms> section of your ESE configuration. The HiveMQ Enterprise Security Extension can work with many different kinds of JWT providers. Here are some of the popular JWT providers ESE supports:

  • Cloud solutions such as Okta or Auth0

  • Social identity providers such as Google or Github

  • On premise services such as Keycloak

  • Embedded solutions such as ORY Hydra

JWT Realm

The abstraction of a JWT provider in the ESE configuration is a jwt-realm. The JWT realm provides all of the information that the ESE needs to fetch the JSON Web Key Set (JWKS) and the optional introspection endpoint for the JWTs.

The JWKS is a set of public keys that the JWT provider issues for verification of the JWTs.

Introspection is an optional method to online verify the validity of a token, after it was issued.

Example JWT realm configuration
<enterprise-security-extension>
    <realms>
        <jwt-realm>
            <name>jwt-provider</name>
            <enabled>true</enabled>
            <configuration>
                <jwks-endpoint>https://jwt-service/keys</jwks-endpoint>
                <jwks-default-cache-duration>1800</jwks-default-cache-duration>
                <tls-trust-store>path/to/truststore.jks</tls-trust-store>
                <introspection-endpoint>https://jwt-service/token/introspect</introspection-endpoint>
                <simple-auth>
                    <username>ese</username>
                    <password>client-secret</password>
                </simple-auth>
            </configuration>
        </jwt-realm>
    </realms>
</enterprise-security-extension>
Table 66. JWT realm parameters
Parameter Type Mandatory Description

jwks-endpoint

URI

Specifies the HTTP (GET) endpoint where the JWKS of the provider is located.

jwks-default-cache-duration

Integer

If this information is not specified by the provider in the HTTP response, this entry configures the duration that the JWKS is cached in seconds.

tls-trust-store

Trust Store

If the default system-wide TLS trust store is not used, this entry specifies an alternative trust store.

introspection-endpoint

URI

Specifies the HTTP (POST) endpoint where the JWT of the client is sent for further validation as specified in RFC-7662.

simple-auth

Simple Auth

Defines the HTTP Simple Auth parameters that are used for introspection requests from ESE to the identity server.
TLS Trust Store
<tls-trust-store password="secret">path/to/truststore.jks</tls-trust-store>
Parameter Type Mandatory Description

XML Value

File Path

Path where the trust store is located.

password

String

Password to open the truststore.
Simple Auth
<simple-auth>
    <username>ese</username>
    <password>client-secret</password>
</simple-auth>
Parameter Type Mandatory Description

username

String

The username portion of the HTTP Simple Auth. This string is usually the name of the OAuth realm.

password

String

The password portion of the HTTP Simple Auth. This string is usually the OAuth client secret.

JWT Authentication Manager

JWT is an open standard (RFC 7519) that defines a compact and self-contained way to securely transmit information as a JSON object. The information in the digitally signed JWT can be trusted and verified.

To implement custom logic for authentication with JWTs, you must configure a JWT authentication manager in your ESE configuration.

Example JWT Authentication Manager configuration
<jwt-authentication-manager>
    <realm>jwt-backend</realm>
    <!-- every tag in the configuration must contain a value. All tags except the realm name and custom claims have a default value. -->
    <jwt-validation>
        <!-- default: no disconnect after expiration date -->
        <exp-grace disconnect-after-expiry="true">300</exp-grace>
        <!-- default: false -->
        <enable-introspection sampling="0.5">true</enable-introspection>
        <reserved-claims>
            <aud>hivemq</aud>
            <sub>${mqtt-clientid}</sub>
            <iss>jwt-service</iss>
            <!-- scope is an OpendID Connect standard not a JWT standard -->
            <scope alt="scp">publish subscribe</scope>
        </reserved-claims>
        <!-- only use direct match string values -->
        <custom-claims>
            <claim>
                <name>defined-name</name>
                <value>defined-value</value>
            </claim>
        </custom-claims>
    </jwt-validation>
</jwt-authentication-manager>
Table 67. JWT Authentication Manager parameters
Parameter Type Default Value Description

realm

Name reference

*

Defines the JWT realm that the authentication manager accesses. * The realm parameter is mandatory and does not have a default value.

exp-grace

Integer

300

Defines the number of seconds before the Issued At (iat) and after the Expired (exp) in which the CONNECT can occur and the claims remain valid. A token presented outside the time interval iat - exp-grace and exp + exp-grace fails verification. The iat and exp are values in the JWT payload that the client sends in the MQTT CONNECT packet at the beginning of the MQTT connection or in the bearer token of the REST API HTTP request.

disconnect-after-expiry

Boolean (XML Attribute)

false

Defines whether the ESE automatically disconnects the client when the token that the client bears expires. This setting is only supported for MQTT client connections.

enable-introspection

Boolean

false

Defines whether the ESE checks with the identity server that issued the JWT for an additional validation of the token after successful local validation.

sampling

Float [0.0 .. 1.0] (XML Attribute)

1.0

Sets the probability that ESE selects a token for introspection.

reserved-claims

Reserved Claims

Shows the reserved claims that are validated.

custom-claims

Custom Claims

Shows the custom claims that are validated.

The JWT authentication manager can contain various validations. By default, every JWT authentication manager includes the following validations:

  • Signature

  • Expiration date

  • Issue date

To successfully complete a JWT authentication flow in the ESE, the provided JWT must pass all configured and default validations. Once validation is successful, the client can proceed to the authorization step of the pipeline.

The JWT authentication manager expects the token to be present in the authentication-byte-secret ESE variable.

Automatic MQTT Client Disconnection

The ESE observes the expiration date of the JWT token. If desired, you can configure the ESE to automatically disconnect MQTT clients when the JWT that the client bears expires. When the JWT token of a client expires the ESE disconnects the client.

You can also configure an expiration grace period in the JWT authentication manager of your ESE. When you configure an expiration grace period, the ESE disconnects the client after the grace period expires.

If you do not configure the ESE to disconnect clients when the JWT expires, clients with an expired token can remain connected.

If a client connects with an additional authentication method, the ESE respects that method even if the JWT expires.

Validation of Claims

The payload of JSON Web Tokens consists of various claims, which are key-value pairs. ESE can validate the claims against static configured values or values dynamically generated from ESE variables with the permission placeholders mechanic.

For example, the <sub>${mqtt-clientid}</sub> line in the configuration example tells the ESE to compare the value of the subject claim (sub) with the value of the clientID in the MQTT CONNECT packet.

Reserved Claims

Example Reserved Claims configuration
<reserved-claims>
    <aud>hivemq</aud>
    <sub>${mqtt-clientid}</sub>
    <iss>jwt-service other-jwt-service</iss>
    <scope alt="scp">publish subscribe</scope>
</reserved-claims>
Table 68. Reserved claims parameters
Claim Type Semantic

aud

String

The audience (aud) claim identifies the intended recipients of the JWT. A single token can have multiple audiences. The validation configuration of the JWT authentication manager must include one of the audiences that is listed in the claim.

sub

String

The subject (sub) claim identifies the bearer of the token. The bearer is the entity the token was issued to. The sub claim must be unique in the token and an exact match to the validation configuration of the JWT authentication manager.

iss

List of Strings

The issuer (iss) claim identifies the service that created and signed the token. Multiple instances can be configured. Only one of them must match the one in the token, which itself must be unique.

scope

List of Strings

The scope claim specifies the intended use of the token. All scope values that are defined in the validation configuration of the JWT authentication manager must be present in the token. If desired, you can use the alt attribute to rename the scope key. Different token providers use different names for this key. For example, scp.
The JWT authentication manager of the ESE uses information from the scope claim to populate the authorization-role-key. Information from the sub claim is used to populate the authorization-key ESE Variables.

Custom Claims

ESE can validate information from custom claims. Custom claims can contain a single string value of your choice. Permission placeholder substitution is performed before validation.

Example Custom Claims configuration
<custom-claims>
    <claim>
        <name>service</name>
        <value>mqtt-broker</value>
    </claim>
    <claim>
        <name>userName</name>
        <value>${mqtt-username}</value>
    </claim>
</custom-claims>
Table 69. Custom claim parameters
Parameter Type Semantic

name

String

The name (key) of the custom claim.

value

String

The value of the custom claim.
The ESE only requires a JWT Bearer Token. Do not put secret information in the JWT or share Identity or Refresh tokens. Although the information in the JWT cannot be altered without detection, a third party can read the information that the JWT contains.

MQTT JWT Authentication Flow in HiveMQ Enterprise Security Extension

JWTs are frequently generated and consumed in authentication flows. Because these flows require HTTP mechanics, it is not possible to implement a JWT authentication directly in MQTT. Before an MQTT client starts a JWT authentication flow with the ESE, the client must acquire a token from the JWT provider of your choice.

JWT Authentication Flow in the ESE

ESE JWT Authentication Flow

  1. The HiveMQ ESE gets the JSON Web Key Set (JWKS) from the JWT provider that is defined in the JWT realm of your ESE configuration.

  2. The MQTT client gets a JWT from the selected JWT provider.

  3. The MQTT client sends a CONNECT packet to HiveMQ with the provided JWT in the password field.

  4. The HiveMQ ESE verifies the claims in the JWT that the MQTT client provided.

  5. The HiveMQ ESE validates the digital signature of the JWT against the JWKS from the JWT provider.

  6. Optionally, HiveMQ introspects the JWT against the JWT provider.

  7. Successfully authenticated MQTT clients proceed to the authorization stage of the HiveMQ ESE pipeline.

  8. After successful authorization processing, HiveMQ ESE sends the MQTT client a CONNACK packet.

If the authentication fails, the MQTT client is not allowed to connect. The HiveMQ ESE sends a CONNACK with an error code and closes the TCP connection.

OAuth 2.0 / JWT for HiveMQ Control Center Login (SSO)

The HiveMQ Enterprise Security Extension makes it possible to integrate your HiveMQ Control Center into an existing OAuth 2.0 framework to provide enterprise-wide single sign-on (SSO) capabilities. Single sign-on refers to the ability for users to log in one time with one set of credentials to access services on multiple software systems. Providing SSO for your HiveMQ Control Center users can provide greater security and compliance as well as improved usability and user satisfaction.

See full examples for auth0 and Okta.

OAuth Realm

To add Oauth-based authentication with JWT tokens for HiveMQ Control Center users to an ESE authentication pipeline, configure an oauth-realm in the <realms> section of your ESE configuration.

The OAuth realm can be used for authentication in the oauth-authentication-manager.

Example OAuth realm configuration
<enterprise-security-extension>
    <realms>
        <oauth-realm>
            <name>oauth-realm-name</name>
            <enabled>true</enabled>
            <configuration>
                <jwks-endpoint>https://path/to/jwks/endpoint</jwks-endpoint>
                <auth-endpoint>https://path/to/auth/endpoint</auth-endpoint>
                <token-endpoint>https://path/to/token/endpoint</token-endpoint>
                <introspection-endpoint>https://path/to/introspection/endpoint</introspection-endpoint>
                <callback-uri>http://localhost:8080/callback</callback-uri>
                <simple-auth>
                    <username>my-oauth-username</username>
                    <password>my-oauth-password</password>
                </simple-auth>
                <tls-trust-store password="secret">path/to/truststore.jks</tls-trust-store>
                <jwks-default-cache-duration>1800</jwks-default-cache-duration>
            </configuration>
        </oauth-realm>
    </realms>
</enterprise-security-extension>
Table 70. OAuth realm parameters
Parameter Type Mandatory Description

jwks-endpoint

URI

Specifies the HTTP (GET) endpoint where the JWKS of the provider is located.

auth-endpoint

URI

Specifies the HTTP (GET) endpoint where the user will be redirected for authentication.

token-endpoint

URI

Specifies the HTTP (GET) endpoint where the JSON Web Token is located.

callback-uri

URI

Specifies the HTTP (GET) callback URI where the user will be redirected after authentication.

jwks-default-cache-duration

Integer

If this information is not specified by the provider in the HTTP response, this entry configures the duration that the JWKS is cached in seconds.

tls-trust-store

Trust Store

If the default system-wide TLS trust store is not used, this entry specifies an alternative trust store.

introspection-endpoint

URI

Specifies the HTTP (POST) endpoint where the JWT of the client is sent for further validation as specified in RFC-7662. It must be set when the authentication manager sets <enable-introspection> to true.

simple-auth

Simple Auth

Defines the HTTP Simple Auth parameters that are used for introspection and token-exchange requests from ESE to the identity server.

OAuth Authentication Manager

The OAuth Authentication Manager can only be used in the Control Center Redirect Pipeline. oauth-authentication-manager cannot be used as a part of any other pipeline.
Example OAuth Authentication Manager configuration
<oauth-authentication-manager>
    <realm>oauth-realm-name</realm> <!-- must be the name of the oauth-realm -->
    <flow>authorization-code-pkce</flow>
    <scopes>
        <scope>my-scope</scope>
    </scopes>
    <jwt-validation>
        <enable-introspection>false</enable-introspection>
        <reserved-claims>
            <aud>https://url/to/audience</aud>
            <iss>https://url/to/issuer</iss>
        </reserved-claims>
        <!-- only use direct match string values -->
        <custom-claims>
            <claim>
                <name>defined-name</name>
                <value>defined-value</value>
            </claim>
        </custom-claims>
    </jwt-validation>
</oauth-authentication-manager>
Table 71. OAuth Authentication Manager parameters
Parameter Type Mandatory Default Value Description

realm

Name reference

References the OAuth realm that the authentication manager accesses.

flow

XML enum

authorization-code

Defines which OAuth flow the authentication manager implements. The following values are possible:

  • authorization-code: The standard authorization code flow.

  • authorization-code-pkce: The authorization code flow with PKCE extension and S256 code challenge method.

TIP: The OAuth specification recommends the use of PKCE to provide an additional layer of security for the authorization code flows of every type of OAuth client.

scope

String

Defines the scope, where the access token will be limited to.

enable-introspection

Boolean

false

Defines whether the ESE checks with the identity server that issued the JWT for an additional validation of the token after successful local validation. If set to true, the introspection endpoint must be set in the specified realm.

sampling

Float [0.0 .. 1.0] (XML Attribute)

1.0

Sets the probability that ESE selects a token for introspection.

reserved-claims

Reserved Claims

Shows the reserved claims that are validated.

custom-claims

Custom Claims

Shows the custom claims that are validated.
The OAuth authentication manager of the ESE uses information from the scope claim to populate the authorization-role-key. Information from the sub claim is used to populate the authorization-key ESE Variables.

OAuth Authentication Flow

OAuth Authentication Flow in the ESE

ESE OAuth Authentication Flow

  1. The ESE redirects the Control Center user to the specified authorization endpoint.

  2. The user completes the authentication process independently at the defined authentication service.

  3. The user returns to the ESE via the callback URI of the ESE with a code from the external authentication source.

  4. The ESE exchanges the provided code with a JWT at the configured token-endpoint.

  5. The ESE validates the JWT against the provided configuration.

  6. Optionally, HiveMQ introspects the JWT against the configured introspection-endpoint of the OAuth provider.

  7. Successfully authenticated users proceed to the authorization stage of the Control Center redirect pipeline. The subject of the JWT will be used as authorization-key and the scopes will be used as role-keys.

Control Center Redirect Pipeline

The Control Center Redirect Pipeline authenticates and authorizes the user.

Authentication and authorization are both mandatory for any Control Center pipeline
Example Control Center Redirect Pipeline Configuration
<pipelines>
    <control-center-redirect-pipeline>

        <!-- authentication manager -->
        <oauth-authentication-manager>
            <!-- must be the name of the configured oauth-realm -->
            <realm>oauth-realm-name</realm>
            <flow>authorization-code-pkce</flow>
            <scopes>
                <scope>my-scope</scope>
            </scopes>
            <jwt-validation>
                <reserved-claims>
                    <aud>https://url/to/audience</aud>
                    <iss>https://url/to/issuer</iss>
                </reserved-claims>
            </jwt-validation>
        </oauth-authentication-manager>

        <authorization-manager>
            ...
        </authorization-manager>

    </control-center-redirect-pipeline>
</pipelines>

Control Center OAuth Configuration Examples

The following full configuration example shows OAuth-based authentication with auth0 and file-based authorization:

Example auth0 Configuration
<enterprise-security-extension>
    <realms>
        <oauth-realm>
            <name>auth0</name>
            <enabled>true</enabled>
            <configuration>
                <jwks-endpoint>https://[Auth0 Domain].auth0.com/.well-known/jwks.json</jwks-endpoint>
                <auth-endpoint>https://[Auth0 Domain].auth0.com/authorize?audience=https://[Auth0 Domain].auth0.com/api/v2/</auth-endpoint>
                <token-endpoint>https://[Auth0 Domain].auth0.com/oauth/token</token-endpoint>
                <callback-uri>http://localhost:8080/callback</callback-uri>
                <simple-auth>
                    <username>[Auth0 Client ID]</username>
                    <password>[Auth0 Client Secret]</password>
                </simple-auth>
            </configuration>
        </oauth-realm>

        <file-realm>
            <name>file-provider</name>
            <enabled>true</enabled>
            <configuration>
                <file-path>conf/ese-file-realm-auth0.xml</file-path>
            </configuration>
        </file-realm>
    </realms>

    <pipelines>
        <control-center-redirect-pipeline>

            <oauth-authentication-manager>
                <realm>auth0</realm>
                <flow>authorization-code-pkce</flow>
                <scopes>
                    <scope>openid</scope>
                </scopes>
                <jwt-validation>
                    <reserved-claims>
                        <aud>https://[Auth0 Domain].auth0.com/api/v2/</aud>
                        <iss>https://[Auth0 Domain].auth0.com/</iss>
                    </reserved-claims>
                </jwt-validation>
            </oauth-authentication-manager>

            <file-authorization-manager>
                <realm>file-provider</realm>
                <use-authorization-key>true</use-authorization-key>
                <use-authorization-role-key>false</use-authorization-role-key>
            </file-authorization-manager>

        </control-center-redirect-pipeline>
    </pipelines>

</enterprise-security-extension>

The following full configuration example shows OAuth-based authentication with Okta and file based authorization:

Example Okta Configuration
<enterprise-security-extension>
    <realms>
        <oauth-realm>
            <name>okta</name>
            <enabled>true</enabled>
            <configuration>
                <jwks-endpoint>https://[Okta Domain].okta.com/oauth2/default/v1/keys</jwks-endpoint>
                <auth-endpoint>https://[Okta Domain].okta.com/oauth2/default/v1/authorize</auth-endpoint>
                <token-endpoint>https://[Okta Domain].okta.com/oauth2/default/v1/token</token-endpoint>
                <callback-uri>http://localhost:8080/callback</callback-uri>
                <simple-auth>
                    <username>[Okta Client ID]</username>
                    <password>[Okta Client Secret]</password>
                </simple-auth>
            </configuration>
        </oauth-realm>

        <file-realm>
            <name>file-provider</name>
            <enabled>true</enabled>
            <configuration>
                <file-path>conf/ese-file-realm-okta.xml</file-path>
            </configuration>
        </file-realm>
    </realms>

    <pipelines>
        <control-center-redirect-pipeline>
            <oauth-authentication-manager>
                <realm>okta</realm>
                <flow>authorization-code-pkce</flow>
                <jwt-validation>
                    <reserved-claims>
                        <aud>HiveMQ</aud>
                        <iss>https://[Okta Domain].okta.com/oauth2/default</iss>
                    </reserved-claims>
                </jwt-validation>
            </oauth-authentication-manager>

            <file-authorization-manager>
                <realm>file-provider</realm>
                <use-authorization-key>true</use-authorization-key>
                <use-authorization-role-key>false</use-authorization-role-key>
            </file-authorization-manager>

        </control-center-redirect-pipeline>
    </pipelines>

</enterprise-security-extension>

OpenID Connect (OIDC) for HiveMQ Control Center Login

The HiveMQ Enterprise Security Extension makes it possible to integrate your HiveMQ Control Center with your preferred OpenID Connect provider to support enterprise-wide single sign-on (SSO) capabilities. Single sign-on refers to the ability for users to log in one time with one set of credentials to access services on multiple software systems. Providing SSO with the Authorization Code Flow for your HiveMQ Control Center users can provide greater security and compliance.

See full examples for auth0 and Okta.

OIDC Realm

To add OIDC-based authentication for HiveMQ Control Center users to an ESE authentication pipeline, configure an oidc-realm in the <realms> section of your ESE configuration.

The OIDC realm can be used for authentication in the oidc-authentication-manager.

Example OIDC realm configuration
<enterprise-security-extension>
    <realms>
        <oidc-realm>
            <name>oidc-realm-name</name>
            <enabled>true</enabled>
            <configuration>
                <issuer>https://path/to/issuer</issuer>
                <client-id>my-client-id</client-id>
                <client-authentication>
                    <client-secret-basic>my-client-secret</client-secret-basic>
                </client-authentication>
                <tls-trust-store password="secret">path/to/truststore.jks</tls-trust-store>
            </configuration>
        </oidc-realm>
    </realms>
</enterprise-security-extension>
Table 72. OIDC realm parameters
Parameter Type Mandatory Description

issuer

URI

Specifies the Issuer Identifier.

client-id

String

Specifies the Client Identifier.

client-authentication

Choice

Specifies the authentication method for the token endpoint. The following values are possible:

tls-trust-store

Truststore

If the default system-wide TLS truststore is not used, this entry specifies the path to an alternative truststore.

jwks-cache-seconds

Integer

  • Positive value: Sets the JWKS expiry time in seconds.
    The default value is 1800.

  • Any negative value: Caches the JWKS indefinitely.

  • 0: Disables JWKS caching. (Not recommend)

end-session-endpoint

URI

Specifies the end session endpoint.
The default setting is the end_session_endpoint from the provider metadata.
You only need to use this setting for implementations in which your OIDC provider does not provide an end_session_endpoint in its metadata response.
Client Secret Basic
<client-secret-basic>my-client-secret</client-secret-basic>
Parameter Type Mandatory/Default Description

String

The client secret for authentication.
TLS Client Auth
<tls-client-auth>
    <keystore>
        <path>path/to/my/keystore.jks</path>
        <type>JKS</type>
        <password>my-keystore-password</password>
        <private-key-password>my-private-key-password</private-key-password>
    </keystore>
</tls-client-auth>
Parameter Type Mandatory/Default Description

keystore

Keystore

The keystore with the private keys and certificates for authentication.
Private Key JWT
<private-key-jwt>
    <keystore-entry-alias>my-alias</keystore-entry-alias>
    <keystore>
        <path>path/to/my/keystore.jks</path>
        <type>JKS</type>
        <password>my-keystore-password</password>
        <private-key-password>my-private-key-password</private-key-password>
    </keystore>
    <kid>my-key-id</kid>
</private-key-jwt>
Parameter Type Mandatory/Default Description

keystore-entry-alias

String

The alias used to retrieve the private key and the certificate chain from the keystore.

keystore

Keystore

The keystore with the private keys and certificates for authentication.

kid

String

The value used as kid header parameter.

x5c

Boolean

false

Enables the x5c header parameter created from the certificate chain.

x5t

Boolean

false

Enables the x5t header parameter created from the end entity certificate.

x5t-sha256

Boolean

false

Enables the x5t#S256 header parameter created from the end entity certificate.
Example JWT Header
{
  "alg": "RS256",
  "kid": "my-key-id",
  "x5c": [ "MIICt...", "MIIC3..."],
  "x5t": "rSrFD...",
  "x5t#S256": "q2gW8..."
}
Example JWT Payload
{
  "iss": "my-client-id",
  "sub": "my-client-id",
  "aud": "https://my-token-endpoint",
  "exp": 1700671236,
  "jti": "xiAITRltzhHyPQDm-2nG5skXP4XTyHz4PyYGfFtaS_U"
}
Keystore
<keystore>
    <path>path/to/my/keystore.jks</path>
    <type>JKS</type>
    <password>my-keystore-password</password>
    <private-key-password>my-private-key-password</private-key-password>
</keystore>
Parameter Type Mandatory/Default Description

path

File Path

The path where the keystore is located. The path can be absolute or relative. Relative paths are relative to the ESE home directory.

type

String

JKS

The type of the keystore.

password

String

The password for the keystore.

private-key-password

String

The password for the private key.
TLS Truststore
<tls-trust-store password="my-truststore-password" type="JKS">path/to/my/truststore.jks</tls-trust-store>
Parameter Type Mandatory/Default Description

XML Value

File Path

The path where the truststore is located. The path can be absolute or relative. Relative paths are relative to the ESE home directory.

@type

String

JKS

The type of the truststore.

@password

String

The password for the truststore.

OIDC Authentication Manager

The OIDC Authentication Manager can only be used in the Control Center Pipeline. An oidc-authentication-manager cannot be used as a part of any other pipeline.
Example OIDC Authentication Manager configuration
<oidc-authentication-manager>
    <realm>oidc-realm-name</realm>
    <redirect-uri>http://localhost:8080/callback</redirect-uri>
    <post-logout-redirect-uri>http://localhost:8080</post-logout-redirect-uri>
    <authorization-key-claim>user</authorization-key-claim>
    <authorization-role-key-claim>roles</authorization-role-key-claim>
    <scopes>
        <scope>profile</scope>
        <scope>my-custom-scope</scope>
    </scopes>
    <expected-claims>
        <claim>
            <name>my-custom-claim</name>
            <value>my-custom-value</value>
        </claim>
    </expected-claims>
</oidc-authentication-manager>
Table 73. OIDC Authentication Manager parameters
Parameter Type Mandatory Default Value Description

realm

Name reference

References the OIDC realm that the authentication manager accesses.

redirect-uri

URI

The Redirection URI.

post-logout-redirect-uri

URI

The Post Logout Redirection URI.

scopes

List

A list of additional requested scopes. These scopes are requested in addition to the mandatory scope openid.

pkce-enabled

Boolean

true

Enables PKCE with S256 code challenge method.

nonce-enabled

Boolean

true

Enables the use of nonce values for ID Token validation.

authorization-key-claim

String

The name of the string claim in the ID Token as source for the authorization-key.

authorization-role-key-claim

String

The name of the array claim in the ID Token as source for the authorization-role-key.

expected-claims

List

Defines a list of name-value pairs that are expected as string claims in the ID Token. In addition to the standard ID Token validation with the RS256 JWS algorithm, the extension validates each claim in the list.

OIDC Authentication Flow

ESE follows the Authorization Code Flow:

  1. The ESE redirects the Control Center user to the external OpenID Connect provider.

  2. The user completes the authentication process independently at the provider.

  3. The user redirects back to the ESE from the provider with the authorization code as the query parameter.

  4. The ESE requests an ID Token via authorization code from the provider.

  5. The ESE validates the ID Token and checks the optional configured expected claims.

  6. The ESE sets the authorization-key and authorization-role-key from the configured source claims.

  7. Successfully authenticated users proceed to the authorization stage of the Control Center pipeline.

Control Center Pipeline

The Control Center Pipeline authenticates and authorizes the user.

Both authentication and authorization are mandatory for every Control Center pipeline.
Example Control Center Pipeline Configuration
<pipelines>
    <control-center-pipeline>

        <!-- authentication manager -->
        <oidc-authentication-manager>
            <!-- must be the name of the configured oidc-realm -->
            <realm>oidc-realm-name</realm>
            <redirect-uri>http://localhost:8080/callback</redirect-uri>
            <post-logout-redirect-uri>http://localhost:8080</post-logout-redirect-uri>
            <authorization-key-claim>user</authorization-key-claim>
            <authorization-role-key-claim>roles</authorization-role-key-claim>
            <scopes>
                <scope>profile</scope>
                <scope>my-custom-scope</scope>
            </scopes>
            <expected-claims>
                <claim>
                    <name>my-custom-claim</name>
                    <value>my-custom-value</value>
                </claim>
            </expected-claims>
        </oidc-authentication-manager>
        <!-- authorization manager -->
        <authorization-manager>
            ...
        </authorization-manager>
    </control-center-pipeline>
</pipelines>

Control Center OIDC Configuration Examples

The following configuration example shows OIDC-based authentication with auth0 and file-based authorization:

Example auth0 Configuration
<enterprise-security-extension>
    <realms>
        <oidc-realm>
            <name>auth0</name>
            <enabled>true</enabled>
            <configuration>
                <issuer>https://[Auth0 Domain].auth0.com</issuer>
                <client-id>[Auth0 Client ID]</client-id>
                <client-authentication>
                    <client-secret-basic>[Auth0 Client Secret]</client-secret-basic>
                </client-authentication>
                <end-session-endpoint>https://[Auth0 Domain].auth0.com/oidc/logout</end-session-endpoint>
            </configuration>
        </oidc-realm>

        <file-realm>
            <name>file-provider</name>
            <enabled>true</enabled>
            <configuration>
                <file-path>conf/ese-file-realm-okta.xml</file-path>
            </configuration>
        </file-realm>
    </realms>

    <pipelines>
        <control-center-pipeline>
            <oidc-authentication-manager>
                <realm>auth0</realm>
                <redirect-uri>http://localhost:8080/callback</redirect-uri>
                <post-logout-redirect-uri>http://localhost:8080</post-logout-redirect-uri>
                <authorization-key-claim>sub</authorization-key-claim>
            </oidc-authentication-manager>

            <file-authorization-manager>
                <realm>file-provider</realm>
                <use-authorization-role-key>false</use-authorization-role-key>
            </file-authorization-manager>
        </control-center-pipeline>
    </pipelines>
</enterprise-security-extension>

The following configuration example shows OIDC-based authentication with Okta and file based authorization:

Example Okta Configuration
<enterprise-security-extension>
    <realms>
        <oidc-realm>
            <name>okta</name>
            <enabled>true</enabled>
            <configuration>
                <issuer>https://[Okta Domain].okta.com/oauth2/default</issuer>
                <client-id>[Okta Client ID]</client-id>
                <client-authentication>
                    <client-secret-basic>[Okta Client Secret]</client-secret-basic>
                </client-authentication>
            </configuration>
        </oidc-realm>

        <file-realm>
            <name>file-provider</name>
            <enabled>true</enabled>
            <configuration>
                <file-path>conf/ese-file-realm-okta.xml</file-path>
            </configuration>
        </file-realm>
    </realms>

    <pipelines>
        <control-center-pipeline>
            <oidc-authentication-manager>
                <realm>okta</realm>
                <redirect-uri>http://localhost:8080/callback</redirect-uri>
                <post-logout-redirect-uri>http://localhost:8080</post-logout-redirect-uri>
                <authorization-key-claim>sub</authorization-key-claim>
            </oidc-authentication-manager>

            <file-authorization-manager>
                <realm>file-provider</realm>
                <use-authorization-role-key>false</use-authorization-role-key>
            </file-authorization-manager>
        </control-center-pipeline>
    </pipelines>
</enterprise-security-extension>

Lightweight Directory Access Protocol (LDAP)

The Lightweight Directory Access Protocol, LDAP for short, is an open standard protocol for interacting with directory servers. LDAP is widely-used to authenticate and store information about users, groups, and applications.

The current standard, LDAPv3, is defined in RFC 4510.

The HiveMQ Enterprise Security Extension (ESE) can work with any standard-compliant LDAP server. For example:

  • Microsoft Active Directory Domain Services (AD DS) and Microsoft Active Directory Lightweight Directory Services (AD LDS), which was previously named Active Directory Application Mode (ADAM)

  • OpenLDAP

  • Apache Directory Studio (ApacheDS)

Basic LDAP Concepts

Unlike SQL databases that store information in structured matrices (tables), LDAP stores information in a hierarchical data structure (tree). The LDAP data structure is called a Directory Information Tree (DIT).

ese basic ldap tree
Figure 1. Example LDAP Directory Information Tree (DIT)
Entries

Each node in an LDAP tree is called an entry. An LDAP entry is a collection of information about a particular object. Entries have three main components:

Distinguished Names (DN)

The distinguished name (DN) of an entry specifies the name and position of the entry in the LDAP tree. In an LDAP tree, each DN is unique and describes the fully qualified path to a particular entry. DN are made up of a sequence of zero or more comma-separated components called relative distinguished names (RDN).

Relative Distinguished Names (RDN)

A relative distinguished name (RDN) is a component of a DN. In an LDAP tree, RDN describe the partial path to an entry relative to another entry in the tree. Each RDN contains at least one attribute name-value pair.

In the DIT example diagram, the DN of the green client consists of the following RDN: cn=client-1,ou=mqtt-clients,dc=hivemq,dc=com. The DN of the red permission consists of these RDN: cn=permission-1,ou=permissions,dc=hivemq,dc=com.
cn stands for common name, ou organizational unit, and dc the domain component.

Distinguished names are written LEFT to RIGHT. Although a DN is similar to an absolute path on a filesystem, filesystem paths typically start with the root of the filesystem and descend the tree from left to right. In LDAP, a DN ascends the tree from left to right. The base DN is on the right.

LDAP Association

There are two main ways to associate entries in LDAP:

  • Direct Reference: Uses information from attributes in the distinguished name of an entry to create a direct association.

  • Descent: Uses the position of the entry in the LDAP tree to create association through a subtree.

ese associate reference
Figure 2. Example direct reference association
Direct reference in LDAP is similar to the way foreign keys are used in an SQL database.

In the direct reference diagram, the RDN of client 1 includes a memberOf attribute that contains the DN of Topic Permissions. The memberOf attribute automatically associates Topic Permissions to client 1 with a direct reference.

ese associate decent
Figure 3. Example subtree/decent association

In the subtree/descent diagram, descent is used to give permission group 1 all of the permissions that appear below permission group 1 in the tree.

LDAP Realms

In the ESE configuration, the abstraction of an LDAP server is an ldap-realm.

The LDAP realm provides all of the information that the ESE needs to connect to the LDAP server, perform the necessary bind requests, and execute LDAP searches.

Bind requests are the way that the LDAP protocol performs authentication.

LDAP search operations are equivalent to an SQL query. Caching on data returned from the LDAP server is also performed in the same manner.

Example LDAP Realm Configuration
<ldap-realm>
    <name>my-ldap-server</name>
    <enabled>true</enabled>
    <configuration>
        <servers>
          <ldap-server>
            <host>host-1</host>
            <port>389</port>
          </ldap-server>
          <ldap-server>
            <host>host-2</host>
            <port>389</port>
          </ldap-server>
        </servers>
        <tls>tcp</tls>
        <tls-trust-store>/path/to/truststore.jks</tls-trust-store>
        <base-dn>dc=hivemq,dc=com</base-dn>
        <simple-bind>
            <rdns>cn=ese,cn=hivemq</rdns>
            <userPassword>password</userPassword>
        </simple-bind>
    </configuration>
</ldap-realm>
Table 74. LDAP Realm Parameters
Parameter Type Mandatory Description

servers

LDAP Server

Specifies one or more LDAP servers that the ESE accesses.

tls

Enum

Specifies the transport security type that the ESE uses to connect to the LDAP servers. Possible settings are: tcp, tls, and start-tls (start-tls is an opportunistic transport encryption mechanism). TCP (no transport security) is the default setting.

tls-trust-store

Trust Store

Specifies an alternative trust store when the default system-wide TLS trust store is not used.

base-dn

LDAP distinguished name

Specifies the starting point that the ESE uses in the LDAP tree. All ESE searches and bind requests are relative to this base DN.

simple-bind

Simple Bind

Defines the simple bind parameters that the ESE uses to authenticate itself to the LDAP servers.

LDAP Server

The LDAP server setting configures a single LDAP instance. If you run LDAP in a clustered environment, the setting configures multiple instances. During runtime, the ESE accesses each instance in a round-robin fashion.

<ldap-server>
    <host>my.ldap.host</host>
    <port>389</port>
</ldap-server>
Table 75. LDAP Server Parameters
Parameter Type Mandatory Description

host

Domain name or IP Address

The network address where the LDAP server is located.

port

Integer

The port on which the LDAP server listens. This port is usually based on the type of transport security that is used. The default port setting is 389 (TCP).
Example Simple bind
<simple-bind>
    <rdns>cn=ese,cn=broker</rdns>
    <userPassword>password</userPassword>
</simple-bind>
Table 76. Simple Bind parameters
Parameter Type Mandatory Description

rdns

String

The relative distinguished names (RDN) of the base DN that the ESE uses to bind to the LDAP server. Make sure that this DN is bindable and has the necessary rights to search for the users and permissions.

userPassword

String

The password portion that the ESE uses to perform a simple bind operation on the LDAP server.

LDAP Authentication Manager

To implement custom logic for authentication over LDAP, you must configure an LDAP authentication manager in your ESE configuration.

Example LDAP Authentication Manager configuration
<ldap-authentication-manager>
    <realm>my-ldap-server</realm>
    <clients-rdns>ou=mqtt-clients,ou=iot-services</clients-rdns>
    <uid-attribute>cn</uid-attribute>
    <required-object-class>hmq-mqttClient</required-object-class>
</ldap-authentication-manager>
Table 77. LDAP Authentication Manager parameters
Parameter Type Mandatory Description

realm

String

The LDAP realm on which this authentication manager is based.

clients-rdns

LDAP directory name.

The starting RDN where ESE begins searches for client authentication information.

uid-attribute

String

The unique LDAP attribute that is used to identify every entry in the subtree of client RDNs. The default setting is uid.

required-object-class

String

Configures an object class that all entries which are considered for authentication purposes, must have. The default is none.

First, the LDAP authentication manager does an LDAP search for the entry that has a uid equals to the authentication-key. Next, LDAP authentication manager binds to the returned DN with the authentication-byte-secret.

After successful authentication, the LDAP authentication manager sets the ESE variable authorization-key to the DN of the client entry that is found.

If the found entry contains memberOf attributes, the authorization-role-key variable is set to the grouping information that the memberOf attributes provide.

LDAP Authorization Manager

To implement custom logic for authorization over LDAP, you must configure an LDAP authorization manager in your ESE configuration.

Example LDAP Authorization Manager configuration
<ldap-authorization-manager>
    <realm>my-ldap-server</realm>
    <directory-descent>false</directory-descent>
    <use-authorization-key>true</use-authorization-key>
    <use-authorization-role-key>true</use-authorization-role-key>
</ldap-authorization-manager>

The LDAP authorization manager requires full DNs in the ESE authorization variables.

Table 78. LDAP Authorization Manager parameters
Parameter Type Mandatory Description

directory-descent

Boolean

Specifies whether permissions are searched by descent (instead of direct reference). The default setting is false.

use-authorization-key

Boolean

Specifies whether the authorization-key variable is used. The default setting is true.

use-authorization-role-key

Boolean

Specifies whether the authorization-role-key variable is used. The default setting is true.

LDAP Server Setup

  • Entries used for authentication must be bindable through simple bind requests

  • Special object classes and attributes are only needed for authorization

  • Scripts for the new LDAP object classes and attributes that the ESE uses are available in [ese-dir]/scripts/ldap/[ese-version] (AD, ApacheDS, OpenLDAP, rfc compliant)

  • The configuration of an LDAP server is specific to the individual server that is used. Always follow the procedures and best practices that are associated with the selected server.

  • To enable direct reference, you must add the auxiliary class hmq-mqttClient or hmq-controlCenterUser to the desired user or group object

  • All LDAP object classes and attributes that are used in the ESE have a unique object identifier (OID) that is registered at the Internet Assigned Numbers Authority (IANA)

LDAP Object Classes

Object classes indicate what type of object an LDAP entry represents. Every LDAP entry must have exactly one structural object class and can have zero or more auxiliary object classes. Each object class has a unique name and defines which attributes are mandatory (Must) or optional (May) for a particular entry.

Since the LDAP schema that the ESE uses has only one structural object class, all entries can include the hmq-mqttPermission object class.

Table 79. LDAP Object classes
Name Type Mandatory Attributes Optional Attributes Description

hmq-mqttPermission

Structural

hmq-topicFilter

hmq-qualityOfService, hmq-publish, hmq-subscribe, hmq-retain, hmq-share, hmq-sharedGroup, description, cn

Defines the authorization data for an MQTT topic. If multiple permissions for the same topic string are used, include the common name in the distinguished name of the entry.

hmq-mqttClient

Auxiliary

hmq-topicPermission

Authorizes an MQTT client with one or more permissions. If attached to a group of names or group of unique names, the entry can represent multiple clients that share an RBAC role.

hmq-controlCenterUser

Auxiliary

hmq-controlCenterPermission

Defines which control center views a control center user is granted.

LDAP Attributes

Attributes hold the data for an LDAP entry. Each attribute has a unique name, a unique object identifier (OID), and is a member of one or more object classes.

Since the hmq-topicFilter attribute is a required element of the mandatory hmq-mqttPermission object class, all entries include this attribute.

In the LDAP schema that the ESE uses, each LDAP attribute begins with 1.3.6.1.4.1.54795. followed by a unique sequence. For example, the OID of the hmq-topicPermission attribute is 1.3.6.1.4.1.54795.2.3.1. The unique portion of the OID is emphasized in the following attributes table.
Table 80. LDAP Attributes
Name Identifier Description

hmq-topicPermission

1.3.6.1.4.1.54795.2.3.1

The topic permission attribute represents all permissions that are associated with a specific MQTT topic.

hmq-topicFilter

1.3.6.1.4.1.54795.2.3.2

The topic filter string that determines to which topics the topic permission applies. The topic permission applies to all topic-strings that are matched by the topic filter.

hmq-controlCenterPermission

1.3.6.1.4.1.54795.2.3.9

The permission string for the HiveMQ Control Center.

hmq-qualityOfService

1.3.6.1.4.1.54795.2.3.3

The quality of service level (QoS) to which the topic permission applies. Possible values are 0, 1, and 2.

hmq-publish

1.3.6.1.4.1.54795.2.3.4

The publish topic to which the topic permission applies. The topic must match the topic filter.

hmq-subscribe

1.3.6.1.4.1.54795.2.3.5

The subscribe topic to which the topic permission applies. The topic must match the topic filter.

hmq-retain

1.3.6.1.4.1.54795.2.3.6

The retained messages to which the topic permission applies. The topic must match the topic filter.

hmq-share

1.3.6.1.4.1.54795.2.3.7

The shared subscriptions to which the topic permission applies. The topic must match the topic filter.

hmq-sharedGroup

1.3.6.1.4.1.54795.2.3.8

The shared group ($share/<group>/topic) to which the topic permission applies. This attribute is only relevant when the hmq-share attribute (OID 1.3.6.1.4.1.54795.2.3.7) is set to TRUE.

Access Log

The access log provides a single, unified log for tracking security-relevant data. This log has several use cases:

  • You can audit all accesses that the ESE grants retroactively.

  • Your intrusion-prevention software (for example, Fail2Ban) can use the access log to create firewall rules.

  • The chronological records of the access log can provide valuable information for the post-mortem of a data breach.

The access log shows you precisely who was granted access to HiveMQ, what kind of access occurred, and when the access was granted. The timezone of events in the access log is always Coordinated Universal Time (UTC).

The following is a good start for a fail-regex: ^ - authentication-failed - Client failed authentication: ID [^\s]+, IP <HOST>, reason \“((authorization timed out)|(unknown authentication key or wrong authentication secret)|(authentication timed out)|(other))\“\.$


Access Events

The three primary access events are Authentication Failed, Authentication Succeeded, and Authorization Succeeded. Each of these events provides the public client IP address and the MQTT client ID of the connecting client.

Table 81. Access log events
Event Type Logged Values Log Statement

Authentication Failed

  • Client ID

  • Client IP

  • Reason of failure

yyyy-MM-dd hh:mm:ss,sss UTC - authentication-failed - Client failed authentication: ID <client id>, IP <public client ip>, reason "<reason string>"

Authentication Succeeded

  • Client ID

  • Client IP

yyyy-MM-dd hh:mm:ss,sss UTC - access.authentication-succeeded - Client succeeded authentication: ID <client id>, IP <public client ip>.

Authorization Succeeded

  • Client ID

  • Client IP

  • List of permissions

yyyy-MM-dd hh:mm:ss,sss UTC - access.authorization-succeeded - Client succeeded authorization: ID <client id>, IP <public client ip>, permissions <[permission]>.

Authentication Failed

This event triggers when an MQTT client does not pass the ESE authentication and is not allowed to connect to HiveMQ. Each Authentication Failed event provides one of the following reasons for the failure:

Table 82. Authentication failed reasons
Reason Description Corresponding Metrics

unknown authentication key or wrong authentication secret

The client cannot be authenticated against the information that the configured realm of the authentication manager provided

unknown-key, wrong-secret

authentication timed out

The process timed out during the authentication step of the pipeline

authentication-timeout

authorization timed out

The process timed out during the authorization step of the pipeline

authorization-timeout

other

Something exceptional happened during the process. In this case, you can look for more information in the DEBUG output of the hivemq.log.

other

Authentication Succeeded

This event triggers when an MQTT client passes the ESE authentication successfully. These events correspond to the authentication-succeeded metric.

Authorization Succeeded

This event triggers after an MQTT client is granted permissions. The event references the Topic Permissions that the client is granted in the following format:

Permission{topicFilter='<topic-string>', qos=[<0, 1, 2>], activity=[<publish, subscribe>], retainedPublishAllowed=<true|false>, sharedSubscribeAllowed=<true|false>, sharedGroup='<group-id>', from='<source>'}.

The <source> in the reference is dependent on the type of authorization manager that you use. For an SQL authorization manager, the source can either be the roles.name or the users.username that the permission is associated with.

Access Log File

By default, the access log writes to the <HiveMQ Home>/log/access/access.log. You can customize this path in the ESE configuration file and by manually setting specific HiveMQ folders.

The HiveMQ access log is a rolling log file. Once per day at midnight, the old access-log files are automatically compressed with the gzip algorithm and archived with the filename access.<yyyy-MM-dd>.log.gz. For example, after two days of operation the access folder contains the following files:

├─ access.2019-07-04.log.gz
├─ access.2019-07-05.log.gz
└─ access.log
The ESE never deletes the access log files that are archived. If you need to remove old access logs regularly, you must take additional action. For example, set up a scheduled cron job to alleviate data protection concerns or storage constraints.

Access Log Configuration

In the config.xml file, you can use the optional <access-log> tag to configure the behavior of the access log.

Implicit default access log configuration
<access-log>
    <enabled>true</enabled>
    <file-name>access</file-name>
    <sub-folder>access</sub-folder>
    <write-through>false</write-through>
</access-log>
Table 83. Access log configuration parameters
Parameter Type Description Default

enabled

Boolean

Enables or disables the writing of access events to the access log file

true

file-name

String

The name of the access log file. The suffix .log is added. This file name changes the name of the archived log file, as well.

access

sub-folder

String

The name of the subfolder in the log directory of HiveMQ where the access log file and the archived log files are stored.

access

write-through

Boolean

Enables or disables also writing access events to the hivemq.log file. The log level of the access events is DEBUG. To only write access events to the hivemq.log file, set the enabled tag to false and write-through tag to true.

false

Control Center Access Control

The ESE provides access control for the HiveMQ Control Center. You can use both fine-grained access control that is based on user permissions and role-based access control (RBAC).

Requirements for Control Center Access Control

  • HiveMQ Enterprise Security Extension 1.2.0 or higher

  • HiveMQ Enterprise Edition 4.2.0 or higher

Control Center Pipeline

Similar to the authentication and authorization of MQTT clients, the handling of control-center users is done in a separate pipeline: the <control-center-pipeline>. This special ESE pipeline works like a listener pipeline for MQTT clients.

The control center pipeline uses the User form of the login page as the authentication-key and authorization-key. After the Password form is decoded with the UTF8 codec, this information is used as the authentication-byte-secret.

Control center login page

HiveMQ control center login page

Example control center pipeline configuration
<enterprise-security-extension>
    <pipelines>
        <!-- only one <control-center-pipeline> allowed -->
        <!-- secure access to the control center -->
        <control-center-pipeline>
            <!-- authenticate over a sql db -->
            <sql-authentication-manager>
                <realm>postgres-backend</realm>
            </sql-authentication-manager>
            <!-- authorize over a sql db -->
            <sql-authorization-manager>
                <realm>postgres-backend</realm>
                <use-authorization-key>true</use-authorization-key>
                <use-authorization-role-key>true</use-authorization-role-key>
            </sql-authorization-manager>
        </control-center-pipeline>
    </pipelines>
</enterprise-security-extension>
Refer to the description of each realm to learn how to enable access control for the control center from the desired realm.

Control Center Permissions

To implement fine-grained access control for your control center, you assign specific control-center permissions to your users. If desired, you can assign a set of control-center permissions to a role. All users with a specific role are granted the permissions that are assigned to the role. Role-based access control can simplify permission management and increase performance.

The following permissions are available for the HiveMQ control center:

Table 84. Control center permissions
Permission Description

HIVEMQ_SUPER_ADMIN

Allows the user to access everything in your control center. This special permission is the default role for users in the HiveMQ config.xml.

HIVEMQ_VIEW_PAGE_CLIENT_LIST

Allows the user to view the client list on your control center.

To view client list information, the user must also have the following permission: HIVEMQ_VIEW_DATA_CLIENT_ID.

HIVEMQ_VIEW_PAGE_CLIENT_DETAIL

Allows the user to view client details on your control center.

To view client detail information, the user must also have the following permission: HIVEMQ_VIEW_DATA_CLIENT_ID.

HIVEMQ_VIEW_PAGE_LICENSE

Allows the user to view the license information on your control center.

HIVEMQ_VIEW_PAGE_TRACE_RECORDINGS

Allows the user to view trace recordings on your control center.

To view the trace recording information, the user must also have the following permissions: HIVEMQ_VIEW_DATA_CLIENT_ID, HIVEMQ_VIEW_DATA_IP, HIVEMQ_VIEW_DATA_PASSWORD, HIVEMQ_VIEW_DATA_PAYLOAD, HIVEMQ_VIEW_DATA_PROXY, HIVEMQ_VIEW_DATA_SUBSCRIPTION, HIVEMQ_VIEW_DATA_TOPIC, HIVEMQ_VIEW_DATA_USERNAME, and HIVEMQ_VIEW_DATA_WILL_MESSAGE.

HIVEMQ_VIEW_PAGE_DROPPED_MESSAGES

Allows the user to view dropped messages on your control center.

To view the dropped message information, the user must also have the following permission: HIVEMQ_VIEW_DATA_CLIENT_ID.

HIVEMQ_VIEW_PAGE_RETAINED_MESSAGE_LIST

Allows the user to view the retained message list on your control center.

To view the retained message list information, the user must also have the following permission: HIVEMQ_VIEW_DATA_TOPIC.

HIVEMQ_VIEW_PAGE_RETAINED_MESSAGE_DETAIL

Allows the user to view the retained messages details on your control center.

To view the retained message details information, the user must also have the following permission: HIVEMQ_VIEW_DATA_TOPIC.

HIVEMQ_VIEW_PAGE_SHARED_SUBSCRIPTION_LIST

Allows the user to view the Shared Subscription list on your control center.

To view the shared subscription list information, the user must also have the following permissions: HIVEMQ_VIEW_DATA_SUBSCRIPTION and HIVEMQ_VIEW_DATA_TOPIC.

HIVEMQ_VIEW_PAGE_SHARED_SUBSCRIPTION_DETAIL

Allows the user to view Shared Subscription details on your control center.

To view the shared subscription details information, the user must also have the following permissions: HIVEMQ_VIEW_DATA_SUBSCRIPTION and HIVEMQ_VIEW_DATA_TOPIC.

HIVEMQ_VIEW_PAGE_EXTENSION_CONSUMER_LIST

Allows the user to view extension consumer list

To view the extension consumer list, the user must also have the following permissions: HIVEMQ_VIEW_DATA_SUBSCRIPTION and HIVEMQ_VIEW_DATA_TOPIC.

HIVEMQ_VIEW_PAGE_BACKUP

Allows the user to view backup information on your control center.

To view the backup information, the user must also have the following permissions: HIVEMQ_VIEW_DATA_CLIENT_ID, HIVEMQ_VIEW_DATA_TOPIC, HIVEMQ_VIEW_DATA_SUBSCRIPTION, and HIVEMQ_VIEW_DATA_PAYLOAD.

HIVEMQ_VIEW_PAGE_SUPPORT

Allows the user to view and use the support page on your control center.

HIVEMQ_VIEW_PAGE_KAFKA_DASHBOARD

Allows the user to view the dashboard of the HiveMQ Enterprise Extension for Kafka.

HIVEMQ_VIEW_DATA_CLIENT_ID

Allows the user to view client identifiers on your control center.

Lack of this permission has the following result:

  • The user cannot view the Clients, Analytics, Trace Recordings, or Backup pages of your control center.

  • Client identifiers are hidden from the user on all pages of your control center.

HIVEMQ_VIEW_DATA_IP

Allows the user to view client IP information in your control center.

Lack of this permission has the following result:

  • The user cannot view the Trace Recordings page of your control center.

  • The IP information of clients is hidden from the user on all pages of your control center.

HIVEMQ_VIEW_DATA_PAYLOAD

Allows the user to view message payloads on your control center.

Lack of this permission has the following result:

  • The user cannot view the Trace Recordings or Backup pages in your control center.

  • Message payloads are hidden from the user on all pages of your control center.

HIVEMQ_VIEW_DATA_PASSWORD

Allows the user to view client passwords in your control center.

Lack of this permission has the following result:

  • The user cannot view the Trace Recordings page of your control center.

  • Client passwords are hidden from the user on all pages of your control center.

HIVEMQ_VIEW_DATA_USERNAME

Allowed the user to view client usernames.

Lack of this permission has the following result:

  • The user cannot view the Trace Recording page of your control center.

  • Username information of clients is hidden from the user on all pages of your control center.

HIVEMQ_VIEW_DATA_WILL_MESSAGE

Allows the user to view the LWT of clients on your control center.

Lack of this permission has the following result:

  • The user cannot view the Trace Recording page of your control center.

  • LWT information is hidden from the user on all pages of your control center.

HIVEMQ_VIEW_DATA_TOPIC

Allows the user to view the PUBLISH and SUBSCRIBE topics of clients on your control center.

Lack of this permission has the following result:

  • The user cannot view the Trace Recordings or Backpup pages of your control center.

  • Topic information is hidden from the user on all pages of your control center.

HIVEMQ_VIEW_DATA_SUBSCRIPTION

Allows the user to see client subscriptions on your control center.

Lack of this permission has the following result:

  • The user cannot view the Trace Recordings or Backup pages of your control center.

  • Subscription information is hidden from the user on all pages of your control center.

HIVEMQ_VIEW_DATA_PROXY

Allows the user to view the proxy information of clients on your control center.

Lack of this permission has the following result:

  • The user cannot view the Trace Recordings page of your control center.

  • Client-proxy information is hidden from the user on all pages of your control center.

HIVEMQ_VIEW_DATA_TLS

Allows the user to view the TLS information of clients on your control center.

Lack of this permission has the following result:

  • TLS information is hidden from the user on all pages of the control center.

HIVEMQ_VIEW_DATA_USER_PROPERTIES

Allows the user to view the User Properties information of MQTT5 messages.

HIVEMQ_VIEW_DATA_SESSION_ATTRIBUTES

Allows the user to view Session Attributes of MQTT clients.

Lack of this permission has the following result:

  • Session Attributes are hidden from the user on all pages of the control center.

HIVEMQ_VIEW_DATA_CLIENT_EVENT_HISTORY

Allows the user to view the Event History of clients.

Lack of this permission has the following result:

  • The Client Event History is hidden from the user on all pages of the control center.

HIVEMQ_VIEW_DATA_CLUSTER_NODES

Allows the user to view granular node information on your control center.

Lack of this permission has the following result:

  • The user cannot view statistics per cluster node on the control center.

  • Only aggregated information for all nodes in total is shown.

HIVEMQ_VIEW_DATA_ACTIVE_LICENSE

Allows the user to view active license information on your control center.

Lack of this permission has the following result:

  • Active license information is hidden from the user on all pages of the control center.

HIVEMQ_VIEW_PAGE_SCHEMAS_LIST

Allows the user to view the schema list on your control center.

HIVEMQ_VIEW_PAGE_SCHEMA_DETAIL

Allows the user to view schema details on your control center.

HIVEMQ_EDIT_DATA_SCHEMA

Allows the user to create/edit schemas on your control center.

To edit data schemas, the user must also have the following permission: HIVEMQ_VIEW_PAGE_SCHEMA_DETAIL.

HIVEMQ_VIEW_PAGE_DATA_POLICIES_LIST

Allows the user to view the data policy list on your control center.

To view the data policy list, the user must also have the following permission: HIVEMQ_VIEW_DATA_TOPIC.

HIVEMQ_VIEW_PAGE_BEHAVIOR_POLICIES_LIST

Allows the user to view behavior policy list on your control center.

To view the behavior policy list, the user must also have the following permission: HIVEMQ_VIEW_DATA_CLIENT_ID.

HIVEMQ_VIEW_PAGE_DATA_POLICY_DETAIL

Allows the user to view data policy details on your control center.

To view data policy details on your control center, the user must also have the following permissions: HIVEMQ_VIEW_DATA_CLIENT_ID and HIVEMQ_VIEW_DATA_TOPIC.

HIVEMQ_VIEW_PAGE_BEHAVIOR_POLICY_DETAIL

Allows the user to view behavior policy details on your control center.

To view behavior policy details on your control center, the user must also have the following permissions: HIVEMQ_VIEW_DATA_CLIENT_ID and HIVEMQ_VIEW_DATA_TOPIC.

HIVEMQ_VIEW_PAGE_DATA_HUB_CHARTS

Allows the user to view Data Hub charts on your control center.

To view Data Hub charts on your control center, the user must also have the following permission: HIVEMQ_VIEW_DATA_CLIENT_ID.

HIVEMQ_EDIT_DATA_DATA_POLICY

Allows the user to create/edit data policies on your control center.

To create and edit data policies on your control center, the user must also have the following permissions: HIVEMQ_VIEW_DATA_TOPIC, HIVEMQ_VIEW_DATA_CLIENT_ID and HIVEMQ_VIEW_PAGE_DATA_POLICY_DETAIL.

HIVEMQ_EDIT_DATA_BEHAVIOR_POLICY

Allows the user to create/edit behavior policies on your control center.

To create and edit behavior policies on your control center, the user must also have the following permissions: HIVEMQ_VIEW_DATA_CLIENT_ID, HIVEMQ_VIEW_DATA_TOPIC and HIVEMQ_VIEW_PAGE_BEHAVIOR_POLICY_DETAIL.

HIVEMQ_VIEW_PAGE_SCRIPTS_LIST

Allows the user to view the script list on your control center.

HIVEMQ_VIEW_PAGE_SCRIPT_DETAIL

Allows the user to view script details on your control center.

HIVEMQ_EDIT_DATA_SCRIPT

Allows the user to create/edit scripts on your control center.

To create and edit scripts on your control center, the user must also have the following permissions: HIVEMQ_VIEW_PAGE_SCRIPT_DETAIL.

REST API Access Control

HiveMQ REST API access control allows you to restrict user access HiveMQ REST API in several ways:

  • Authenticate your REST API users with username and password or JWT.

  • Manage authenticated access to the HiveMQ REST API on a per-endpoint basis.

  • Configure fine-grained access control for individual users and user roles.

Requirements for REST API Access Control

  • HiveMQ Enterprise Security Extension 4.9.0 or higher

  • HiveMQ Enterprise Edition 4.9.0 or higher

REST API Pipeline

Similar to the authentication and authorization of MQTT clients, the handling of REST API users is done in a separate pipeline: the <rest-api-pipeline>.

The REST API pipeline requires an HTTP Headers preprocessor that extracts authentication information from the header of the HTTP request of the REST API client.

ESE extracts the information in accordance with RFC7235 authentication scheme. For more information, see HTTP Headers Preprocessor.

ESE REST API pipeline configuration allows selection of multiple listeners. The listeners attribute refers to the name in the HiveMQ config.xml file. For more information, see REST API Pipelines.
Example REST API pipeline configuration
<enterprise-security-extension>
    <pipelines>
        <!-- secure access to the Rest API -->
        <rest-api-pipeline listener="ALL">
            <authentication-preprocessors>
                <http-headers-preprocessor>
                    <basic-auth-extraction/>
                </http-headers-preprocessor>
            </authentication-preprocessors>
            <!-- authenticate over an sql db -->
            <sql-authentication-manager>
                <realm>mysql-backend</realm>
            </sql-authentication-manager>
            <!-- authorize over an sql db -->
            <sql-authorization-manager>
                <realm>mysql-backend</realm>
                <use-authorization-key>true</use-authorization-key>
                <use-authorization-role-key>true</use-authorization-role-key>
            </sql-authorization-manager>
        </rest-api-pipeline>
    </pipelines>
</enterprise-security-extension>
HTTP authentication is insecure without TLS. The extracted credentials are in clear text in the HTTP header and therefore vulnerable to man-in-the-middle attacks, unless TLS is used. For more information, see REST API HTTPS listeners.

REST API Permissions

To implement fine-grained access control for the HiveMQ REST API resources your users can access, you can assign specific permissions to users and user roles. When you assign specific REST API permissions to a role, all users that have the role are granted the permissions you assign to the role.
Role-based access control can simplify permission management and increase performance.

The following permissions are available for the HiveMQ REST API:

Table 85. REST API permissions
Permission Description

HIVEMQ_SUPER_ADMIN

Allows the user to access all resources of your HiveMQ REST API. This special permission is the default role for users in the HiveMQ config.xml file.

HIVEMQ_MANAGEMENT_BACKUPS_GET

Allows the user to view the available backups

HIVEMQ_MANAGEMENT_BACKUPS_POST

Allows the user to create a new backup.

HIVEMQ_MANAGEMENT_BACKUPS_BACKUPID_GET

Allows the user to view information and the current state for a specific backup.

HIVEMQ_MANAGEMENT_BACKUPS_BACKUPID_POST

Allows the user to restore a specific stored backup.

HIVEMQ_MANAGEMENT_FILES_BACKUPS_BACKUPID_GET

Allows the user to download a specific backup.

HIVEMQ_MANAGEMENT_TRACE_RECORDINGS_GET

Allows the user to view all know trace recordings.

HIVEMQ_MANAGEMENT_TRACE_RECORDINGS_POST

Allows the user to create a new trace recording.

HIVEMQ_MANAGEMENT_TRACE_RECORDINGS_TRACERECORDINGID_PATCH

Allows the user to stop an existing trace recording.

HIVEMQ_MANAGEMENT_TRACE_RECORDINGS_TRACERECORDINGID_DELETE

Allows the user to delete an existing trace recording.

HIVEMQ_MANAGEMENT_FILES_TRACE_RECORDINGS_TRACERECORDINGID_GET

Allows the user to download a specific trace recording.

HIVEMQ_MANAGEMENT_DIAGNOSTIC_ARCHIVES_POST

Allows the user to create a new diagnostic archive.

HIVEMQ_MQTT_CLIENTS_GET

Allows the user to view all known online and offline client sessions for the HiveMQ cluster.

HIVEMQ_MQTT_CLIENTS_CLIENTID_SUBSCRIPTIONS_GET

Allows the user to view all subscriptions for a specific client.

HIVEMQ_MQTT_CLIENTS_CLIENTID_GET

Allows the user to view information and the current state for a specific client.

HIVEMQ_MQTT_CLIENTS_CLIENTID_DELETE

Allows the user to invalidate the session of a specific client. If the client is currently connected, the client is disconnected.

HIVEMQ_MQTT_CLIENTS_CLIENTID_CONNECTION_GET

Allows the user to view whether a specific client is currently connected.

HIVEMQ_MQTT_CLIENTS_CLIENTID_CONNECTION_DELETE

Allows the user to disconnect a specific client if the client is currently connected.

HIVEMQ_DATA_HUB_SCHEMAS_POST

Allows the user to create Data Hub schemas.

HIVEMQ_DATA_HUB_SCHEMAS_SCHEMAID_GET

Allows the user to retrieve the content of Data Hub schema <schemaId>.

HIVEMQ_DATA_HUB_SCHEMAS_SCHEMAID_DELETE

Allows the user to DELETE all versions of Data Hub schema <schemaId>.

HIVEMQ_DATA_HUB_SCHEMAS_GET

Allows the user to retrieve the content of Data Hub schemas with the content type JSON.

HIVEMQ_DATA_HUB_DATA_VALIDATION_POLICIES_POST

Allows the user to create Data Hub data policies.

HIVEMQ_DATA_HUB_DATA_VALIDATION_POLICIES_POLICYID_GET

Allows the user to retrieve the content of Data Hub data policy <policyId>.

HIVEMQ_DATA_HUB_DATA_VALIDATION_POLICIES_POLICYID_DELETE

Allows the user to DELETE Data Hub data policy <policyId>.

HIVEMQ_DATA_HUB_DATA_VALIDATION_POLICIES_GET

Allows the user to retrieve the content of all Data Hub data policies.

HIVEMQ_DATA_HUB_DATA_VALIDATION_POLICIES_POLICYID_PUT

Allows the user to update Data Hub data policy <policyId>.

HIVEMQ_DATA_HUB_BEHAVIOR_VALIDATION_POLICIES_POST

Allows the user to create Data Hub behavior validation policies.

HIVEMQ_DATA_HUB_BEHAVIOR_VALIDATION_POLICIES_POLICYID_GET

Allows the user to retrieve the content of Data Hub behavior policy <policyId>.

HIVEMQ_DATA_HUB_BEHAVIOR_VALIDATION_POLICIES_POLICYID_DELETE

Allows the user to DELETE Data Hub behavior policy <policyId>.

HIVEMQ_DATA_HUB_BEHAVIOR_VALIDATION_POLICIES_GET

Allows the user to retrieve the content of all Data Hub behavior validation policies.

HIVEMQ_DATA_HUB_BEHAVIOR_VALIDATION_POLICIES_POLICYID_PUT

Allows the user to update Data Hub behavior policy <policyId>.

HIVEMQ_DATA_HUB_BEHAVIOR_VALIDATION_STATES_CLIENTID_GET

Allows the user to retrieve the states of client <clientId> for Data Hub.

HIVEMQ_DATA_HUB_MANAGEMENT_START_TRIAL_POST

Allows the user to start the Data Hub trial mode.

HIVEMQ_DATA_HUB_SCRIPTS_POST

Allows the user to create Data Hub scripts.

HIVEMQ_DATA_HUB_SCRIPTS_SCRIPTID_GET

Allows the user to retrieve specific Data Hub scripts.

HIVEMQ_DATA_HUB_SCRIPTS_SCRIPTID_DELETE

Allows the user to delete specific Data Hub scripts.

HIVEMQ_DATA_HUB_SCRIPTS_GET

Allows the user retrieve a list of Data Hub scripts.

Client Certificate Revocation Check

The Enterprise Security Extension gives you the option to configure OCSP (Online Certificate Status Protocol) and CRL (Certificate Revocation List) client certificate revocation checking during the TLS handshake for secure TCP listeners and secure WebSocket listeners.

The handshake check requires a secure listener with enabled client-authentication and a configured trust store that contains the whole chain of CA certificates.
Since TLS session resumption bypasses the handshake check, it is sensible to configure the TLS session timeout on the listener to achieve a balance between performance and security.
The handshake check adds overhead to every TLS handshake. In case of OCSP, an OCSP-Responder is requested on every TLS handshake. The additional overhead is highly dependent on this request. In case of CRL, a list of revoked certificates is fetched for each listener and cached until the list expires.
Example handshake revocation check
<enterprise-security-extension>
    <client-certificate-revocation-checks>
        <client-certificate-revocation-check listener="ocsp-minimal">
            <on-tls-handshake>
                <method>
                    <ocsp/>
                </method>
            </on-tls-handshake>
        </client-certificate-revocation-check>

        <client-certificate-revocation-check listener="crl-minimal">
            <on-tls-handshake>
                <method>
                    <crl/>
                </method>
            </on-tls-handshake>
        </client-certificate-revocation-check>

        <client-certificate-revocation-check listener="ocsp-full">
            <enabled>false</enabled>
            <on-tls-handshake>
                <method>
                    <ocsp>
                        <endpoint>https://ocsp-endpoint</endpoint>
                        <crl-fallback/>
                    </ocsp>
                </method>
                <soft-fail>true</soft-fail>
            </on-tls-handshake>
        </client-certificate-revocation-check>

        <client-certificate-revocation-check listener="crl-full">
            <enabled>false</enabled>
            <on-tls-handshake>
                <method>
                    <crl>
                        <ocsp-fallback>
                            <endpoint>https://ocsp-endpoint</endpoint>
                        </ocsp-fallback>
                    </crl>
                </method>
                <soft-fail>true</soft-fail>
            </on-tls-handshake>
        </client-certificate-revocation-check>
    </client-certificate-revocation-checks>
</enterprise-security-extension>
Table 86. Certificate revocation check parameters
Parameter Type Mandatory / Default Description

@listener

String

The name of the related listener.

enabled

Boolean

true

Optional setting that defines whether the revocation check is used. The default setting is true. To disable the revocation check, set the value to false.

on-tls-handshake

Complex

Specifies the TLS handshake check.
Table 87. TLS handshake check parameters
Parameter Type Mandatory / Default Description

method

Complex

Specifies the check method. This is either ocsp or crl.

soft-fail

Boolean

false

When set to true, the following errors do not cause the check to fail:

  • Network errors

  • The OCSP errors

    • internalError

    • tryLater

In case of a configured fallback, both checks must state this condition to succeed.
Table 88. OCSP check method parameters
Parameter Type Mandatory / Default Description

endpoint

URI

AuthorityInfoAccess

Overrides the OCSP responder endpoint from the certificate.

crl-fallback

Empty

No fallback

Specifies a CRL fallback, in case the revocation status could not be determined via OCSP.
Table 89. CRL check method parameters
Parameter Type Mandatory / Default Description

ocsp-fallback

Complex

No fallback

Specifies an OCSP fallback, in case the revocation status could not be determined via the CRL from the CRL endpoint.
Table 90. OCSP fallback parameters
Parameter Type Mandatory / Default Description

endpoint

URI

AuthorityInfoAccess

Overrides the OCSP responder endpoint from the certificate.

Monitoring in HiveMQ Enterprise Security Extension

Your HiveMQ Enterprise Security Extension exposes numerous metrics over the standard HiveMQ monitoring interface.

Available Metrics in HiveMQ Enterprise Security Extension

All HiveMQ Enterprise Security Extension metrics start with the prefix com.hivemq.extensions.ese.

The following table lists each metric that ESE exposes. For increased readability, the com.hivemq.extensions.ese. prefix is omitted in the tables. For more information on HiveMQ metrics, see metric types.

Table 91. ESE Standard Metrics
Metric (without prefix) Type Description

connect

Counter

Shows the total number of incoming MQTT client connection attempts that the ESE processes

access.authentication-succeeded

Counter

Shows the total number of successful authentication requests

access.authorization-succeeded

Counter

Shows the total number of successful authorization requests

access.authentication-failed

Counter

Shows the total number of unsuccessful authentication requests

access.authentication-failed.unknown-key

Counter

Shows the total number of authentication requests that failed due to unknown authentication keys

access.authentication-failed.wrong-secret

Counter

Shows the total number of authentication requests that failed due to incorrect authentication secrets

access.authentication-failed.authentication-timeout

Counter

Shows the total number of authentication requests that failed due to timed-out authentications

access.authentication-failed.authorization-timeout

Counter

Shows the total number of authentication requests that failed due to timed-out authorizations

access.authentication-failed.other

Counter

Shows the total number of authentication requests that failed due to other reasons

Each SQL realm has its own metrics. The name of the realm is included in the metric name.

Table 92. ESE SQL Metrics
Metric (without prefix) Type Description

sql.permission.role.cache.hitrate.<realm_name>

Meter

Shows the rate of requests for role-based permission data that are handled from the ESE cache. The rate is provided in requests per a pre-defined time interval for the specified SQL realm.

sql.permission.user.cache.hitrate.<realm_name>

Meter

Shows the rate of requests for user-based permission data that are handled from the ESE cache. The rate is provided in requests per a pre-defined time interval for the specified SQL realm.

sql.permission.time.<realm_name>

Timer

Shows the duration and frequency of requests for authorization data from the specified SQL realm

sql.user.time.<realm_name>

Timer

Shows the duration and frequency of requests for authentication data from the specified SQL realm

sql.authentication.cache.hitrate.<realm_name>

Meter

Shows the rate of requests for user-based permission data that are handled from the ESE cache. The rate is provided in requests per a pre-defined time interval for the specified SQL realm.

ESE exposes several metrics about the usage of JSON Web Tokens (JWT).
Most of the JWT metrics are realm-specific and include the realm name in the metric name.

Table 93. ESE JWT Metrics
Metric (without prefix) Type Description

jwt.parsed.total

Counter

Shows the total number of parsed JWTs. This includes JWTs parsed by the JWT preprocessor.

jwt.validate.success.<realm_name>

Meter

Shows the rate of successfully validated JWTs for the specified realm.

jwt.validate.failed.<realm_name>

Meter

Shows the rate of unsuccessfully validated JWTs for the specified realm.

jwt.introspection.rate.<realm_name>

Meter

Shows the rate of token introspection requests against the specified realm.

jwt.introspection.time.<realm_name>

Timer

Shows the duration and frequency of introspection requests against the specified realm.

ESE exposes the following Open ID Connect metrics:

Table 94. ESE Open ID Connect Metrics
Metric (without prefix) Type Description

oidc.tokens.validate.success.<realm_name>

Meter

Shows the rate of successfully validated tokens for the specified realm.

oidc.tokens.validate.failed.<realm_name>

Meter

Shows the rate of unsuccessfully validated tokens for the specified realm.

ESE exposes the following file-based metrics:

Table 95. ESE File Metrics
Metric (without prefix) Type Description

file.parse.time.<realm-name>

Timer

Shows the amount of time ESE requires to parse XML authorization files for the specified realm.

ESE exposes the following LDAP metrics:

Table 96. ESE LDAP Metrics
Metric (without prefix) Type Description

ldap.user.time

Timer

Shows the duration of queries for authentication data from LDAP realms.

ldap.bind.time

Timer

Shows the duration of LDAP bind requests to authenticate users.

ldap.user.search.failed

Counter

Shows the number of failed searches for LDAP users (MQTT and control center).

ldap.mqtt-permission.time

Timer

Shows the duration of LDAP searches for hmq-mqttPermissions.

ldap.mqtt-permission.role.cache.hitrate

Meter

Shows the rate of requests for LDAP role permissions that are handled from the ESE cache.

ldap.mqtt-permission.user.cache.hitrate

Meter

Shows the rate of requests for LDAP user permissions that are handled from the ESE cache.

ldap.mqtt-permission.cache.hitrate

Meter

Shows the rate of requests for hmq-mqttPermissions that are handled from the ESE cache.

ldap.mqtt-permission.search.failed

Counter

Shows the number of failed searches for MQTT permissions.

ldap.control-center-permission.search.failed

Counter

Shows the number of failed searches for control center permissions.

ldap.authentication.cache.hitrate

Meter

Shows the rate of requests for authentication credentials that are handled from the ESE cache.

Support

If you need help with the HiveMQ Enterprise Security Extension or have suggestions on how we can improve the extension, please contact us at contact@hivemq.com.