HiveMQ Modules for Data Hub

HiveMQ Modules for Data Hub offer easy-to-use functionality that is quickly accessible from your HiveMQ Control Center. You can use modules to define fine-grained and fully flexible Data Hub policies or utilize pre-defined Modules that implement ready-to-use functionalities.

To enable the Modules feature, all Data Hub features must be enabled. For more information, see Configuration.

HiveMQ manages all HiveMQ for Data Hub Modules. Additional modules and new module versions will be shown in the Control Center.

Module Lifecycle

The Modules overview lists all available modules and provides additional information such as the latest version, a description, and the maintainer of the module.

You can create multiple instances of each module. For more information see Module Instances.

The Instances tab of your HiveMQ Control Center lists all instantiated modules on your system and allows you to take further actions.

List of Module instances

Available Instance Actions

Disable or Enable a Module Instance

From the Instances overview, you can disable and enable module instances as desired. After a confirmation message, the status of the selected instance changes as requested.

Enabled instances are highlighted in green on the Instances overview and disabled instances are highlighted in red:

Disable and Enable Module instances

View Module Instance Configuration

The view icon opens a configuration summary for the selected instance :

View Module Instance Configuration

Update a Module Instance

Module instances can be updated to another version. You can edit an existing instance and select the version to be updated to. During the update step, you can also modify the configuration of the module instance. After a confirmation request, the update of the Module instance with the new version and configuration runs seamlessly in the background.

Delete a Module Instance

From the Instances overview, you can delete module instances as desired.
After a confirmation message, the module instance is permanently deleted and removed from the Instances overview.

Available HiveMQ Modules

All available Data Hub Modules can be accessed from the HiveMQ Control Center. The Modules navigation point in the Data Hub menu of the Control Center opens an overview of all Modules available on your system:

Data Hub Modules

HiveMQ provides several managed modules for Data Hub:

  • hivemq-duplicate-message: The Data Hub duplicate message module identifies consecutive identical client messages to prevent unnecessary resource consumption. For example, this module can save bandwidth and storage costs by dropping duplicate readings from sensors that repeatedly send the same value.

  • hivemq-sparkplug: The Data Hub Sparkplug module offers essential functionalities such as schema validation, Protobuf to JSON conversion, metric fan-out, and the ability to migrate to a flexible MQTT topic structure. These capabilities enable data format conformity, simplify data integration, ensure relevant data for consumers, and allow for flexible topic structuring.

  • hivemq-validate-simple-json: The Data Hub validate simple JSON module helps you quickly add JSON validation capabilities to your HiveMQ deployment. For example, configure whether to log a message or drop invalid messages to avoid non-compliant JSON messages.

All module features can be turned on or off as needed.

Module Instances

You can create multiple instances of each Data Hub module.

The Data Hub view in the HiveMQ Control Center facilitates instance creation with an intuitive user interface. Our instance creation wizards offer an intuitive way to create new module instances with context-sensitive help and immediate feedback on configuration validity.

For more information, see HiveMQ Control Center.

Create Module Instances

To create an instance of a module, select the desired module version from the list of available versions on the Modules overview and click + Create Instance.

A dialog to configure the module instance opens automatically. For more information, see Configure Module Instances.

Sensible default values are provided for each module.
You can use the pre-defined values or adjust the instance configuration to fit your individual business needs.

Once your configuration is complete, you can select Create to generate your new instance.

Configure Module Instances

The available configuration parameters vary based on the type of module instance you choose to create:

  • HiveMQ Duplicate Messages Module Instance Configuration

Create HiveMQ Duplicate Messages Module instances

Table 1. hivemq-duplicate-messages configuration options
Setting Default Required Description

Instance ID

The unique identifier of the module instance. The ID must begin with a letter and can contain only uppercase/lowercase letters, numbers, periods, hyphens, or underscores. If no custom instance ID is defined, HiveMQ automatically generates a unique instance ID.

Duplicate message handling

drop

Specifies how the module instance processes duplicate PUBLISH messages sent by the same client. The default setting is drop.

  • drop: Drops all duplicate PUBLISH messages that match the configured client IDs.

  • disconnect: Disconnects the client when a duplicate PUBLISH message is detected.

  • log: Generates a log statement when a duplicate PUBLISH message is detected.

MQTT client ID matching

*

A regular expression to define the client IDs to which the instance applies. The default setting .* matches all client IDs.

  • Hivemq Sparkplug Module Instance Configuration

    Create HiveMQ Sparkplug Module instances

    Table 2. hivemq-sparkplug configuration options
    Setting Default Required Description

    Instance ID

    The unique identifier of the module instance. The ID must begin with a letter and can contain only uppercase/lowercase letters, numbers, periods, hyphens, or underscores. If no custom instance ID is defined, HiveMQ automatically generates a unique instance ID.

    Topic Prefix

    spBv1.0//DDATA//+

    The SparkplugB topic namespace and the initial segments of the topic that indicate the group and type of message. The default setting matches all incoming Sparkplug DDATA (device data) messages.

    Fan out Sparkplug metrics

    When selected, publishes each SparkplugB metric on an individual sub-topic (metric fan-out).

    Increment Metric

    When selected, increments the configured metric for each incoming message that contains an invalid Sparkplug message payload.

    Metric Name

    invalid-sparkplugb-messages

    The metric name on which the invalid SparkplugB messages are tracked.
    If you reuse the default metric name for additional hivemq-sparkplug module instances, all instances that use the default name increment the same metric. To track metrics separately for each instance, enter a custom metric name for each instance.
    It is not possible to create multiple module instances for the same Topic Prefix.

  • HiveMQ Validate Simple JSON Module Instance Configuration

    Create HiveMQ Validate Simple JSON Module instances

    Table 3. hivemq-validate-simple-json configuration options
    Setting Default Required Description

    Instance ID

    The unique identifier of the module instance. The ID must begin with a letter and can contain only uppercase/lowercase letters, numbers, periods, hyphens, or underscores. If no custom instance ID is defined, HiveMQ automatically generates a unique instance ID.

    Print log message

    When selected, HiveMQ prints a log message when a client sends a PUBLISH message with a valid or invalid JSON payload to a topic that matches the configured topic filter.

    Drop invalid JSON messages

    When selected, HiveMQ drops all PUBLISH messages that contain a payload with an invalid JSON format and match the configured topic filter.

    Topic Filter Matching

    #

    Defines the PUBLISH message topic filter to which the module instance applies. The default setting matches all topics.
    It is not possible to create multiple module instances for the same topic filter. For more information, see Topic Tree.
    Example invalid JSON message System.log function arguments
    {
  "id": "fail-log-operation",
  "functionId": "System.log",
  "arguments": {
  "level": "ERROR",
  "message": "The client ${clientId} sent an invalid message: ${validationResult}"
  }
}
    Example valid JSON message System.log function arguments
    {
  "id": "success-log-operation",
  "functionId": "System.log",
  "arguments": {
  "level": "INFO",
  "message": "The client ${clientId} sent a valid JSON message."
  }
}
    Example drop invalid JSON message System.log function arguments
    {
  "id": "fail-drop-operation",
  "functionId": "Mqtt.drop",
  "arguments": {
  "reasonString": "The message you sent was not in JSON format."
  }
}

Create Custom Modules

You can develop a custom module based on your individual business needs and import it into your HiveMQ deployment.

Custom modules require a full HiveMQ Modules for Data Hub license.

Create Custom Module from the HiveMQ Control Center

The HiveMQ Control Center provides a function to create a new module from an existing data or behavior policy.

To start, open the policy you want to use in the module and select Create Module:

Create Module from Policy

A dialog opens to enter module details such as the module name:

Create Module Dialog

After you define all mandatory fields, you can create and download your new module from the broker:

Download Created Module

The module you created contains all the necessary information from the policy used. The policy information in the module includes all referenced schemas and scripts. Each module is a ZIP compressed archive. The following directory structure and files are created when you unzip the module archive:

File Structure Hello World Module

Create Custom Module from HiveMQ Hello World Module

You can also develop a module from policies, schemas, and scripts you have written in a text editor or IDE (Integrate Development Environment). Our 'Hello World' Data Hub Module Example is available to fork and build upon. The hello world example also contains a simple CI (Continue Integration) pipeline.

For more information and configuration details, see Modules Specification.

Import Custom Modules

The HiveMQ Control Center offers a convenient method to import the custom modules you have built into your HiveMQ deployment. Go to the Data Hub | Modules view of your HiveMQ Control Center and select Import Custom Module.

Import Custom Module

A dialog opens that allows you to select a module from your local disk:

Dialog to Import Custom Module

After the module is uploaded, you can configure your input parameters and select Create to instantiate the module.

The created module is available on the Instances tab of your HiveMQ Control Center:

Instances Tab