HiveMQ Data Governance Hub

The HiveMQ Data Governance Hub provides mechanisms to define how MQTT data is handled in the HiveMQ broker. The first feature of our new Data Governance Hub is Data Validation.

The HiveMQ Data Governance Hub data validation feature is currently available for testing in an Early Access Preview (EAP) version. Additionally, starting with HiveMQ Platform 4.17, a free version of the Data Governance Hub with basic functions is included in your HiveMQ bundle. To learn more about the different ways you can explore our new EAP, see HiveMQ Data Governance Hub Licensing. For more information about the terms and conditions of the EAP, contact our sales team they are happy to assist you.

Data validation in the HiveMQ Data Governance Hub allows you to implement declarative policies that check whether your data sources are sending data in the data format you expect.
This process ensures that the value of the data is assessed at an early stage in the data supply chain.
To eliminate the need for subscribing clients to perform resource-intensive validation, checks occur before your data reaches downstream devices or upstream services.

Features

  • Validate MQTT messages with JSON Schema or Protobuf.

  • Enforce policies across the entire MQTT topic tree structure.

  • Reroute valid and invalid MQTT messages to different topics based on the data validation results.

  • Increase the observability of client performance through additional metrics and log statements.

Our data validation functionality lets your development teams use the HiveMQ broker to automatically enforce a data validation strategy of their own design (including fine-tuned control over how the broker handles incoming valid and invalid MQTT messages).

Example Data Validation Workflow
Figure 1. Example Data Validation Workflow

In this user journey, an engineering team develops a microservice that consumes sensor data on specific MQTT topics. To ensure the reliability of the microservice, the engineers upload a JSON Schema to the HiveMQ Data Governance Hub that outlines the data structure they want to receive.

Example geographical location JSON Schema
{
  "$id": "https://example.com/geographical-location.schema.json",
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "title": "Longitude and Latitude Values",
  "description": "A geographical coordinate.",
  "required": [
    "latitude",
    "longitude"
  ],
  "type": "object",
  "properties": {
    "latitude": {
      "type": "number",
      "minimum": -90,
      "maximum": 90
    },
    "longitude": {
      "type": "number",
      "minimum": -180,
      "maximum": 180
    }
  }
}

To enforce the rules and structure that the schema outlines, the operations team registers an appropriate policy that tells HiveMQ how to handle incoming MQTT messages.

Example policy definition
{
  "id": "com.hivemq.policy.coordinates",
  "matching": {
    "topicFilter": "coordinates"
  },
  "validation": {
    "validators": [
      {
        "type": "schema",
        "arguments": {
          "strategy": "ALL_OF",
          "schemas": [
            {
              "schemaId": "gps_coordinates",
              "version": "latest"
            }
          ]
        }
      }
    ]
  },
  "onFailure": {
    "pipeline": [
      {
        "id": "logFailure",
        "functionId": "log",
        "arguments": {
          "level": "WARN",
          "message": "${clientId} sent invalid coordinates on topic '${topic}' with result '${validationResult}'"
        }
      }
    ]
  }
}

In the example, the policy ensures that the HiveMQ broker drops MQTT messages published to the MQTT topic coordinates that do not match the desired gps_coordinates schema. In addition, HiveMQ prints a log statement to help debug the misbehaving clients.

The result is a more efficient microservice that only processes MQTT messaging data that adheres to the specified format. This type of well-defined data validation can save time and resources:

  • Enables parallel development

  • Reduces the number of issues that reach production systems

  • Simplifies the identification of invalid data sources

HiveMQ Data Governance Hub Licensing

The HiveMQ Data Governance Hub offers various license tiers to cater to different needs. For detailed information or to discuss which option is the best for your individual use case, contact our sales team.

Since HiveMQ 4.17, a free version is included in your HiveMQ platform bundle. Free mode enables you to create a single policy and use a limited set of functionality.

To explore the complete capabilities of the HiveMQ Data Governance Hub, you can activate a 5-hour trial mode at no additional cost. For more information, see Activate HiveMQ Data Governance Hub Trial Mode.

Activate Data Governance Hub Trial Mode

The 5-hour trial mode of the HiveMQ Data Governance Hub lets you explore the complete capabilities of the HiveMQ Data Governance Hub at no additional cost. To utilize features after your trial period ends, you can continue using the Free mode or obtain a valid license based on your requirements. For more information, contact our sales team.

Before you proceed to activate your trial mode, make sure the <enabled> tags for the <data-governance-hub> and the <rest-api> are set to true in the config.xml file of your HiveMQ instance. For more information, see Configuration.

  • Send a POST-request with an empty body to the endpoint:
    <host>:<port>/api/v1/management/data-governance-hub/start-trial
    with the default values for the REST API listener:
    localhost:8888/api/v1/management/data-governance-hub/start-trial

  • The status code 204 is returned upon successful activation.

The trial mode gives you full access to all HiveMQ Data Governance Hub functionality for a duration of 5 hours.
When your trial period ends, the Data Governance Hub reverts to the Free mode or the installed license, depending on your configuration.

Existing policies that exceed the limitations of the mode you return to after your trial remain in place. However, it is not possible to add new policies that exceed the limits of the active mode that is currently active.

Get Started

The best way to learn how to set up the HiveMQ Data Governance Hub is to follow our Quick Start Guide.