HiveMQ Data Governance Hub

The HiveMQ Data Governance Hub provides mechanisms to define how MQTT data is handled in the HiveMQ broker. The first feature of our new Data Governance Hub is Data Validation.

The HiveMQ Data Governance Hub data validation feature is currently available for testing in a closed beta version only.
For more information or to request access to the closed beta, contact datagovernancehub@hivemq.com

In the HiveMQ Data Governance Hub, data validation implements a declarative policy that checks whether your data sources are sending data in the data format you expect.
This process ensures that the value of the data is assessed at an early stage in the data supply chain.
To eliminate the need for subscribing clients to perform resource-intensive validation, checks occur before your data reaches downstream devices or upstream services.

Features

Validate MQTT messages with JSON Schema or Protobuf.
Enforce policies across the entire MQTT topic tree structure.
Reroute valid and invalid MQTT messages to different topics based on the data validation results.
Increase the observability of client performance through additional metrics and log statements.

Our data validation functionality lets your development teams use the HiveMQ broker to automatically enforce a data validation strategy of their own design (including fine-tuned control over how the broker handles incoming valid and invalid MQTT messages).

Figure 1. Example Data Validation Workflow

In this user journey, an engineering team develops a microservice that consumes sensor data on specific MQTT topics. To ensure the reliability of the microservice, the engineers upload a JSON Schema to the HiveMQ Data Governance Hub that outlines the data structure they want to receive.

Example geographical location JSON Schema

{
  "$id": "https://example.com/geographical-location.schema.json",
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "title": "Longitude and Latitude Values",
  "description": "A geographical coordinate.",
  "required": [
    "latitude",
    "longitude"
  ],
  "type": "object",
  "properties": {
    "latitude": {
      "type": "number",
      "minimum": -90,
      "maximum": 90
    },
    "longitude": {
      "type": "number",
      "minimum": -180,
      "maximum": 180
    }
  }
}

To enforce the rules and structure that the schema outlines, the operations team registers an appropriate policy that tells HiveMQ how to handle incoming MQTT messages.

Example policy definition

{
  "id": "com.hivemq.policy.coordinates",
  "matching": {
    "topicFilter": "coordinates"
  },
  "validation": {
    "validators": [
      {
        "type": "schema",
        "arguments": {
          "strategy": "ALL_OF",
          "schemas": [
            {
              "schemaId": "gps_coordinates"
            }
          ]
        }
      }
    ]
  },
  "onFailure": {
    "pipeline": [
      {
        "id": "logFailure",
        "functionId": "log",
        "arguments": {
          "level": "WARN",
          "message": "$clientId sent invalid coordinates on topic '$topic' with result '$validationResult'"
        }
      }
    ]
  }
}

In the example, the policy ensures that the HiveMQ broker drops MQTT messages published to the MQTT topic coordinates that do not match the desired gps_coordinates schema. In addition, HiveMQ prints a log statement to help debug the misbehaving clients.

The result is a more efficient microservice that only processes MQTT messaging data that adheres to the specified format. This type of well-defined data validation can save time and resources:

Enables parallel development
Reduces the number of issues that reach production systems
Simplifies the identification of invalid data sources

The closed beta version of the HiveMQ Data Governance Hub requires a special license.
For more information or to request to join the closed beta, contact datagovernancehub@hivemq.com

Get Started

The best way to learn how to set up the HiveMQ Data Governance Hub is to follow our Quick Start Guide.