HiveMQ Kubernetes Operator (legacy)

Starting in April 2024, HiveMQ 4.28 becomes the Long Term Support (LTS) version of the HiveMQ Platform. The HiveMQ Platform Operator for Kubernetes (new) provides usability, stability, and performance enhancements and is the supported operator for HiveMQ 4.28 and future releases. Support and updates for the preceding HiveMQ 4.9 LTS version and the older HiveMQ Kubernetes Operator (legacy) will continue for 6 months after the HiveMQ 4.28 release, ending in November 2024.
For more information, see Flexible MQTT Platform Upgrades with HiveMQ and HiveMQ Versioning.

We highly recommend using the HiveMQ Platform Operator for your HiveMQ deployments on Kubernetes to ensure uninterrupted updates and support long into the future. For more information, see HiveMQ Platform Operator for Kubernetes.

The HiveMQ Kubernetes Operator is an application-specific controller that helps orchestrate and manage the lifecycle of a HiveMQ cluster deployment within Kubernetes.
The HiveMQ Kubernetes Operator runs as a custom controller on Kubernetes and communicates with the Kubernetes API Server (kube-api server) to convert high-level descriptions into normal Kubernetes resources that maintain the desired application state:

  • Automate orchestration of your entire HiveMQ deployment (platform agnostic) on Kubernetes

  • Monitor, maintain, recover, and upgrade your HiveMQ application with ease

  • Automatically deploy and operate HiveMQ on your Kubernetes cluster with custom or enterprise extensions

The HiveMQ Kubernetes Operator can run and maintain multiple HiveMQ clusters.

For production deployments of HiveMQ on Kubernetes, we recommend exclusive allocation of one HiveMQ pod to one Kubernetes node. For more information, see Assigning Pods to Nodes.
The storage option you select determines the level of performance your HiveMQ Broker can deliver. To ensure high durability and optimal performance, we recommend the use of dedicated persistent volumes that deliver a minimum of 900 IOPS for HiveMQ in production. For more information, see HiveMQ Minimum Hardware Requirements.

Features

  • Predefined HiveMQ Helm Chart: Quickly deploy the HiveMQ Operator and HiveMQ Cluster custom resource with convenient default settings that can be used as defined or adapted to meet your individual needs.

  • Seamless HiveMQ Upgrades: Built-in automated zero-downtime upgrade feature for your HiveMQ installation that ensures continuous availability during software upgrades.

  • Extension Lifecycle Integration: Lifecycle management for the entire range of extensions the HiveMQ extension framework provides:

  • Monitoring with Prometheus: Easy integration with your Prometheus Operator. All HiveMQ metrics are included in the integration and can be monitored with the tool of your choice.

Requirements

  • Running Kubernetes cluster version 1.18.0 or higher on the cloud provider of your choice

  • Helm version 3 or higher (to deploy with the HiveMQ Helm Chart)

  • Current version of the Kubernetes command line tool kubectl

Mandatory Custom Resource Definition Update
Since HiveMQ 4.4, the HiveMQ Kubernetes Operator is part of the HiveMQ platform release bundle. Since HiveMQ 4.7, the HiveMQ Kubernetes Operator supports the stable v1 api version of CustomResourceDefinition. Use of the v1 CustomResourceDefinition requires a mandatory update of your CRD. The update does not require any downtime. For more information, see HiveMQ 4.6 to 4.7 Migration Guide.

HiveMQ Kubernetes Operator Quick Start Guide

Use the preconfigured HiveMQ Helm Chart to install a HiveMQ cluster on Kubernetes with the HiveMQ Kubernetes Operator.

Prepare

Take a moment to make sure that you have all the necessary tools and resources on hand:

This quick-start guide shows you how to deploy a HiveMQ cluster on Kubernetes that is suitable for testing purposes only.
To configure the HiveMQ cluster for production, see Configure Your HiveMQ Cluster for Production.

Install

Use Helm to install the HiveMQ Kubernetes Operator on Kubernetes and deploy a HiveMQ cluster with the default HiveMQ settings:

The HiveMQ Kubernetes Operator creates a default three-node HiveMQ cluster custom resource during installation. Each node is configured to request 4 CPUs, 4GB memory, and 15GB of disk space. If you want to test the HiveMQ Kubernetes Operator on a local Kubernetes cluster, decrease the amount of requested resources to match your system resources.
1. Add the HiveMQ Helm Chart repository to your Helm installations:
helm repo add hivemq https://hivemq.github.io/helm-charts
2. Install the HiveMQ Helm Chart:
helm upgrade --install hivemq hivemq/hivemq-operator

This command does the following:

  • Extends the Kubernetes API with the HiveMQ Cluster custom resource definition (CRD).

  • Installs the HiveMQ Kubernetes Operator as a custom controller.

  • Adds a HiveMQ Cluster custom resource (CR) that contains standard default HiveMQ configuration settings.

  • Saves this instance of the chart with the release name hivemq. For more information, see Helm releases.

The HiveMQ Kubernetes Operator uses the configuration settings that are defined in the HiveMQ Cluster custom resource to create and maintain a HiveMQ cluster on your Kubernetes cluster.

3. Check if your HiveMQ cluster has started:
kubectl get hivemq-clusters

The output lists key information about your HiveMQ cluster including the number of cluster nodes and a human readable description of the current state of the cluster.

Adapt

The HiveMQ cluster custom resource that the HiveMQ Kubernetes Operator creates has standard default settings. To complete this quick start guide and connect MQTT clients you must adapt the default settings of the HiveMQ Cluster custom resource. You can use the same method to adapt your HiveMQ cluster settings for production.

When you modify the settings of your HiveMQ cluster custom resource, the HiveMQ Kubernetes Operator initiates a rolling upgrade of your HiveMQ cluster on Kubernetes. To maintain a steady load distribution during the upgrade, the operator starts a new HiveMQ pod before it shuts down and upgrades an existing HiveMQ pod. This process temporarily increases the HiveMQ cluster by one pod. The additional HiveMQ pod requires proportionally more system resources. For example, enough CPUs, memory, and disk storage. Once the rolling upgrade is complete, the HiveMQ cluster returns to the configured number of pods and resources.
1. To view the current settings the Helm chart maintains for the selected HiveMQ cluster custom resource, enter:
helm get values <release-name>

You can adapt the values of the settings as desired. For more information, see Basic Configuration.

2. To have Helm change the configuration settings of your HiveMQ cluster as needed for this quick start guide, save the following configuration into a new myCustomValues.yaml file.

These settings scale the HiveMQ cluster to 4 nodes, disallow empty client IDs, expose the MQTT port 1883 as a LoadBalancer for MQTT clients from outside the Kubernetes cluster to connect to your HiveMQ cluster, and make it possible to connect to the HiveMQ control center service.

hivemq:
  nodeCount: "4"
  security:
    allowEmptyClientId: false

  ports:
    - name: "mqtt"
      port: 1883
      expose: true
      patch:
      - '[{"op":"add","path":"/spec/selector/hivemq.com~1node-offline","value":"false"},{"op":"add","path":"/metadata/annotations","value":{"service.spec.externalTrafficPolicy":"Local"}}]'
      # If you want Kubernetes to expose the MQTT port
      - '[{"op":"add","path":"/spec/type","value":"LoadBalancer"}]'
    - name: "cc"
      port: 8080
      expose: true
      patch:
      - '[{"op":"add","path":"/spec/sessionAffinity","value":"ClientIP"}]'
      # If you want Kubernetes to expose the HiveMQ control center via load balancer.
      # Warning: You should consider configuring proper security and TLS beforehand. Ingress may be a better option here.
      # - '[{"op":"add","path":"/spec/type","value":"LoadBalancer"}]'
3. To have Helm upgrade your deployment with the values you entered in the myCustomValues.yaml, enter:
helm upgrade --install -f myCustomValues.yaml hivemq hivemq/hivemq-operator
4. To verify that your the HiveMQ cluster updated, enter:
kubectl get hivemq-clusters

As soon as the displayed state returns to RUNNING, your HiveMQ cluster uses the new settings.

5. To view the settings of your HiveMQ cluster, list the values once more:
helm show values hivemq/hivemq-operator

The expected result is that the values shown include the changes you entered in your myCustomValues.yaml file.

Connect

Use the MQTT CLI to connect MQTT clients to your HiveMQ cluster on Kubernetes and start sending MQTT messages.

We use the open source MQTT CLI for ease of use and to simplify the explanation of this sample procedure. However, feel free to use whichever tool and method you prefer.
Download packages with complete installation instructions for the MQTT CLI are available at no cost on GitHub. An introductory tutorial for the MQTT CLI is available on YouTube.
1. List the information of your HiveMQ cluster again:
kubectl get hivemq-clusters

This shows an endpoint URL to which you can connect MQTT clients.

If the Kubernetes cluster can provide load balancers through a cloud controller and the port is configured as type: LoadBalancer using a patch, an endpoint URL is listed. If the Kubernetes cluster lacks cloud integration, no endpoint URL appears in the status of the custom resource. 
$ kubectl get hivemq-clusters
NAME       SIZE   IMAGE                VERSION    STATUS    ENDPOINT     MESSAGE
cluster1   3      hivemq/hivemq4       4.7.0      Running   10.2.7.144   Live updates completed successfully
2. To start the MQTT CLI in shell mode, open a new terminal window and enter the following command:
mqtt sh

The MQTT CLI starts and displays a list of useful options and commands.

Make sure that you expose the MQTT port as described in the section Adapt deployment settings. For MQTT clients to connect to your HiveMQ cluster, port 1883 must be exposed .
3. To connect to your HiveMQ Cloud cluster, type the following command with the appropriate information for your cluster and press enter:
con -h <your-cluster-url>

Information about the HiveMQ connection displays.
This is your first connected MQTT client.

4. To create another connected client and test your cluster, open a second terminal window and enter the following command once more (keep your original terminal window open):
mqtt sh

Copy and paste the connection command that you created in the first window to the second window and press enter. Information about the second HiveMQ connection displays. This is your second connected MQTT client.

Test

Publish MQTT messages and subscribe to MQTT topics with your connected MQTT clients.

In shell mode, adding the -s flag to the command blocks the MQTT CLI console so that the incoming messages are printed out to the user.
1. In the terminal window of the second MQTT client, enter:
sub -t testTopic -s

This command subscribes the second MQTT client to all messages that are published with the topic testTopic.

2. In the first terminal window, enter:
pub -t testTopic -m Hello

This command makes the first MQTT client a publisher and publishes the message Hello with the topic testTopic.
The message Hello appears immediately in the terminal window of the second MQTT client.

3. To publish another message, enter the following command in the first terminal window:
pub -t testTopic -m Goodbye

This command publishes the message Goodbye with the topic testTopic.
The message Goodbye appears immediately in the terminal window of the second MQTT client.

To view the activity of your new MQTT clients on the HiveMQ control center, see Monitor MQTT Clients on Your HiveMQ Control Center.

Monitor

Observe the activity of your MQTT clients on your HiveMQ Control Center.

The HiveMQ control center provides a wide range of metrics that help you manage and analyze your cluster. Use the detailed information the control center provides to maintain a clear overview of your clusters and quickly identify irregular client behavior.

1. List all the services of your cluster:
kubectl get services -l hivemq-cluster=my-hivemq-cluster
2. Connect to the control center (cc) service:
kubectl port-forward svc/<service-name> 8080:8080
3. To open the HiveMQ control center for your cluster, navigate to http://localhost:8080 and enter the default control center credentials in the HiveMQ control center login dialog (username: admin and password: hivemq).

The control center dashboard for the cluster opens and displays detailed information for your HiveMQ cluster and MQTT clients.

To learn more about the HiveMQ control center, see HiveMQ Control Center.

When you deploy with the HiveMQ Helm Chart, the HiveMQ Kubernetes Operator provides an integration with the Prometheus Operator that includes a Grafana dashboard for monitoring your HiveMQ cluster.
By default, monitoring with Prometheus is not active. For information on how to enable this setting, see Prometheus Configuration.

Next Steps

To help you on your way as quickly as possible, the HiveMQ Helm Chart creates a HiveMQ cluster custom resource that uses predefined default settings. The settings of the HiveMQ cluster that is created are suitable for testing purposes only.

Before you use your HiveMQ cluster in production, you must adapt the settings of the HiveMQ cluster custom resource to match the requirements of your use case.

Configure Your HiveMQ Cluster for Production

To change the configuration of your HiveMQ cluster with Helm (for example, the cluster created in the Quick Start Guide), see Revise HiveMQ Cluster Configuration with Helm.

To change the configuration of your HiveMQ cluster when you deploy the operator with manifests, see Adapt Manifests

For a complete overview of all the configuration information that is available in the HiveMQ custom resource definition, you can view and download the complete hivemqCluster.json schema file.