HiveMQ Kubernetes Operator (legacy)
Starting in April 2024, HiveMQ 4.28 becomes the Long Term Support (LTS) version of the HiveMQ Platform.
The HiveMQ Platform Operator for Kubernetes (new) provides usability, stability, and performance enhancements and is the supported operator for HiveMQ 4.28 and future releases.
Support and updates for the preceding HiveMQ 4.9 LTS version and the older HiveMQ Kubernetes Operator (legacy) will continue for 6 months after the HiveMQ 4.28 release, ending in November 2024. For more information, see Flexible MQTT Platform Upgrades with HiveMQ and HiveMQ Versioning. |
We highly recommend using the HiveMQ Platform Operator for your HiveMQ deployments on Kubernetes to ensure uninterrupted updates and support long into the future. For more information, see HiveMQ Platform Operator for Kubernetes.
The HiveMQ Kubernetes Operator is an application-specific controller that helps orchestrate and manage the lifecycle of a HiveMQ cluster deployment within Kubernetes.
The HiveMQ Kubernetes Operator runs as a custom controller on Kubernetes and communicates with the Kubernetes API Server (kube-api server) to convert high-level descriptions into normal Kubernetes resources that maintain the desired application state:
-
Automate orchestration of your entire HiveMQ deployment (platform agnostic) on Kubernetes
-
Monitor, maintain, recover, and upgrade your HiveMQ application with ease
-
Automatically deploy and operate HiveMQ on your Kubernetes cluster with custom or enterprise extensions
The HiveMQ Kubernetes Operator can run and maintain multiple HiveMQ clusters.
For production deployments of HiveMQ on Kubernetes, we recommend exclusive allocation of one HiveMQ pod to one Kubernetes node. For more information, see Assigning Pods to Nodes. |
The storage option you select determines the level of performance your HiveMQ Broker can deliver. To ensure high durability and optimal performance, we recommend the use of dedicated persistent volumes that deliver a minimum of 900 IOPS for HiveMQ in production. For more information, see HiveMQ Minimum Hardware Requirements. |
Features
-
Predefined HiveMQ Helm Chart: Quickly deploy the HiveMQ Operator and HiveMQ Cluster custom resource with convenient default settings that can be used as defined or adapted to meet your individual needs.
-
Seamless HiveMQ Upgrades: Built-in automated zero-downtime upgrade feature for your HiveMQ installation that ensures continuous availability during software upgrades.
-
Extension Lifecycle Integration: Lifecycle management for the entire range of extensions the HiveMQ extension framework provides:
-
Add or remove extensions at runtime
-
Update extension configurations without restarting pods
-
Support for serial and parallel update strategies
-
-
Monitoring with Prometheus: Easy integration with your Prometheus Operator. All HiveMQ metrics are included in the integration and can be monitored with the tool of your choice.
Requirements
-
Running Kubernetes cluster version 1.18.0 or higher on the cloud provider of your choice
-
Helm version 3 or higher (to deploy with the HiveMQ Helm Chart)
-
Current version of the Kubernetes command line tool kubectl
Mandatory Custom Resource Definition Update
Since HiveMQ 4.4, the HiveMQ Kubernetes Operator is part of the HiveMQ platform release bundle.
Since HiveMQ 4.7, the HiveMQ Kubernetes Operator supports the stable v1 api version of CustomResourceDefinition.
Use of the v1 CustomResourceDefinition requires a mandatory update of your CRD.
The update does not require any downtime.
For more information, see HiveMQ 4.6 to 4.7 Migration Guide.
|
HiveMQ Kubernetes Operator Quick Start Guide
Use the preconfigured HiveMQ Helm Chart to install a HiveMQ cluster on Kubernetes with the HiveMQ Kubernetes Operator.
Prepare
Take a moment to make sure that you have all the necessary tools and resources on hand:
-
A running Kubernetes cluster on any environment or providers cloud provider
-
Helm version 3 or higher
-
kubectl Kubernetes command line tool
-
MQTT CLI MQTT command line tool
This quick-start guide shows you how to deploy a HiveMQ cluster on Kubernetes that is suitable for testing purposes only. To configure the HiveMQ cluster for production, see Configure Your HiveMQ Cluster for Production. |
Install
Use Helm to install the HiveMQ Kubernetes Operator on Kubernetes and deploy a HiveMQ cluster with the default HiveMQ settings:
The HiveMQ Kubernetes Operator creates a default three-node HiveMQ cluster custom resource during installation. Each node is configured to request 4 CPUs, 4GB memory, and 15GB of disk space. If you want to test the HiveMQ Kubernetes Operator on a local Kubernetes cluster, decrease the amount of requested resources to match your system resources. |
helm repo add hivemq https://hivemq.github.io/helm-charts
helm upgrade --install hivemq hivemq/hivemq-operator
This command does the following:
-
Extends the Kubernetes API with the HiveMQ Cluster custom resource definition (CRD).
-
Installs the HiveMQ Kubernetes Operator as a custom controller.
-
Adds a HiveMQ Cluster custom resource (CR) that contains standard default HiveMQ configuration settings.
-
Saves this instance of the chart with the release name
hivemq
. For more information, see Helm releases.
The HiveMQ Kubernetes Operator uses the configuration settings that are defined in the HiveMQ Cluster custom resource to create and maintain a HiveMQ cluster on your Kubernetes cluster.
kubectl get hivemq-clusters
The output lists key information about your HiveMQ cluster including the number of cluster nodes and a human readable description of the current state of the cluster.
Adapt
The HiveMQ cluster custom resource that the HiveMQ Kubernetes Operator creates has standard default settings. To complete this quick start guide and connect MQTT clients you must adapt the default settings of the HiveMQ Cluster custom resource. You can use the same method to adapt your HiveMQ cluster settings for production.
When you modify the settings of your HiveMQ cluster custom resource, the HiveMQ Kubernetes Operator initiates a rolling upgrade of your HiveMQ cluster on Kubernetes. To maintain a steady load distribution during the upgrade, the operator starts a new HiveMQ pod before it shuts down and upgrades an existing HiveMQ pod. This process temporarily increases the HiveMQ cluster by one pod. The additional HiveMQ pod requires proportionally more system resources. For example, enough CPUs, memory, and disk storage. Once the rolling upgrade is complete, the HiveMQ cluster returns to the configured number of pods and resources. |
helm get values <release-name>
You can adapt the values of the settings as desired. For more information, see Basic Configuration.
These settings scale the HiveMQ cluster to 4 nodes, disallow empty client IDs, expose the MQTT port 1883 as a LoadBalancer for MQTT clients from outside the Kubernetes cluster to connect to your HiveMQ cluster, and make it possible to connect to the HiveMQ control center service.
hivemq:
nodeCount: "4"
security:
allowEmptyClientId: false
ports:
- name: "mqtt"
port: 1883
expose: true
patch:
- '[{"op":"add","path":"/spec/selector/hivemq.com~1node-offline","value":"false"},{"op":"add","path":"/metadata/annotations","value":{"service.spec.externalTrafficPolicy":"Local"}}]'
# If you want Kubernetes to expose the MQTT port
- '[{"op":"add","path":"/spec/type","value":"LoadBalancer"}]'
- name: "cc"
port: 8080
expose: true
patch:
- '[{"op":"add","path":"/spec/sessionAffinity","value":"ClientIP"}]'
# If you want Kubernetes to expose the HiveMQ control center via load balancer.
# Warning: You should consider configuring proper security and TLS beforehand. Ingress may be a better option here.
# - '[{"op":"add","path":"/spec/type","value":"LoadBalancer"}]'
helm upgrade --install -f myCustomValues.yaml hivemq hivemq/hivemq-operator
kubectl get hivemq-clusters
As soon as the displayed state returns to RUNNING
, your HiveMQ cluster uses the new settings.
helm show values hivemq/hivemq-operator
The expected result is that the values shown include the changes you entered in your myCustomValues.yaml file.
Connect
Use the MQTT CLI to connect MQTT clients to your HiveMQ cluster on Kubernetes and start sending MQTT messages.
We use the open source MQTT CLI for ease of use and to simplify the explanation of this sample procedure.
However, feel free to use whichever tool and method you prefer. Download packages with complete installation instructions for the MQTT CLI are available at no cost on GitHub. An introductory tutorial for the MQTT CLI is available on YouTube. |
kubectl get hivemq-clusters
This shows an endpoint URL to which you can connect MQTT clients.
If the Kubernetes cluster can provide load balancers through a cloud controller and the port is configured as type: LoadBalancer using a patch, an endpoint URL is listed.
If the Kubernetes cluster lacks cloud integration, no endpoint URL appears in the status of the custom resource.
|
$ kubectl get hivemq-clusters
NAME SIZE IMAGE VERSION STATUS ENDPOINT MESSAGE
cluster1 3 hivemq/hivemq4 4.7.0 Running 10.2.7.144 Live updates completed successfully
mqtt sh
The MQTT CLI starts and displays a list of useful options and commands.
Make sure that you expose the MQTT port as described in the section Adapt deployment settings. For MQTT clients to connect to your HiveMQ cluster, port 1883 must be exposed . |
con -h <your-cluster-url>
Information about the HiveMQ connection displays.
This is your first connected MQTT client.
mqtt sh
Copy and paste the connection command that you created in the first window to the second window and press enter. Information about the second HiveMQ connection displays. This is your second connected MQTT client.
Test
Publish MQTT messages and subscribe to MQTT topics with your connected MQTT clients.
In shell mode, adding the -s flag to the command blocks the MQTT CLI console so that the incoming messages are printed out to the user. |
sub -t testTopic -s
This command subscribes the second MQTT client to all messages that are published with the topic testTopic.
pub -t testTopic -m Hello
This command makes the first MQTT client a publisher and publishes the message Hello with the topic testTopic.
The message Hello appears immediately in the terminal window of the second MQTT client.
pub -t testTopic -m Goodbye
This command publishes the message Goodbye with the topic testTopic.
The message Goodbye appears immediately in the terminal window of the second MQTT client.
To view the activity of your new MQTT clients on the HiveMQ control center, see Monitor MQTT Clients on Your HiveMQ Control Center.
Monitor
Observe the activity of your MQTT clients on your HiveMQ Control Center.
The HiveMQ control center provides a wide range of metrics that help you manage and analyze your cluster. Use the detailed information the control center provides to maintain a clear overview of your clusters and quickly identify irregular client behavior.
kubectl get services -l hivemq-cluster=my-hivemq-cluster
kubectl port-forward svc/<service-name> 8080:8080
http://localhost:8080
and enter the default control center credentials in the HiveMQ control center login dialog (username: admin and password: hivemq).The control center dashboard for the cluster opens and displays detailed information for your HiveMQ cluster and MQTT clients.
To learn more about the HiveMQ control center, see HiveMQ Control Center.
When you deploy with the HiveMQ Helm Chart, the HiveMQ Kubernetes Operator provides an integration with the Prometheus Operator that includes a Grafana dashboard for monitoring your HiveMQ cluster. By default, monitoring with Prometheus is not active. For information on how to enable this setting, see Prometheus Configuration. |
Next Steps
To help you on your way as quickly as possible, the HiveMQ Helm Chart creates a HiveMQ cluster custom resource that uses predefined default settings. The settings of the HiveMQ cluster that is created are suitable for testing purposes only.
Before you use your HiveMQ cluster in production, you must adapt the settings of the HiveMQ cluster custom resource to match the requirements of your use case.
Configure Your HiveMQ Cluster for Production
For production usage, we highly recommend that you adjust your HiveMQ cluster configuration as follows:
-
Remove the allow-all extension that allows all MQTT clients to connect to HiveMQ without authentication.
-
Add a security extension that allows MQTT clients to connect to HiveMQ using authentication and authorization mechanisms. For example, the HiveMQ Enterprise Security Extension.
-
Add a loadbalancer that handles the connections to your HiveMQ Cluster.
-
Add a valid HiveMQ license to enable the usage of HiveMQ and our Enterprise Extensions beyond the freely available trial mode.
To change the configuration of your HiveMQ cluster with Helm (for example, the cluster created in the Quick Start Guide), see Revise HiveMQ Cluster Configuration with Helm.
To change the configuration of your HiveMQ cluster when you deploy the operator with manifests, see Adapt Manifests
For a complete overview of all the configuration information that is available in the HiveMQ custom resource definition, you can view and download the complete hivemqCluster.json
schema file.