Observability
Kubernetes Events
Events provide detailed information that helps you monitor your HiveMQ platforms on Kubernetes. Events are generated when a resource such as your HiveMQ platform changes state. Events are not persisted in Kubernetes and are typically only available for one hour.
The HiveMQ Platform Operator publishes events whenever the state of a managed HiveMQ platform changes. Events from the operator allow for effective alerting and monitoring of your HiveMQ platform operations.
To view events in Kubernetes, enter:
kubectl get events
To view events from a selected namespace sorted by the time the event was created, enter:
kubectl get events --namespace <namespace> --sort-by='.metadata.creationTimestamp'
To view events from a HiveMQ Platform object, enter:
kubectl get events --field-selector involvedObject.kind=HiveMQPlatform
TYPE REASON OBJECT MESSAGE
Normal RollingRestart hivemqplatform/hmq-pl HiveMQ Platform is starting a surge Pod (rolling restart)
Normal RollingRestart hivemqplatform/hmq-pl HiveMQ Platform is restarting Pods (rolling restart)
Normal RollingRestart hivemqplatform/hmq-pl HiveMQ Platform is stopping the surge Pod (rolling restart)
Normal Ready hivemqplatform/hmq-pl HiveMQ Platform is ready
Monitoring with Prometheus
The HiveMQ Platform Operator and the HiveMQ platform provide rich metrics that enable extensive monitoring of day-to-day operations. Monitoring is in fact a crucial part of observability and any production HiveMQ deployment should not be used without monitoring setup.
To visualize metrics, you can use a monitoring stack consisting of Prometheus and Grafana on Kubernetes along with the HiveMQ Helm charts. To simplify and ease the monitoring of the HiveMQ Platform Operator and HiveMQ Platform, the HiveMQ Helm charts can provide a Prometheus ServiceMonitor resource along with the corresponding Grafana dashboard available via a ConfigMap.
Prometheus ServiceMonitor resources enable the Prometheus operator to scrape metrics from all pods the Kubernetes Service endpoint defines. The HiveMQ Platform Operator ServiceMonitor uses the endpoints the created operator service defines. The HiveMQ Platform ServiceMonitor uses the endpoints the created metric service defines. |
The following steps show how to install Prometheus and Grafana and how to configure the HiveMQ Helm charts to monitor HiveMQ:
-
Add the Prometheus community Helm charts to your local Helm repository:
helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
-
Create a dedicated Kubernetes namespace for monitoring:
kubectl create namespace monitoring
-
Install the monitoring stack into the new namespace:
helm upgrade --install monitoring-stack prometheus-community/kube-prometheus-stack -n monitoring
-
Add the following section to your HiveMQ Platform Operator
platform-operator-values.yaml
configuration file:monitoring: enabled: true dashboard: namespace: monitoring serviceMonitor: namespace: monitoring releaseName: monitoring-stack # Helm release name of prometheus-stack
-
Install the HiveMQ Platform Operator with the
platform-operator-values.yaml
configuration file:helm upgrade --install <your-hivemq-platform-operator> hivemq/hivemq-platform-operator -f platform-operator-values.yaml
-
Add the following section to your HiveMQ Platform
platform-values.yaml
configuration file:monitoring: enabled: true dashboard: namespace: monitoring serviceMonitor: namespace: monitoring releaseName: monitoring-stack # Helm release name of prometheus-stack
-
Install the HiveMQ Platform with the
platform-values.yaml
configuration file:helm upgrade --install <your-hivemq-platform> hivemq/hivemq-platform -f platform-values.yaml
-
Test that the Prometheus dashboard is accessible.
kubectl port-forward svc/monitoring-stack-kube-prom-prometheus 9090 -n monitoring
-
Browse to
http://localhost:9090
and view the Prometheus dashboard. -
Test that the Grafana dashboard is accessible.
kubectl port-forward svc/monitoring-stack-grafana 8080:80 -n monitoring
-
Browse to
http://localhost:8080
and view the Grafana dashboard.
The default login credentials for the Grafana dashboard are user:admin
and password:prom-operator
.