HiveMQ Cloud Installation

HiveMQ is a cloud-native MQTT messaging broker that is enterprise-ready and well-designed for organizations that want to build and run scalable applications in modern, dynamic environments such as public, private, and hybrid clouds.

Depending on your use case, you can select flexible HiveMQ Cloud packages that provide a fully-managed messaging platform that does not rely on the services of a particular cloud vendor. Or, you can install HiveMQ on the cloud service of your choice.

For your convenience, we have compiled basic installation information for some frequently-used public cloud services:

Microsoft Azure
- Deploy with an Azure Resource Manager quickstart template
- Deploy with Azure Kubernetes Service (AKS)
Amazon Web Services (AWS)
- Launch an AWS EC2 installation for HiveMQ
- Build a production-ready HiveMQ Cluster on AWS EC2

To try out HiveMQ Cloud on different cloud platforms, including Microsoft Azure or AWS, go to HiveMQ Cloud.

HiveMQ on Microsoft Azure

There are several ways to deploy HiveMQ on Azure. The following procedures walk you through how to deploy a HiveMQ cluster on Azure with an ARM quickstart template or with Azure Kubernetes Service (AKS):

Quick-Launch HiveMQ Virtual Machine Cluster on Azure with HiveMQ ARM Quickstart Template

The arm-quickstart-templates folder that is included in the hivemq-azure-cluster-discovery-extension contains a hivemq-vm-cluster ARM template that lets you quickly deploy one or more HiveMQ instances on Microsoft Azure.

The following installation sets up a fully operational HiveMQ virtual machine cluster on your Azure account with the sizing of your choice.

HiveMQ ARM Quickstart Templates

Azure Resource Manager is the deployment and management service for Azure. A resource manager template is a JSON file that defines one or more resources to deploy to a resource group, subscription, management group, or tenant. The template can be used to deploy resources consistently and repeatedly.

The HiveMQ Virtual Machines Cluster ARM template deploys a cluster of HiveMQ virtual machines to Azure with the number of HiveMQ instances you select. Each HiveMQ instance (node) is one virtual machine. The HiveMQ Azure Cluster Discovery Extension that the template installs on each node enables the HiveMQ cluster nodes to discover each other dynamically through the regular exchange of information via Azure blobs in an Azure blob-storage container.

The hivemq-vm-cluster ARM quickstart template automates deployment of the following resources to Azure:

One or more virtual machines preinstalled with HiveMQ and the HiveMQ Azure Cluster Discovery Extension
An Azure Storage Account for the extension to use
An Azure Availability Set in which the virtual machines are located
An Azure load balancer to access the HiveMQ services
A virtual network in which the virtual machines and load balancer are located
Network interfaces with public IP addresses for the virtual machines
A network interface with a public IP-Address for the load balancer

Requirements

Microsoft Azure account
GitHub account
HiveMQ License (optional)

If you do not have a valid HiveMQ license, the HiveMQ cluster uses a trial licence that allows up to 25 concurrent client connections and is limited to testing and evaluation purposes only. To obtain a HiveMQ licence that is suitable for production use, or request an evaluation licence that allows more connections, contact our sales team.

Deploy Your HiveMQ Cluster with an ARM Quickstart Template

Before you get started, make sure that you have the sign-in information for your Microsoft account available.

Go to the HiveMQ Azure Cluster Discovery repository on GitHub and select arm-quickstart-templates/hivemq-vm-cluster.
On the HiveMQ Virtual Machine Cluster page, click Deploy to Azure:

A window to sign in to your Microsoft account opens:

After you sign in to your Microsoft account, a page to edit the customized template you are about to deploy opens:

On the Custom deployment page, configure your Azure Resource Manager template to meet the needs of your individual use case. Required fields are marked with an asterisk (*):

Project details
Subscription *	Select the Microsoft Azure subscription in which the resource group is crested. All resources in an Azure subscription are billed together.
Resource group *	Select the container Azure uses to hold the resources related to your HiveMQ ARM template. Select Create new to enter a name for your new resource group.

Project details

Subscription *

Select the Microsoft Azure subscription in which the resource group is crested. All resources in an Azure subscription are billed together.

Resource group *

Select the container Azure uses to hold the resources related to your HiveMQ ARM template.
Select Create new to enter a name for your new resource group.

Instance details
Region *	Select the Azure region in which the virtual machine is deployed.
Admin Username	Define a name for the virtual machine. To successfully complete the deployment, this name must be set.
Authentication Type *	Select the type of authentication the deployment requires. The following two options are possible: Password SSH Public Key
Admin password or SSH Key	Based on the authentication type you select, enter the password or SSH key of the virtual machine. For more information, see Create and use an SSH public-private key pair for Linux VMs in Azure.
VM Size *	Select the size of virtual machine you want to deploy. The default VM size is Standard_F4s_v2.
HiveMQ Version	Enter the version of HiveMQ you want to deploy. The default version is 4.6.0.
Number of Instances	Enter the number of virtual machine you want to deploy in the cluster. The default value is 2.

Instance details

Region *

Select the Azure region in which the virtual machine is deployed.

Admin Username

Define a name for the virtual machine. To successfully complete the deployment, this name must be set.

Authentication Type *

Select the type of authentication the deployment requires. The following two options are possible:

Password
SSH Public Key

Admin password or SSH Key

Based on the authentication type you select, enter the password or SSH key of the virtual machine. For more information, see Create and use an SSH public-private key pair for Linux VMs in Azure.

VM Size *

Select the size of virtual machine you want to deploy. The default VM size is Standard_F4s_v2.

HiveMQ Version

Enter the version of HiveMQ you want to deploy. The default version is 4.6.0.

Number of Instances

Enter the number of virtual machine you want to deploy in the cluster. The default value is 2.

To save your changes and open a validation window, select Review + Create:

To confirm the template configuration and continue with the deployment, review the Azure terms of service and select Create.

Your Microsoft Azure account bills you for all resources you create. When you delete a template deployment, the resources you created are not affected. To remove resources, you must delete the resource group. Deletion of a resource group permanently deletes all resources in the resource group and cannot be undone.

An overview window opens and shows the progress of your deployment:

To view progress information for each resource in the deployment as it is added, expand the Deployment details section.

The azuredeploy.json file of the hivemq-vm-cluster ARM template lists the parameter and variable configurations that the resource deploys.

Once your deployment is complete, a deployment overview for your template opens:

Since version 4.3, HiveMQ only allows MQTT clients to connect if a security extension is present. For testing purposes, HiveMQ includes a hivemq-allow-all-extension that authorizes all MQTT clients to connect to HiveMQ. Before you use HiveMQ in production, you must add an appropriate security extension and remove the hivemq-allow-all-extension. You can download security extensions from the HiveMQ Marketplace or develop your own security extension.

If you have a valid HiveMQ license, install the license information on each HiveMQ node in your cluster.
For more information, see Install a HiveMQ License. Skip this step if you are using a HiveMQ trial license.
To verify that your HiveMQ cluster is running, check to see whether your HiveMQ Control Center is accessible:
- If you are using a load balancer, make sure that you activate session persistence in your Azure load balancer configuration. Navigate with your browser to the public IP address of the load balancer: http://your-load-balancer-public-IP-address:8080.
- If you are not using a load balancer, navigate with your browser to the public IP address of one of the nodes in your HiveMQ cluster: http://your-cluster-node-public-IP-address:8080.

When you see the login dialog for the HiveMQ Control Center, you know that HiveMQ is up and running.

The HiveMQ Control Center is part of the standard HiveMQ distribution and is a web application that you can access from your browser for quick insights into your HiveMQ cluster.

By default, the HiveMQ broker starts on localhost IP address (127.0.0.1) and listens on port 1883.

You do not need to install any additional software on your computer to use the control center. The default login credentials are: user: admin, password: hivemq. For more information, see HiveMQ Control Center.

When you no longer need your HiveMQ cluster on Azure, be sure to delete the Azure resource group that you created. Deletion of an Azure resource group permanently removes the resource group and all associated resources. This action can not be undone. Deletion of an Azure deployment does not affect the Azure resource group or the associated resources.

Deploy HiveMQ with the Azure Kubernetes Service (AKS) and the HiveMQ Kubernetes Operator

Azure Kubernetes Service (AKS) is a fully managed Kubernetes service that is designed to simplify the deployment and management of containerized applications. AKS offers serverless Kubernetes, integrated continuous integration and continuous delivery (CI/CD), and enterprise-grade security and governance.

The HiveMQ Kubernetes Operator is an application-specific controller that makes it easy to deploy, configure, and run high-availability HiveMQ clusters on Kubernetes. The HiveMQ operator is built to work with any Kubernetes (K8s) cluster that runs Kubernetes version 1.13 or higher (on premises or in the cloud). With the HiveMQ Kubernetes Operator, you can automate orchestration of your entire HiveMQ cluster deployment on AKS with a minimal amount of effort.

The combination of AKS and the HiveMQ Kubernetes Operator allows you to set up a fully operational HiveMQ cluster on Azure with just a few steps.

To get you started, we have created an example procedure that shows you how to accomplish the following:

Create an Azure Resource Group with the Azure CLI in which you deploy a 4-node Kubernetes cluster.
Install the HiveMQ Kubernetes Operator with Helm.
Use the HiveMQ Kubernetes Operator with a customized YAML file to install a 3-node HiveMQ cluster and set up Prometheus and Grafana on the fourth node to monitor HiveMQ.
Define a load balancer with the HiveMQ Kubernetes Operator that you can use to connect MQTT clients to your HiveMQ cluster using the open-source MQTT-CLI tool.

Requirements

Microsoft Azure Account
HiveMQ License (optional)

Azure CLI
The Azure command-line interface is used to create and manage Azure resources. In this procedure, you use the Azure CLI to create an Azure Resource Group with a Kubernetes cluster on the Azure cloud.
To install the Azure CLI on macOS with Homebrew, open a terminal and enter:

brew install az

For other operating systems, see Azure CLI installation.

kubectl
Kubectl is the official command-line interface for interacting with Kubernetes clusters. Here, you use kubectl to manage the created Kubernetes cluster on Azure.
Once you install Azure CLI, you can enter the following command to install kubectl on any operating system:

az aks install-cli

Helm
Helm is the most commonly used package manager for Kubernetes. Helm helps you easily define, install, and update the software that you run on a Kubernetes cluster. In this procedure, you use Helm to install and configure HiveMQ on your K8s cluster.
To install Helm on macOS with Homebrew, enter:

brew install helm

For other operating systems, see Helm installation.

MQTT-CLI
MQTT-CLI is an open-source command-line interface from HiveMQ that allows you to quickly simulate MQTT clients. Here, you use the MQTT-CLI to connect clients and interact with your HiveMQ cluster.
To install MQTT-CLI on macOS with Homebrew, enter:

brew install hivemq/mqtt-cli/mqtt-cli

For other operating systems, see MQTT-CLI installation.

Procedure

Set Up Your Kubernetes Cluster With AKS

To create a Kubernetes cluster on Azure with the Azure CLI, open a terminal and enter:

az login

A window to sign in to your Microsoft account opens in your browser:

After you successfully submit your credentials, a sign-in confirmation prints in your terminal.

Before you create the cluster, you need to select the region where the cluster is hosted. To view a list of all available locations, enter:

az account list-locations

Excerpt of Azure locations JSON

  ...

  },
  {
    "displayName": "Germany North",
    "id": "/subscriptions/661ab844-9998-4e03-b857-62ecbd9f34e8/locations/germanynorth",
    "metadata": {
      "geographyGroup": "Europe",
      "latitude": "53.073635",
      "longitude": "8.806422",
      "pairedRegion": [
        {
          "id": "/subscriptions/661ab844-9998-4e03-b857-62ecbd9f34e8/locations/germanywestcentral",
          "name": "germanywestcentral",
          "subscriptionId": null
        }
      ],
      "physicalLocation": "Berlin",
      "regionCategory": "Other",
      "regionType": "Physical"
    },
    "name": "germanynorth",
    "regionalDisplayName": "(Europe) Germany North",
    "subscriptionId": null
  },
...

To create an Azure Resource Group with all the necessary resources for your cluster, enter the following command with the desired location (in this procedure, we use germanywestcentral):

az group create --name hmqResourceGroup --location germanywestcentral

To create a four-node AKS-managed Kubernetes cluster in your resource group with the Azure virtual machine type Standard_A8_v2 (8 CPUs, 16 GiB RAM), enter the following command and wait until the process completes.
Processing time can vary:

az aks create -g hmqResourceGroup -n HiveMQCluster --node-count 4 --node-vm-size Standard_A8_v2

Your Microsoft Azure account bills you for all resources you create.

To manage the resulting Kubernetes cluster with kubectl, download the access credentials of the cluster:

az aks get-credentials -g hmqResourceGroup -n HiveMQCluster

To verify that all four nodes are available, enter:

kubectl get nodes

The output from the command is similar to the following:

NAME                                STATUS   ROLES   AGE     VERSION
aks-nodepool1-37471664-vmss000000   Ready    agent   5m44s   v1.18.10
aks-nodepool1-37471664-vmss000001   Ready    agent   5m50s   v1.18.10
aks-nodepool1-37471664-vmss000002   Ready    agent   5m47s   v1.18.10
aks-nodepool1-37471664-vmss000003   Ready    agent   5m47s   v1.18.10

Add A License To Your HiveMQ Cluster (skip this step if you are using a HiveMQ trial license)

If you have a valid HiveMQ license, create a hivemq-license Kubernetes ConfigMap from your licence file to make the licence information accessible for the HiveMQ deployment you execute in the next step:

kubectl create configmap hivemq-license --from-file=hivemq-license.lic

Deploy HiveMQ On Your Kubernetes Cluster

To use the HiveMQ Kubernetes Operator to deploy and manage your HiveMQ cluster on the Kubernetes nodes, you need to add the HiveMQ Helm repository to your Helm installation:

helm repo add hivemq https://hivemq.github.io/helm-charts

To set up the desired configuration of your HiveMQ cluster, define and save a custom configuration YAML file. In this example, the configuration creates a 3-node HiveMQ cluster with a load balancer that exposes the MQTT port, access to the HiveMQ Control Center, and a dedicated monitoring instance. Save the following configuration as a yaml file called hivemq.yaml:

hivemq:
  nodeCount: "3"

  ports:
    - name: "mqtt"
      port: 1883
      expose: true
      patch:
        - '[{"op":"add","path":"/spec/type","value":"LoadBalancer"}]'
    - name: "cc"
      port: 8080
      expose: true
      patch:
        - '[{"op":"add","path":"/spec/sessionAffinity","value":"ClientIP"}]'

 #  Uncomment the following lines if you have a valid HiveMQ license
 #  configMap:
 #   - name: hivemq-license
 #     path: /opt/hivemq/license

monitoring:
  dedicated: true
  enabled: true

Install the HiveMQ Kubernetes Operator to deploy the configured HiveMQ cluster from the defined hivemq.yaml file:

helm upgrade --install -f hivemq.yaml hivemq hivemq/hivemq-operator

To verify that your cluster is running, enter:

kubectl get hivemq-clusters

As soon as the STATUS of the cluster changes to Running, you can proceed to next steps.

Connect To Your HiveMQ Cluster

Once your HiveMQ cluster is up and running, you are ready to connect clients to the cluster with the MQTT-CLI.
To obtain the exposed ENDPOINT of the created load balancer on which clients connect to your cluster, enter:

kubectl get hivemq-clusters

Output similar to the following displays:

$ kubectl get hivemq-clusters
NAME     SIZE   IMAGE            VERSION     STATUS    ENDPOINT       MESSAGE
hivemq   3      hivemq/hivemq4   k8s-4.4.3   Running   20.52.209.35   Live updates completed successfully

To start up the MQTT-CLI in shell mode, enter:

mqtt sh

In shell-mode, connect a new MQTT client to your HiveMQ cluster on the previously acquired endpoint address:

con -h 20.52.209.35

You have now successfully connected a new MQTT client to a HiveMQ cluster hosted on Azure.
To learn more ways to interact with HiveMQ, see the MQTT-CLI documentation.

Monitor Your HiveMQ Cluster

Monitoring your MQTT brokers is vital, especially in clustered environments. The highly-performant metrics subsystem of HiveMQ lets you monitor relevant metrics with no reduction in system performance.

HiveMQ Control Center

The HiveMQ Control Center provides a quick and convenient way to get insights into your HiveMQ cluster.
To access your HiveMQ Control Center, forward the port of the control center to our local machine:

kubectl port-forward svc/hivemq-hivemq-cc 8080:8080

This command blocks the console and makes the HiveMQ Control Center available on http://localhost:8080.
You do not need to install any additional software on your computer to use the control center.
To login, enter the default login credentials: user: admin, password: hivemq.

The dashboard of your HiveMQ Control Center opens:

Grafana

The HiveMQ Kubernetes Operator provides a Grafana dashboard that you can use to gain an in-depth view into your HiveMQ deployment.
To access the Grafana dashboard, forward the Grafana port to your local machine:

kubectl port-forward svc/hivemq-grafana 8081:80

To open your Grafana dashboard, go to http://localhost:8081 and log in with the default login credentials: username: admin, password: prom-operator.
Select the HiveMQ Cluster dashboard to view the most important HiveMQ metrics:

Edit the Configuration of Your HiveMQ Cluster

The HiveMQ Kubernetes Operator makes it easy to adapt the configuration of your HiveMQ cluster on Kubernetes.

To tailor your HiveMQ cluster for your individual use case, adapt your HiveMQ custom configuration YAML file that you made previously (hivemq.yaml) and use Helm to apply the changes.
To view all the configuration options that you can edit, enter the following Helm command:

helm show values hivemq/hivemq-operator

After you modify your YAML file to your desired configuration, apply the changes with Helm:

helm upgrade --install -f hivemq.yaml hivemq hivemq/hivemq-operator

The HiveMQ Kubernetes Operator initiates a rolling upgrade of your cluster and updates the cluster to the new desired state.
To track the state of your cluster, enter:

kubectl get hivemq-clusters

Tear Down Your Azure Kubernetes Service Cluster

To delete everything that you just created, use the Azure CLI to delete your entire Azure resource group:

az aks delete -g hmqResourceGroup -n HiveMQCluster

This action permanently deletes the Azure resource group and all related resources. This action can not be undone.

HiveMQ on Amazon Web Services (AWS)

There are several ways to deploy HiveMQ on Amazon Web Service (AWS). The following procedure shows you how to easily deploy HiveMQ on an AWS EC2 instance:

Quick-Launch an AWS EC2 Installation for HiveMQ

HiveMQ pre-built Amazon Machine Images (AMI) help you deploy a HiveMQ instance on AWS EC2 with ease.
The following procedure sets up a fully operational HiveMQ AWS EC2 instance with the sizing of your choice on your AWS account. You do not need to provide any AWS credentials for this installation. A standard EC2 AWS web console dialog in your web browser is used and no data is transmitted to us.

Select Your AMI

The AMI that you select determines the AWS region where your HiveMQ instance is deployed. The following HiveMQ AMI for AWS are available:

AWS Region Description Action

AWS Region	Description	Action
*EU Central 1 (Frankfurt)*	Deploys HiveMQ on the AWS `eu-central-1` region.	*Launch eu-central-1 AMI*
*EU West 1 (Ireland)*	Deploys HiveMQ on the AWS `eu-west-1` region.	*Launch eu-west-1 AMI*
*EU West 2 (London)*	Deploys HiveMQ on the AWS `eu-west-2` region.	*Launch eu-west-2 AMI*
*EU West 3 (Paris)*	Deploys HiveMQ on the AWS `eu-west-3` region.	*Launch eu-west-3 AMI*
*US East 1 (N. Virginia)*	Deploys HiveMQ on the AWS `us-east-1` region.	*Launch us-east-1 AMI*
*US East 2 (Ohio)*	Deploys HiveMQ on the AWS `us-east-2` region.	*Launch us-east-2 AMI*
*US West 1 (N. California)*	Deploys HiveMQ on the AWS `us-west-1` region.	*Launch us-west-1 AMI*
*US West 2 (Oregon)*	Deploys HiveMQ on the AWS `us-west-2` region.	*Launch us-west-2 AMI*
*Canada (Central)*	Deploys HiveMQ on the AWS `ca-central-1` region.	*Launch ca-central-1 AMI*
*Asia-Pacific (Tokyo)*	Deploys HiveMQ on the AWS `ap-northeast-1` region.	*Launch ap-northeast-1 AMI*
*Asia-Pacific (Seoul)*	Deploys HiveMQ on the AWS `ap-northeast-2` region.	*Launch ap-northeast-2 AMI*
*Asia-Pacific (Singapore)*	Deploys HiveMQ on the AWS `ap-southeast-1` region.	*Launch ap-southeast-1 AMI*
*Asia-Pacific (Sydney)*	Deploys HiveMQ on the AWS `ap-southeast-2` region.	*Launch ap-southeast-2 AMI*
*South America (São Paulo)*	Deploys HiveMQ on the AWS `sa-east-1` region.	*Launch sa-east-1 AMI*

EU Central 1 (Frankfurt)

Deploys HiveMQ on the AWS eu-central-1 region.

Launch eu-central-1 AMI

EU West 1 (Ireland)

Deploys HiveMQ on the AWS eu-west-1 region.

Launch eu-west-1 AMI

EU West 2 (London)

Deploys HiveMQ on the AWS eu-west-2 region.

Launch eu-west-2 AMI

EU West 3 (Paris)

Deploys HiveMQ on the AWS eu-west-3 region.

Launch eu-west-3 AMI

US East 1 (N. Virginia)

Deploys HiveMQ on the AWS us-east-1 region.

Launch us-east-1 AMI

US East 2 (Ohio)

Deploys HiveMQ on the AWS us-east-2 region.

Launch us-east-2 AMI

US West 1 (N. California)

Deploys HiveMQ on the AWS us-west-1 region.

Launch us-west-1 AMI

US West 2 (Oregon)

Deploys HiveMQ on the AWS us-west-2 region.

Launch us-west-2 AMI

Canada (Central)

Deploys HiveMQ on the AWS ca-central-1 region.

Launch ca-central-1 AMI

Asia-Pacific (Tokyo)

Deploys HiveMQ on the AWS ap-northeast-1 region.

Launch ap-northeast-1 AMI

Asia-Pacific (Seoul)

Deploys HiveMQ on the AWS ap-northeast-2 region.

Launch ap-northeast-2 AMI

Asia-Pacific (Singapore)

Deploys HiveMQ on the AWS ap-southeast-1 region.

Launch ap-southeast-1 AMI

Asia-Pacific (Sydney)

Deploys HiveMQ on the AWS ap-southeast-2 region.

Launch ap-southeast-2 AMI

South America (São Paulo)

Deploys HiveMQ on the AWS sa-east-1 region.

Launch sa-east-1 AMI

Select an Instance Type

Define an appropriate instance size for your EC2 instance. To fulfill the minimum requirements for HiveMQ, use a m5.xlarge or c5.xlarge instance.

Configure Your Instance Details

The default settings are appropriate for most use cases.

Add Storage

The minimum recommended amount of storage is 20 GB. For production deployments, we recommend 100 GB or more.

Add Tags

Optional tags are useful for adding metadata to your instance. To make it easier to identify the launched instance on your AWS web console, add a Name tag.

Configure Security Groups

To make your HiveMQ instance accessible to your MQTT clients, you must expose port 1883. If you want your instance to be accessible over the Internet, select anywhere as the Source.

To access the HiveMQ Control Center, configure port 8080 in your security group. If you want SSH access to your machine, you must add an SSH rule.

Make sure that the HiveMQ Control Center port 8080 and port 22 are only accessible from your IP address or trusted IP addresses. Do not allow access to these ports over the Internet.

Launch the AMI

To launch the AMI, select Launch and wait for the EC2 instance to launch. This process usually takes a few minutes.

Navigate to your EC2 Instance Overview and go to the address of your newly spawned EC2 instance with HiveMQ:

Monitor on HiveMQ Control Center

The HiveMQ Control Center offers a wide range of analytics that can help you to closely examine information for each client and quickly identify irregular client behavior.

To access the HiveMQ Control Center, navigate with your browser to http://your-IP-address:8080 (use the address you located in the EC2 Instance Overview) and open the HiveMQ Control Center.

Log in with the default login credentials, user: admin password: hivemq.
If the HiveMQ Control Center is available, you know that HiveMQ is up and running.

Once the HiveMQ MQTT broker is running, you can use any MQTT client to connect to your HiveMQ MQTT broker on port 1883. For your convenience, the full-featured, open-source MQTT-CLI is included in the tools folder of your HiveMQ instance.

Build an Elastic High-Availability HiveMQ Cluster on AWS EC2

HiveMQ is a cloud-native MQTT broker that offers superior elastic clustering capabilities and a resilient software design that make it a perfect fit for common cloud infrastructures. When you run HiveMQ in production, the use of cloud infrastructure such as AWS ensures that elastic scalability and adequate security standards are in place on the infrastructure side.

The steps that we outline to set up and run an elastic HiveMQ cluster on AWS can be applied to many other cloud platforms.

Requirements

Amazon Web Services Account
HiveMQ License (optional)

Procedure

Amazon Webs Services prohibits the use of UDP multicast, which is the default HiveMQ cluster discovery mode. The use of Amazon Simple Storage Service (S3) buckets for auto-discovery is a perfect alternative when the individual HiveMQ broker nodes run on AWS EC2 instances. To simplify installation, HiveMQ offers a free pre-built extension for AWS S3 cluster discovery.

The following procedure shows you how to set up a HiveMQ cluster on AWS EC2 instances that use Amazon S3 buckets for automatic cluster discovery.

Create a Security Group

To allow inbound traffic to the listeners that you configure for MQTT communication in the following steps, you need to set up a security group. After you create the security group, you can edit the group and add a rule for the internal communication between cluster nodes on all TCP ports that uses the security group as the source. It is also essential to have SSH access on the instances.

To create a security group, go to the AWS Management Console and sign in to your account:

On the EC2 Dashboard, go to Network & Security | Security Groups, and select Create security group:

On the Create a security group page, enter the basic details for your new security group:

Basic details
Security group name	The name for your security group. The name cannot be edited after you create the security group.
Description	A description to help you identify the security group. Maximum 255 characters.
VPC	The Amazon virtual private cloud on which the security group is created.

Basic details

Security group name

The name for your security group. The name cannot be edited after you create the security group.

Description

A description to help you identify the security group. Maximum 255 characters.

VPC

The Amazon virtual private cloud on which the security group is created.

After you enter the basic details, select Create security group:

A page on which you can view and edit the newly created security group opens:

From the Actions menu, use the Edit Inbound rules action and add five rules to your security group configuration (Outbound rules are optional):

Table 1. Inbound rules example
Type	Protocol	Port range	Source	Description (optional)
All TCP	TCP	0 - 65535	ID number / YourSecurityGroupName	ID of the security group you created
Custom TCP	TCP	8000	0.0.0.0/0	-
SSH	TCP	22	Trusted IP address	Enter the IP of your trusted network
Custom TCP	TCP	8883	0.0.0.0/0	-
Custom TCP	TCP	1883	0.0.0.0/0	-

Create an S3 Bucket

Navigate to the Amazon S3 Console and select Create bucket:

On the Create bucket page, configure your S3 bucket:

We recommend that you select a region for the S3 bucket that is located near the region in which you plan to deploy HiveMQ.

When your configuration is complete, select Create bucket:

Create AWS Identity and Access Management (IAM) Role

We recommend that you configure your Amazon EC2 instances in a way that allows the instances to access your Amazon S3 Bucket. For more information, see Getting set up with IAM.

Install HiveMQ on AWS

Pre-configured HiveMQ Amazon Machine Images (AMI) provide all the information that you need to launch two HiveMQ broker nodes on the AWS EC2 instances.

Choose the AWS region to which you want to deploy your HiveMQ instances and select Launch:

Select the instance type that you want to deploy. For testing purposes, we recommend c5.xlarge:

Select Next: Configure Instance Details. Change the number of instances to 2 and adjust settings appropriately for your use case. Typically, you can use the defaults here:

Select Next: Add Storage and make sure that you have at least 20 GB storage (100 GB storage is recommended for production deployments):

Go to Configure Security Group and select the security group that you previously created:

Select Review and Launch to verify your instance details, select Launch to automatically spawn two separate EC2 instances that run HiveMQ as a service:

Your Amazon Web Services account bills you for all services that you launch.

Install and Configure HiveMQ S3 Cluster Discovery Extension

In the following steps, you enable the cluster mode on both of our HiveMQ instances and provide a way for the instances to discover each other with the HiveMQ S3 Cluster Discovery Extension:

Create an S3 Bucket that your HiveMQ instances can use, and make sure to remember the bucket name. You can use the default configuration at this time.
Repeat the following steps individually for each of your HiveMQ instances:
- Connect to the HiveMQ instance via SSH:

ssh -i <your-deployment-key> ec2-user@<instance-ip-address>

Switch to the root user:

sudo su

Download the HiveMQ S3 Cluster Discovery Extension:

wget https://www.hivemq.com/releases/extensions/hivemq-s3-cluster-discovery-extension-4.1.1.zip

Unzip the distribution (this action creates a hivemq-s3-cluster-discovery-extension folder:

unzip hivemq-s3-cluster-discovery-extension-4.1.1.zip

Open the HiveMQ S3 Cluster Discovery Extension configuration file (if desired, use a different text editor):

vi hivemq-s3-cluster-discovery-extension/hivemq-s3-cluster-discovery-extension.xml

Configure the S3 Bucket region and name:

############################################################
# S3 Bucket                                                #
############################################################

#
# Region for the S3 bucket used by hivemq
# see http://docs.aws.amazon.com/general/latest/gr/rande.html#s3_region for a list of regions for S3
# example: us-west-2
#
s3-bucket-region:<your-region>

#
# Name of the bucket used by HiveMQ
#
s3-bucket-name:<your-bucket-name>

Change ownership of the extension folder to the hivemq user:

chown -R hivmq:hivemq hivemq-s3-cluster-discovery-extension

To complete the installation of the S3 cluster discovery extension, move the folder of the extension to the HiveMQ extensions folder:

mv hivemq-s3-cluster-discovery-extension/ /opt/hivemq/extensions/

Adjust the HiveMQ /opt/hivemq/conf/config.xml configuration file as follows and enter the internal IP address of your EC2 instance as the bind address for TCP transport:

<?xml version="1.0"?>
<hivemq>

    <listeners>
        <tcp-listener>
            <port>1883</port>
            <bind-address>0.0.0.0</bind-address>
        </tcp-listener>
    </listeners>

    <cluster>
        <enabled>true</enabled>
        <transport>
            <tcp>
                <!--enter IP address of your EC2 instance-->
                <bind-address>IP_ADDRESS</bind-address>
                <bind-port>7800</bind-port>
            </tcp>
        </transport>

        <discovery>
            <extension/>
        </discovery>
    </cluster>

    <anonymous-usage-statistics>
        <enabled>true</enabled>
    </anonymous-usage-statistics>

    <control-center>
        <listeners>
            <http>
                <port>8080</port>
                <bind-address>0.0.0.0</bind-address>
            </http>
        </listeners>
    </control-center>
</hivemq>

Restart HiveMQ on both EC2 instances:

/etc/init.d/hivemq restart

Check the log statement in the /opt/hivemq/log/hivemq.log file. A statement similar to the following shows that the cluster is successfully established:

INFO - Cluster size = 2, members : [8Jojp, WlF1S]

Repeat the previous steps to create a HiveMQ cluster with the desired number of nodes.

Launch and Configure an AWS Network Load Balancer (NLB)

Add a load balancer to your setup that allows your HiveMQ broker cluster to act as a single logical broker node for all connecting MQTT clients. The MQTT client only needs to know the load-balancer URL to connect, publish, and subscribe.

On your AWS EC2 Dashboard, go to Load Balancing | Target Groups and select Create target group:

on the Specify group details page, configure the following:
- Select Instances as the target type.
- Enter your desired Target group name.
- Select TCP for the Protocol and enter 1883 for the Port.
- Select the VPC with the HiveMQ instances that you want to include in the target group.
- Select TCP as the Health check protocol. _
Select Next to review your configuration, then select Create target group_.
In the confirmation window that opens, select your newly created target group, and select Register targets:

Select your HiveMQ instances from the list of Available instances to add the instances to the list of pending targets. Select Register pending targets to register the targets:

Return to your AWS EC2 Dashboard and go to Load Balancing | Load Balancers | Create Load Balancer:

From the available load balancer options, select Network Load Balancer TCP, TLS, UDP:

On the Create Network Load Balancer page, configure your load balancer as follows:
- Enter a load balancer name.
- Select the Internet-facing scheme.
- Select IPv4 for the IP address type.
- In Networking mapping, configure your VPC and availability zones according to your needs. Best practise is to choose all availability zones.
- In Listeners and routing, select TCP as the Protocol of the listener, enter 1883 as the Port, and your target group as the Default action.
Select Create load balancer to create a network load balancer with the selected configuration:

Once the load balancer finishes provisioning, clients can connect to your HiveMQ cluster with the DNS name of the load balancer.

For production environments, best practice is to use automatic provisioning of the EC2 instances that eliminates the need to configure each EC2 instance manually. For example, tools such as Chef, Puppet, or Ansible.

It is also possible to use HiveMQ with Docker to simplify the provisioning of HiveMQ nodes. For more information, see