Control Center Analytics

To help you access helpful insights into your HiveMQ deployment quickly, your HiveMQ Control Center provides a wide range of analytics functionality.

For production environments, we highly recommend the use of a full monitoring setup in addition to your HiveMQ Control Center. There are many commercially-available monitoring solutions to choose from. For more information on the ready-to-use monitoring extensions HiveMQ offers, visit the HiveMQ Marketplace.

Dropped Messages

The Dropped Messages view in your HiveMQ Control Center enables you to analyze all dropped messages on your HiveMQ broker. A dropped message is a message that the broker does not publish to the intended recipient for a variety of reasons. Dropped messages never leave the broker.

HiveMQ provides the following information for dropped messages:

The reasons why messages dropped. For more information, see Dropped messages by reason.
The clients for which messages dropped. For more information, see Clients with dropped messages.
The shared subscription groups for which messages dropped. For more information, see Shared subscription groups with dropped messages.

Figure 1. Example Dropped Messages Overview

Dropped Messages by Reason

The Dropped Messages per Reason chart visualizes the causes and quantities of dropped messages on your broker over the last hour grouped by the reason.

Figure 2. Dropped Messages per Reason (stacked)

To adjust the reasons that are visible in the chart, select or deselect the reason in the legend of the chart.
Information automatically updates every 10 seconds.

The dropped message pie chart visualizes the total count and the percentage for each dropped message reason.

To view or hide a reason, select the reason in the legend of the chart.

Dropped Messages Reasons

Your HiveMQ broker can drop a message for the following reasons:

Table 1. Possible reasons for dropped messages
Reason	Description
Client Message Queue Full	The message queue of the client is full when the message arrives.
QoS 0 Memory Exceeded	The maximum memory allowed for QoS 0 messages on the broker has been exceeded.
QoS 0 Channel Not Writable	The QoS 0 message could not be delivered because it was impossible to write to the client socket.
Maximum Packet Size Exceeded	The size of the message exceeds the maximum message size that the receiving client allows.
Extension Prevented	An extension publish inbound interceptor prevented delivery of the message.
Internal Error	The message dropped due to an internal error on the broker.

Dropped Messages by Client

The Clients with Dropped Messages overview shows the first 100 clients per node that dropped messages over the last four days:

Table 2. Available dropped messages by client information
Column	Descriptions
ClientID	The ID of the client that dropped messages
First Dropped Message	The timestamp of the first message the client dropped
Last Dropped Message	The timestamp of the last message the client dropped
Client Message Queue Full	Indicates that the message queue of the client was already full when the new message for the client arrived.
QoS 0 Memory Exceeded	Indicates that the broker-wide memory for queueing Quality of Service level 0 (QoS 0) messages has been exceeded.
QoS 0 Channel Not Writable	Indicates that the QoS 0 message could not be delivered because it was impossible to write to the client socket.
Maximum Packet Size Exceeded	Indicates that the size of the incoming message exceeded the maximum message size the client allows.
Extension Prevented	Indicates that a publish inbound interceptor of an extension prevented delivery of the message.
Internal Error	Indicates that an internal error in the broker prevented message delivery.

The columns in the overview can be sorted in ascending or descending order and filtered by the client ID.

To open the Client Detail View of a client, select the client name. To clear all data shown in the detail view, select Clear Data. To refresh the data shown in the detail view, select Refresh Data.

Dropped Messages by Shared Subscription Groups

The Dropped Messages by Shared Subscription Groups overview shows the first 100 shared groups that have dropped messages over the last four days.

Shared Subscription Groups dropping messages

Table 3. Dropped messages by subscription group information
Column	Descriptions
Shared Group	The ID of the client that dropped messages
First Dropped Message	The timestamp of the first message the client dropped
Last Dropped Message	The timestamp of the last message the client dropped
Client Message Queue Full	Client’s message queue was full as another message for that client arrived.
QoS 0 Memory Exceeded	Broker wide memory for queueing Quality of Service 0 messages has been exceeded.
Internal Error	Messages that were dropped because of internal errors.

Message Drop Reasons in Detail

For most use cases, message delivery must be as reliable as possible and the dropping of messages is not a desired outcome. However, this does not mean that every dropped message represents a problem that requires your immediate attention. The HiveMQ Control Center offers an in-depth look into why messages are dropped and for which clients.
The following information outlines some possible causes of dropped messages and offers insights for debugging purposes.

Client Message Queue Full

The client message queue full reason indicates that the internal message queue of the client for incoming messages that are not yet processed has reached its maximum capacity. If your clients frequently experience this kind of dropped message, it can mean that your clients are not able to process received messages fast enough or that the network connection of the client is very unstable.

QoS 0 Memory Exceeded

Message queueing can occur when a client with a persistent session is temporarily offline or when the broker sends messages faster than the client can consume the messages. When the broker receives messages with Quality of Service 0 (QoS 0), the broker puts the message into a global in-memory queue to await delivery to the intended client.
If the size of the QoS 0 queue exceeds the configured memory limit of the broker, additional incoming QoS 0 messages are dropped.
If your clients frequently experience this kind of dropped message, it can indicate that your clients are consuming messages too slowly, that clients have unstable network connections, or that your broker is not configured with sufficient memory.

You can configure how many messages your HiveMQ broker is allowed to queue per client and how received messages that exceed the maximum queue size are handled. For more information, see Queued Messages.

QoS 0 Channel Not Writable

When the broker receives a message with QoS 0 and the socket of the intended receiving client cannot be written to, the broker drops the message.
This drop reason can indicate that the client is unable to read messages fast enough to keep up with the incoming messages.

Maximum Packet Size Exceeded

MQTT 5 allows you to set the maximum message packet size that an MQTT client accepts. If a message exceeds the allowed maximum packet size of the receiving client, the broker drops the message.
A high number of this kind of dropped messages can indicate suboptimal coordination between your sending and receiving clients.

Extension Prevented

The use of a publish inbound interceptor in an extension allows you to prevent the onward delivery of specific messages. If a publish inbound interceptor blocks delivery, the broker drops the message.
A best practice for extension development is to log and/or forward the reason for the delivery prevention to the intended client.

Internal Error

If an internal broker error occurs during the handling of a message, the broker drops the message. The reason for the internal error is not specified.
Messages that drop due to an internal error indicate an unexpected underlying problem. If you observe internal errors, check your HiveMQ log file for the referenced error or contact us for additional assistance.

Cluster Metrics

The dashboard of your HiveMQ Control Center provides a quick overview of the available cluster metrics and access to further metric details:

Figure 3. Metrics quick view on the control center dashboard

Table 4. Metrics
Metric	Description
Connections	The current total number of active connections for all cluster nodes.
Inbound Publish Rate	The current total number of incoming PUBLISH packets per second for all cluster nodes
Outbound Publish Rate	The current total number of outgoing PUBLISH packets per second for all cluster nodes.
Subscriptions	The current number of subscriptions and replicas stored in the cluster.
Retained Messages	The current number of retained messages and replicas stored in the cluster.
Queued Messages	The current number of queued messages and replicas stored in the cluster. The queued messages count can include queued messages of clients with clean sessions that have already disconnected).
Cluster Nodes	The current number of nodes in the cluster.

For more information, see Clustering MQTT.

Detailed Metric View

Further information is available for each cluster metric the dashboard displays.
To open a more detailed visualization, click the thumbnail of the metric. Details are provided for each node in the cluster.

Figure 4. Example detail of connections metrics

The name of the currently selected metric is underlined in the top section:

Figure 5. Example detail of inbound publish rate metrics

To view or hide individual nodes, select the name of the node in the legend. By default, all nodes are shown:

Figure 6. Detail with all nodes selected:

Figure 7. Detail with one node selected:

Trace Recordings

This is a HiveMQ Enterprise Edition feature. Find out more about HiveMQ Editions.

Your HiveMQ Enterprise MQTT broker can send and receives millions of MQTT messages over a short period of time. Sorting through all the information on your HiveMQ broker to understand the behavior of a specific MQTT client or topic can be a very time-consuming task. To simplify the process, HiveMQ gives you the ability to write log files for individual clients and topics.

A trace recording is a combination of filters that allows you to track the message activity of specific clients or topics and log information to a file in a human-readable format. Each trace recording creates a .trace file in your HiveMQ log/recordings folder. Trace recordings are created cluster-wide. Each node in the HiveMQ cluster writes an individual trace file and stores the information in its own log folder.

You can apply a filter to log information for any MQTT message that your HiveMQ broker sends or receives. A filter can either be a client identifier or a topic. Many filter combinations are possible.
All filters are regular expressions.

To add a trace recording, navigate to the Trace Recordings view of your HiveMQ Control Center and select .
For more information, see Add new Trace Recording.

All trace recording log files are stored as trace-recording-name.trace in the hivemq/log/recordings folder of the HiveMQ instance.

Figure 8. HiveMQ Control Center trace recordings view

Table 5. Trace recording attributes
Attribute	Description
State	Running , Waiting , Stopped , or Aborted
Name	The name of the trace recording log file
Start	The date when HiveMQ begins to record messages for the selected trace recording
End	The date when HiveMQ stops to record messages for the selected trace recording

See Active Trace Recordings

This is a HiveMQ Enterprise Edition feature. Find out more about HiveMQ Editions.

A trace recording is considered active when the recording is in a Running Running or Waiting Waiting state.

Active trace recordings can be viewed, downloaded, stopped, and deleted.

To view a trace recording, select the recording name in the overview.
For more information, seeTrace Recording Details.
To download a trace recording, select ⇒ .
For more information, seeDownload.
To stop a trace recording, select ⇒ .

A stopped trace recording remains available in your HiveMQ log/recordings folder and is visible on the Finished Trace Recordings overview, but cannot be restarted.

To delete a trace recording, select ⇒ .

A deleted trace recording is no longer visible on the overview. All log files related to the trace race recording are permanently removed.

Figure 9. Active trace recordings overview

Add new Trace Recording

This is a HiveMQ Enterprise Edition feature. Find out more about HiveMQ Editions.

To add a new trace recording, select .

The following information must be specified: Basic Settings, Filters, and MQTT Packets.

When you add a trace recording, you have the option to or schedule the recording.
If the start date is in the future, the trace recording can be scheduled. If the start date is immediate or in the past, the trace recording starts instantly.

Figure 10. Add new trace recording dialog

Trace Recording Details

This is a HiveMQ Enterprise Edition feature. Find out more about HiveMQ Editions.

When you select the name of a trace recording, you are redirected to the Trace Recording Details page of the recording. The details page lists Basic Settings, Filters, and Trace Recording Contents for the selected recording.
All information is read-only.
For more information, see Add New Trace Recording.

To stop an active trace recording, select ⇒ Stop .
To download a trace recording, select ⇒ . For more information, see Download.

A stopped trace recording remains available in your HiveMQ log/recordings folder and in the Finished Trace Recordings overview, but cannot be started again.

Basic Settings

This is a HiveMQ Enterprise Edition feature. Find out more about HiveMQ Editions.

All fields in the Basic Settings section are required. If a field fails to validate, a red warning message displays below the corresponding input field.

Name
The name defines the filename of the .trace log file in your HiveMQ log/recordings folder. The name must be unique and contain at least three characters. Only combinations of numbers, letters, dashes, and underscores are allowed. For example, publishing-clients or subscribe_my_topic_1 .

Start
When the start date is immediate or in the past, HiveMQ starts recording log messages immediately after successful creation of the trace recording.
When the start date is in the future, HiveMQ schedules the trace recording. The default value is now.

The start date must be before the end date and use the following format: yyyy-MM-dd HH:mm:ss.
For example, 2017-10-18 19:42:00.

If the start date is after the end date, you can either change the end date or the start date to make the date fields valid.

You can enter a date and time into the input field manually or select the calendar icon to choose a date and time.

End
The end date determines when HiveMQ stops the trace recording. The default value is now plus one hour.

The end date must be in the future and after the start date and use the following format: yyyy-MM-dd HH:mm:ss.
For example, 2017-10-18 20:42:00.

You can enter a date and time into the input field manually or select the calendar icon to choose a date and time.

Filters

This is a HiveMQ Enterprise Edition feature. Find out more about HiveMQ Editions.

In the Filters section, you can add Client Identifier and Topic filters to the trace recording. At least one filter (ClientID or Topic) must be set. Filters are based on regular expressions.

For more information, see Regular Expressions.

You can combine as many ClientID and Topic filters as your use case requires. Contents are recorded if at least one filter of each category matches.
For example, if you set the ClientID filters client1 and client2 and a Topic filter my/topic, content is recorded for client1 and my/topic and for client2 and my/topic.

Table 6. Available trace recording filters
Filter	Instructions
Client Identifier:	To add a Client Identifier* filter, enter a regular expression in the field and select Enter or Add ClientID Filter.*
Topic:	To add a Topic* filter, type a regular expression in the field and select Enter or Add Topic Filter.*

To remove a filter, select "x" next to the filter you want to delete.

MQTT Packet Selection

This is a HiveMQ Enterprise Edition feature. Find out more about HiveMQ Editions.

You can define the type of MQTT messages that you want to trace.
MQTT messages are messages sent or received by MQTT clients and MQTT brokers.

Client filters can be used to filter any type of MQTT message. Topic filters only apply to MQTT messages that contain the selected topic. Topic filters can be used to filter MQTT PUBLISH, SUBSCRIBE, and UNSUBSCRIBE messages.

You can select from the following MQTT message types. At least one message type must be selected.

Trace recording MQTT message type selection

Figure 11. Example MQTT message type selection for trace recording content

Table 7. Available MQTT message types
Content	Description
`CONNECT`	The first message the client sends to the broker when the client establishes a connection to the broker.
`CONNACK`	The message the broker sends to the client to acknowledge the connection request of the client.
`SUBSCRIBE`	A message the client sends to the broker to subscribe to one or more topics.
`SUBACK`	The message the broker sends to the client to acknowledge the subscription request of the client.
`PUBLISH`	A message the client sends to the broker to publish data on a specific topic.
`PUBACK`	The message the broker sends to the client to acknowledge receipt and successful processing of a PUBLISH message with QoS 1.
`PUBREC`	The message the broker sends to the client to acknowledge receipt of a QoS 2 PUBLISH message. The PUBREC message is the first step in the QoS 2 publish/subscribe message flow.
`PUBREL`	The message that the client sends to the broker to acknowledge receipt of a PUBREC message. The PUBREL message is the second step in the QoS 2 publish/subscribe message flow.
`PUBCOMP`	The message the broker sends to the client to acknowledge receipt of the PUBREL message. The PUBCOMP message is the final step in the QoS 2 publish/subscribe message flow and confirms that the PUBLISH message with QoS 2 was processed successfully.
`UNSUBSCRIBE`	A message the client sends to the broker to remove a subscription to one or more topics.
`UNSUBACK`	The message the broker sends to the client to acknowledge the unsubscribe request of the client.
`PINGREQ`	A keep-alive message the client sends to the broker to indicate that the client is still connected and to prevent the broker from closing the connection due to inactivity.
`PINGRESP`	A keep-alive message that the broker must send to the client to upon receipt of a PINGREQ message to confirm that the broker is still available and connected to the client.
`DISCONNECT`	The message a client sends to the broker to gracefully end an existing connection to the broker.

To learn more about MQTT and MQTT message types, we highly recommend our MQTT Essentials series.

Finish Trace Recording Creation

This is a HiveMQ Enterprise Edition feature. Find out more about HiveMQ Editions.

After your trace recording configuration is complete, you can select "Start/Schedule Trace Recording" to start or schedule your trace recording. HiveMQ automatically validates your settings. If an error is detected, a notification with the invalid settings displays:

If everything is configured correctly, a confirmation with a duration is shown.
To stop the trace recording process, select Cancel. To confirm the trace recording and return to the Trace Recordings overview, select Start/Schedule.

Finished trace recording log files are located in your hivemq/log/recording folder.

Regular Expressions

Regular Expressions or regex for short are search patterns to find matching character sequences.

For example, to find all names that contain the letters ik the pattern is ik.

To find all words that contain the letter a followed by at least one n, the regex would be an+.
The + works as a quantifier to specify that at least one n must follow the a.
This regex finds sequences such as an, ann , and annn, but not a, n, or na.

The quantifier always refers to the regex expression that precedes the quantifier:

a+: + refers to a
[a-c]+: + refers to [a-c]

Table 8. Available regex quantifiers
Quantifier	Description	Example	matches	non-matches
?	one or zero times	`ha?llo`	`hallo` `hllo`	`haallo`
*	zero or more times	`ha*llo`	hllo, `hallo, `haallo`, ….	`hakallo`
+	one or more times	`ha+llo`	`hallo`, `haallo`, `haaallo`	`hllo`
{number}	exactly number-times	`hal{+2+}o`	`hallo`	`halo`, `hao`, halllo`
{min, }	at least min times	`hal{2,}o`	`hallo`, `halllo`	`halo`
{min, max}	at least min and maximal max times	`hal{1,2}`	`halo`, `hallo`	`hao`, `halllo`

You can use brackets [ ] to define a set of characters that matches a single character of the set. For example, h[a, e]llo matches hallo and hello. It is also possible to set ranges of characters within the bracket: [c-f] matches c, d, e, and f.

Further it is possible to use the wildcard-character: . to match any character. For example h.llo matches hallo, hbllo, h llo, and so on.

Combined with the quantification * any string can be matched with .* .

Since many characters are used as metacharacters, you must escape metacharacters with a leading \:

Table 9. Metacharacters
Metacharacter	Escaped
`*`	`\*`
`+`	`\+`
`.`	`\.`
`[` and `]`	`\[` and `\]`
`{` and `}`	`\{` and `\}`
`(` and `)`	`\(` and `\)`
`\`	`\\`
`\|`	`\\|`

Finished Trace Recordings

This is a HiveMQ Enterprise Edition feature. Find out more about HiveMQ Editions.

Trace recordings in a Stopped Stopped or Aborted Aborted state are considered finished.

Finished trace recordings can be viewed and deleted.

To view a trace recording, select the name of the recording. For more information, see Trace Recording Details.
Download a trace recording by clicking ⇒ . For more detail, have a look here: Download
Delete a trace recording by clicking ⇒

A deleted trace recording is not visible anymore. All log files of the recording are deleted from disk.

Download a Trace Recording

This is a HiveMQ Enterprise Edition feature. Find out more about HiveMQ Editions.

Every running, stopped, or aborted trace recording can be downloaded as a ZIP file. The download file contains one folder per node with every requested trace recording file.
To download a trace recording, select ⇒ .

When the trace recording is in a running state, all trace files are copied into a temporary folder.

The trace recording files from each cluster node are requested and temporarily stored.

The trace recording files are saved in ZIP file.

After the ZIP file is created, the temporary files are deleted.

To cancel the download process, select Cancel.
When the download is complete, you can save the ZIP file to your local file system or close the window.

If an error occurs during the download process, HiveMQ provides a warning message with additional information.

Possible warning messages include the following:

Not enough disk space (At least 100 MB must be free on every cluster node)

Unable to locate trace recording files on a node

Unable to read files on a node

Unable to write files to a node

The tracing feature is disabled during rolling upgrade

An internal error