MQTT Broker High Availability: Concepts, Client Design and Questions to Ask

MQTT broker high availability is about keeping devices and applications communicating when part of the system fails, and it depends as much on your clients as on the broker. This article explains the key concepts, what really happens to sessions and messages during a failover, how to design clients that recover cleanly, and which questions to ask any provider, ConnectMQ included, before you rely on it.

What High Availability Means for MQTT

In general infrastructure terms, high availability means designing a system so that the failure of one component does not take the whole service down for long. For an MQTT broker, that breaks into three separate questions:

1. Can clients reconnect quickly? If the broker process, machine or network path fails, how long until devices can connect again?
2. What happens to session state? Are subscriptions and queued QoS 1 and 2 messages for offline clients preserved across the failure?
3. What happens to in-flight messages? Are messages that were accepted but not yet delivered lost, delivered once or delivered twice?

These have different answers on different architectures, and a provider that is excellent at the first may make no promise about the third. Be precise about which one you need.

Common High Availability Architectures

None of these is universally best. Many reliable IoT systems use a single well-operated broker per partition plus resilient clients, because the simplest architecture has the fewest ways to fail.

Where a managed plan fits

It is worth being direct. ConnectMQ does not offer message-replication clustering, and sessions and queued messages are not persisted across broker restarts. That means you should design clients to tolerate a broker restart by reconnecting, resubscribing and republishing anything important, which is good practice on any broker anyway.

What it does offer for availability planning is a choice of deployment model. A Dedicated plan gives you an isolated cluster, so other tenants' traffic cannot affect yours. BYOC runs the broker in your own cloud account, managed by ConnectMQ, so it lives inside the infrastructure, networking and region choices you already control. If you need a specific availability target, discuss it with the team for a Dedicated or BYOC deployment rather than assuming a number.

What Happens During a Failover

When a broker restarts or a client is moved to another node, a typical sequence looks like this:

1. The TCP connection drops. The client notices immediately or after its keepalive interval.
2. Other subscribers receive the Last Will message, if the broker processes it before the failure, or a status update later.
3. The client reconnects with backoff.
4. If the broker still has the session, CONNACK reports session present and subscriptions are intact. If not, the client starts with a clean session and must resubscribe.
5. Messages published while the client was offline are delivered only if the broker kept them; otherwise they are gone.

The practical conclusion: never assume step 4 reports session present. Resubscribe on every connect, and keep anything you cannot afford to lose on the client until it is acknowledged.

Designing Clients for High Availability

Resilient clients turn a broker failover from an incident into a blip. The patterns below work on every broker.

Resubscribe on every connect

Put subscriptions in the connect callback, not in startup code, so they are restored after every reconnect regardless of session state.

Buffer locally until acknowledged

For important data, keep a local queue and remove an item only after the broker acknowledges a QoS 1 publish. Our store-and-forward MQTT guide covers persistent buffers on gateways.

Fail over between endpoints

If you operate more than one broker, for example two independent deployments in different locations, the client can rotate between them. Here is a minimal Python example with paho-mqtt 2.x that rotates endpoints, resubscribes on connect and holds unacknowledged messages in memory.

import json
import random
import threading
import time

import paho.mqtt.client as mqtt

ENDPOINTS = [("broker.connectmq.com", 8883), ("YOUR_SECONDARY_HOST", 8883)]
BASE = "acme/site1/line1/gw-01"
pending = {}              # mid -> (topic, payload), removed when acknowledged
lock = threading.Lock()

def send(client, topic, payload):
    with lock:
        info = client.publish(topic, payload, qos=1)
        pending[info.mid] = (topic, payload)

def on_connect(client, userdata, flags, reason_code, properties):
    if reason_code.is_failure:
        print("refused:", reason_code)
        return
    print("connected; session present:", flags.session_present)
    client.subscribe(f"{BASE}/cmd/#", qos=1)           # always resubscribe
    client.publish(f"{BASE}/status", "online", qos=1, retain=True)
    with lock:
        unacked = list(pending.values())
        pending.clear()
    for topic, payload in unacked:                       # resend unacked data
        send(client, topic, payload)

def on_publish(client, userdata, mid, reason_code, properties):
    with lock:
        pending.pop(mid, None)

def on_message(client, userdata, msg):
    print("command:", msg.topic, msg.payload.decode())

client = mqtt.Client(mqtt.CallbackAPIVersion.VERSION2,
                     client_id="YOUR_CLIENT_ID", protocol=mqtt.MQTTv5)
client.username_pw_set("YOUR_USERNAME", "YOUR_PASSWORD")
client.tls_set()
client.will_set(f"{BASE}/status", "offline", qos=1, retain=True)
client.on_connect = on_connect
client.on_publish = on_publish
client.on_message = on_message

index, delay = 0, 1
while True:
    host, port = ENDPOINTS[index % len(ENDPOINTS)]
    try:
        client.connect(host, port, keepalive=30)
    except OSError as exc:
        print(f"{host} unreachable: {exc}")
        index += 1
        time.sleep(delay + random.random())              # backoff with jitter
        delay = min(delay * 2, 60)
        continue
    delay = 1
    client.loop_start()
    time.sleep(2)                                        # wait for CONNACK
    while client.is_connected():
        reading = json.dumps({"ts": int(time.time()), "rpm": 1480})
        send(client, f"{BASE}/telemetry", reading)
        time.sleep(5)
    client.loop_stop()
    print(f"lost {host}, trying the next endpoint")
    index += 1

In production you would persist pending to disk and cap its size, but the structure is the same. Because QoS 1 allows duplicates, receivers should tolerate a message arriving twice after a failover. Note that each endpoint needs credentials valid for that broker.

Make commands idempotent

After a failover, a QoS 1 command may arrive twice. Include a command ID and ignore duplicates on the device.

Republish state, do not rely on the broker to keep it

Retained status and configuration messages are useful, but if the broker restarts without persistence they need to be published again. Have devices republish retained state on every connect, as the example does.

Questions to Ask Any MQTT Provider

Use this list in evaluations and RFPs. Ask for written answers, and treat vague answers as "no".

1. What is the deployment topology for my plan: single node, active-passive or clustered?
2. Are sessions and queued messages persisted across a broker restart?
3. Are messages replicated between nodes before they are acknowledged?
4. How are clients redirected after a failure: same hostname, DNS change or new endpoint?
5. What is the typical recovery time after a node failure, and is it contractual?
6. Is there a published SLA, what does it cover and what are the credits?
7. How are maintenance and upgrades performed, and are clients disconnected?
8. Where is the status page, and how are incidents communicated?
9. Can I run a deployment in a second location or in my own cloud account?
10. What happens to my workload if another tenant misbehaves on a shared plan?

On our own service, the answers to questions 2 and 3 are that persistence across restarts and message replication are not offered, and for SLA terms you should ask the team directly. Our MQTT broker SLA article explains how to read SLA language, and the MQTT broker RFP checklist has a longer evaluation list.

Practical High Availability Patterns on a Managed Broker

Combining the concepts above, a realistic approach for most teams looks like this:

• Choose a hosted MQTT broker so patching, TLS and machine failures are someone else's job.
• Use a Dedicated deployment if you need isolation from other tenants.
• Use BYOC if the broker should live in your own cloud account next to your other systems.
• Partition by site or customer where one outage should not affect everyone.
• Build every client with resubscribe-on-connect, local buffering, idempotent commands and backoff with jitter.
• Monitor connection state, for example via $SYS/broker/connection/{clientId}/state, and alert when devices go offline.

For capacity alongside availability, see how to scale an MQTT broker, and for the wider hosting landscape read the cloud MQTT broker guide.

Plan Your MQTT Availability with ConnectMQ

ConnectMQ is a fully managed MQTT broker with Serverless, Dedicated and BYOC options, so you can pick the deployment model that matches your availability needs. Start free at app.connectmq.com with no credit card, build resilient clients against it, and talk to the team when you need a Dedicated or BYOC deployment.

Frequently Asked Questions

Does MQTT have built-in high availability?

No. The MQTT protocol defines sessions, QoS and Last Will, but high availability depends on how the broker is deployed and how clients handle reconnects.

Will my MQTT session survive a broker restart?

Only if the broker persists sessions. Many deployments, including ours, do not persist sessions across restarts, so clients should resubscribe and republish important state on every connect.

Is clustering required for a reliable MQTT system?

Not always. Well-designed clients, partitioned brokers and isolated deployments give many systems strong reliability without a replicated cluster.

What should I ask a provider about MQTT broker high availability?

Ask about topology, session persistence, message replication, failover behavior, maintenance windows and written SLA terms. Get the answers in writing before you commit.