MQTT Device Online/Offline Status with LWT and Retained Messages

Knowing which devices are connected right now is the first thing every operator asks for, and MQTT device online status is surprisingly easy to get wrong. The reliable pattern combines three building blocks: a retained birth message, a retained Last Will and an explicit offline message on shutdown. In this guide you will build that pattern step by step, add a fleet monitor in Python, and learn how to avoid the edge cases that make dashboards lie.

The Building Blocks

Each piece solves one part of the problem:

Leave out any one of them and you get a blind spot. Without the birth message, a device that recovers stays "offline". Without the will, crashes go unnoticed. Without the shutdown message, a planned shutdown leaves the device "online" forever, because a clean DISCONNECT tells the broker to discard the will. If the will mechanism is new to you, read MQTT Last Will and Testament first.

Designing the Status Topic

Give every device exactly one status topic, in a predictable place:

connectmq/demo/<prefix>/<site>/<device-id>/status

A wildcard like connectmq/demo/<prefix>/+/+/status then returns the status of the whole fleet in one subscription. Because all status messages are retained, a monitor that starts up receives every device's last known state immediately. How retention works is covered in MQTT retained messages.

For the payload, a plain online or offline string is enough for many tools. JSON adds context that helps operations:

{"state":"online","ts":1760600000,"fw":"2.4.1","ip":"10.0.4.17","reason":"boot"}

For the will, the device cannot know the timestamp of a future failure, so keep it simple: {"state":"offline","reason":"lwt"}. The monitor records when it received it.

Step 1: The Device

This paho-mqtt 2.x script behaves like a well-designed device. It sets a retained will before connecting, publishes a retained birth message inside on_connect so it repeats after every reconnect, and publishes a retained offline message on graceful shutdown. It connects to the public MQTT test broker at broker.connectmq.com:

import json
import signal
import sys
import time
import paho.mqtt.client as mqtt

BROKER = "broker.connectmq.com"
DEVICE_ID = sys.argv[1] if len(sys.argv) > 1 else "pump-07"
STATUS = f"connectmq/demo/z8b2/plant1/{DEVICE_ID}/status"

client = mqtt.Client(mqtt.CallbackAPIVersion.VERSION2,
                     client_id=f"z8b2-{DEVICE_ID}")
client.will_set(STATUS, json.dumps({"state": "offline", "reason": "lwt"}),
                qos=1, retain=True)
client.reconnect_delay_set(min_delay=1, max_delay=30)

def on_connect(client, userdata, flags, reason_code, properties):
    if reason_code.is_failure:
        print("connect failed:", reason_code)
        return
    birth = {"state": "online", "ts": int(time.time()), "reason": "connect"}
    client.publish(STATUS, json.dumps(birth), qos=1, retain=True)
    print("online")

def shutdown(signum, frame):
    bye = {"state": "offline", "ts": int(time.time()), "reason": "shutdown"}
    client.publish(STATUS, json.dumps(bye), qos=1, retain=True).wait_for_publish(timeout=5)
    client.disconnect()
    client.loop_stop()
    sys.exit(0)

client.on_connect = on_connect
signal.signal(signal.SIGINT, shutdown)
signal.signal(signal.SIGTERM, shutdown)

client.connect(BROKER, 1883, keepalive=20)
client.loop_start()
while True:
    time.sleep(1)  # real telemetry would be published here

Run it as python device.py pump-07, and in another terminal as python device.py fan-02.

Step 2: The Fleet Monitor

The monitor subscribes to every status topic and keeps a table of devices. Because statuses are retained, it knows the full fleet state seconds after starting:

import json
import time
import uuid
import paho.mqtt.client as mqtt

BROKER = "broker.connectmq.com"
FILTER = "connectmq/demo/z8b2/+/+/status"
fleet = {}

def on_connect(client, userdata, flags, reason_code, properties):
    client.subscribe(FILTER, qos=1)

def on_message(client, userdata, msg):
    device = msg.topic.split("/")[-2]
    if not msg.payload:  # empty retained payload = device removed
        fleet.pop(device, None)
        return
    try:
        state = json.loads(msg.payload).get("state", "unknown")
    except ValueError:
        state = msg.payload.decode(errors="replace")
    previous = fleet.get(device, {}).get("state")
    fleet[device] = {"state": state, "since": time.strftime("%H:%M:%S")}
    tag = "retained" if msg.retain else "live"
    if previous != state:
        print(f"[{tag}] {device}: {previous} -> {state}")

client = mqtt.Client(mqtt.CallbackAPIVersion.VERSION2,
                     client_id=f"monitor-{uuid.uuid4().hex[:8]}")
client.on_connect = on_connect
client.on_message = on_message
client.connect(BROKER, 1883, keepalive=30)
client.loop_forever()

Step 3: Test Every Failure Mode

Run the monitor and both devices, then work through this checklist and confirm the monitor reacts correctly each time:

1. Monitor restart. Stop and start the monitor. Both devices appear as online immediately, tagged retained.
2. Graceful shutdown. Press Ctrl+C on fan-02. The monitor shows offline with reason shutdown straight away.
3. Crash. Kill pump-07 with kill -9. The OS closes the socket, the broker publishes the will, and the monitor shows offline.
4. Silent network loss. Disconnect the device's network without killing the process. After about 1.5 times the 20-second keepalive, the will fires.
5. Recovery. Restore the network. paho reconnects, on_connect runs, and the birth message flips the device back to online.
6. Decommissioning. Publish an empty retained message to the status topic. The monitor removes the device.

You can watch the same topics live in a browser too; the in-browser client and topic explorer in the ConnectMQ portal show status changes as they happen when you point devices at your own broker.

Edge Cases That Make Dashboards Lie

Duplicate client IDs. Two devices with the same ID kick each other off. Each takeover can fire a will and a birth, so status flaps constantly. Unique IDs are non-negotiable; see the MQTT client ID guide.

Birth published only once at boot. If you publish online outside the connect callback, a device that reconnects after a network blip stays offline forever. Always publish it on every connect.

Will not retained. A dashboard that opens after the failure never learns about it.

Quick reconnects causing flicker. A device that reconnects within seconds still generates an offline/online pair. In MQTT 5 a will delay interval suppresses the will if the session resumes in time.

Long keepalive. With a 300-second keepalive, a dead device looks online for over seven minutes. Pick a keepalive that matches how fast you need to know.

"Connected" is not "healthy". A device can hold its MQTT connection while its sensor has failed. Publish a separate health or heartbeat field if you need application-level liveness.

Presence vs Heartbeat: Which One Do You Need?

Connection-based presence and application heartbeats answer different questions, and many fleets need both.

A device whose main loop has hung can still answer keepalive pings if the MQTT client runs in its own thread, so connection status stays online. A heartbeat published from the main loop, or a consumer that flags devices whose telemetry has gone stale, catches that case. Keep heartbeats non-retained and modest in frequency; once a minute is plenty for most monitoring.

Using the Status Topics in Other Tools

Because the pattern is just retained MQTT messages, any MQTT-capable tool can consume it without special integration. In Node-RED, an mqtt in node subscribed to the status wildcard feeds a dashboard or a notification flow. Home Assistant's MQTT integration can map a status topic to an availability setting with online and offline payloads. Telegraf's MQTT consumer input can write state changes into a time-series database for uptime reporting. Each of these connects to the broker as an ordinary MQTT client, so the only requirement is agreeing on topic names and payload values. Plain online and offline strings are the most portable choice if you plan to use off-the-shelf tools; JSON gives richer context for your own code.

Broker-Side Status on ConnectMQ

The pattern above is portable to any broker. In addition, ConnectMQ publishes connection state on $SYS/broker/connection/{clientId}/state, which a monitor can subscribe to as an independent source that does not depend on device firmware configuring its will correctly. The portal's devices view organises your devices by ISA-95 levels, and threshold alert rules can notify you about values that cross limits. For production, use TLS on port 8883 with YOUR_USERNAME, YOUR_PASSWORD and YOUR_CLIENT_ID from the portal. A managed MQTT broker keeps the presence infrastructure running so you do not have to; for the fundamentals behind it, see the MQTT tutorial.

Get Started with ConnectMQ

Create a free broker at app.connectmq.com on the Serverless plan, no credit card required. Deploy the device and monitor scripts with your credentials, and you have reliable fleet presence on a cloud MQTT broker in minutes.

Frequently Asked Questions

Why does my device show online after it was shut down?

A clean DISCONNECT discards the Last Will. Publish a retained offline message yourself before disconnecting.

Why does my device stay offline after it reconnects?

The birth message is probably published only once at startup. Publish it in the on-connect callback so every reconnect updates the status.

How fast is an offline device detected?

Immediately if the TCP connection closes. If the device disappears silently, about 1.5 times the keepalive interval.

Should status messages be retained?

Yes. Retaining birth, will and shutdown messages ensures any client that subscribes later gets the current state at once.