Home Assistant MQTT Discovery: Auto-Create Entities for Custom Devices

Home Assistant MQTT discovery lets a device describe itself so Home Assistant creates its entities automatically, with no YAML on the Home Assistant side. In this tutorial you will learn the discovery topic format, write config payloads for sensors and a switch, group them into one device, handle availability and Home Assistant restarts, and remove entities cleanly, all with a Python device simulator connected to a ConnectMQ broker.

How MQTT Discovery Works

When the MQTT integration starts, it subscribes to topics under a discovery prefix, which is homeassistant by default. A device publishes one retained JSON message per entity to a config topic:

<discovery_prefix>/<component>/[<node_id>/]<object_id>/config

homeassistant/sensor/hvac-node-1/temperature/config
homeassistant/switch/hvac-node-1/relay/config

• component is the entity platform: sensor, binary_sensor, switch, light, button, number, select, climate and others.
• node_id is optional and just groups topics; it does not create the device.
• object_id identifies the entity within the node.

The config payload tells Home Assistant where to read state, where to send commands, how to parse values and which device the entity belongs to. Because the message is retained, the broker replays it whenever Home Assistant reconnects, so entities reappear after a restart.

Discovery is only metadata. The real data still flows on your own state and command topics, which you design as you would for any Home Assistant MQTT integration.

Prerequisites

• Home Assistant with the MQTT integration configured and discovery enabled (it is on by default).
• A private ConnectMQ broker: create one in the portal and note the generated username, password and client ID. Home Assistant and the simulator each need their own credentials.
• Python 3.10+ with pip install "paho-mqtt>=2.0".

You can experiment on the public MQTT test broker as well, but two things make it awkward for discovery. First, retained config topics under the shared homeassistant prefix are visible to every Home Assistant instance connected there, so set a custom discovery prefix in the integration options (for example cmqx7d2). Second, the public broker allows 10 subscriptions per client, and Home Assistant needs several for discovery plus one per entity. A private broker avoids both issues.

A Discovery Payload, Field by Field

Here is the config for a temperature sensor:

{
  "name": "Supply Air Temperature",
  "unique_id": "hvacnode1_supply_temp",
  "object_id": "hvac_node_1_supply_temp",
  "state_topic": "home/hvac-node-1/state",
  "value_template": "{{ value_json.supply_c }}",
  "unit_of_measurement": "°C",
  "device_class": "temperature",
  "state_class": "measurement",
  "availability_topic": "home/hvac-node-1/availability",
  "device": {
    "identifiers": ["hvacnode1"],
    "name": "HVAC Node 1",
    "manufacturer": "DIY",
    "model": "ESP32 duct sensor",
    "sw_version": "1.3.0"
  }
}

Long payloads can use the documented abbreviations, for example stat_t for state_topic, uniq_id for unique_id and dev for device. They are useful on microcontrollers with small buffers; on a PC, the full names are easier to read.

A Complete Python Device with Discovery

The simulator below behaves like a real device: it announces three entities, publishes JSON state every ten seconds, obeys switch commands, sets a last will for availability, and republishes its config when Home Assistant comes back online.

import json
import random
import time

import paho.mqtt.client as mqtt

BROKER, PORT = "broker.connectmq.com", 8883
USERNAME, PASSWORD = "YOUR_USERNAME", "YOUR_PASSWORD"
CLIENT_ID = "YOUR_CLIENT_ID"

DISCOVERY_PREFIX = "homeassistant"
NODE = "hvac-node-1"
BASE = f"home/{NODE}"
STATE_T = f"{BASE}/state"
AVAIL_T = f"{BASE}/availability"
RELAY_CMD_T = f"{BASE}/relay/set"
RELAY_STATE_T = f"{BASE}/relay/state"

DEVICE = {
    "identifiers": ["hvacnode1"],
    "name": "HVAC Node 1",
    "manufacturer": "DIY",
    "model": "ESP32 duct sensor",
    "sw_version": "1.3.0",
}

ENTITIES = [
    ("sensor", "supply_temp", {
        "name": "Supply Air Temperature",
        "state_topic": STATE_T,
        "value_template": "{{ value_json.supply_c }}",
        "unit_of_measurement": "°C",
        "device_class": "temperature",
        "state_class": "measurement",
    }),
    ("sensor", "filter_dp", {
        "name": "Filter Pressure Drop",
        "state_topic": STATE_T,
        "value_template": "{{ value_json.filter_dp_pa }}",
        "unit_of_measurement": "Pa",
        "device_class": "pressure",
        "state_class": "measurement",
    }),
    ("switch", "relay", {
        "name": "Fan Relay",
        "command_topic": RELAY_CMD_T,
        "state_topic": RELAY_STATE_T,
        "payload_on": "ON",
        "payload_off": "OFF",
    }),
]

relay_on = False


def publish_discovery(client):
    for component, object_id, cfg in ENTITIES:
        payload = {
            **cfg,
            "unique_id": f"hvacnode1_{object_id}",
            "availability_topic": AVAIL_T,
            "device": DEVICE,
        }
        topic = f"{DISCOVERY_PREFIX}/{component}/{NODE}/{object_id}/config"
        client.publish(topic, json.dumps(payload), qos=1, retain=True)
    client.publish(AVAIL_T, "online", qos=1, retain=True)
    client.publish(RELAY_STATE_T, "ON" if relay_on else "OFF", qos=1, retain=True)


def on_connect(client, userdata, flags, reason_code, properties):
    print("connected:", reason_code)
    client.subscribe(RELAY_CMD_T, qos=1)
    client.subscribe("homeassistant/status", qos=1)   # Home Assistant birth message
    publish_discovery(client)


def on_message(client, userdata, msg):
    global relay_on
    if msg.topic == "homeassistant/status" and msg.payload == b"online":
        time.sleep(random.uniform(0.5, 2.0))          # spread load after HA restarts
        publish_discovery(client)
    elif msg.topic == RELAY_CMD_T:
        relay_on = msg.payload == b"ON"
        client.publish(RELAY_STATE_T, "ON" if relay_on else "OFF", qos=1, retain=True)


client = mqtt.Client(mqtt.CallbackAPIVersion.VERSION2, client_id=CLIENT_ID)
client.username_pw_set(USERNAME, PASSWORD)
client.tls_set()
client.will_set(AVAIL_T, "offline", qos=1, retain=True)
client.on_connect = on_connect
client.on_message = on_message
client.connect(BROKER, PORT, keepalive=30)
client.loop_start()

while True:
    state = {"supply_c": round(random.uniform(12.5, 14.5), 1),
             "filter_dp_pa": random.randint(150, 190)}
    client.publish(STATE_T, json.dumps(state), qos=0)
    time.sleep(10)

Run it, then open Settings > Devices & services > MQTT. A device called HVAC Node 1 appears with two sensors and a switch. Stop the script with Ctrl+C and, within the keep-alive window, the broker publishes the last will and the entities become unavailable.

The birth message handling is important. Home Assistant publishes online to homeassistant/status when it starts. Retained configs usually make republishing unnecessary, but a device that republishes on birth also recovers if retained messages were cleared.

Device-Based Discovery

Recent Home Assistant releases also accept a single config message per device on <prefix>/device/<object_id>/config, listing all entities under a components key (abbreviated cmps), each with a platform (p). It also needs an origin (o) block naming the software that publishes it:

{
  "dev": {"ids": ["hvacnode1"], "name": "HVAC Node 1", "mf": "DIY", "mdl": "ESP32 duct sensor"},
  "o": {"name": "hvac-node-firmware", "sw": "1.3.0"},
  "avty_t": "home/hvac-node-1/availability",
  "stat_t": "home/hvac-node-1/state",
  "cmps": {
    "supply_temp": {"p": "sensor", "dev_cla": "temperature", "unit_of_meas": "°C",
                    "val_tpl": "{{ value_json.supply_c }}", "uniq_id": "hvacnode1_supply_temp"},
    "filter_dp":   {"p": "sensor", "dev_cla": "pressure", "unit_of_meas": "Pa",
                    "val_tpl": "{{ value_json.filter_dp_pa }}", "uniq_id": "hvacnode1_filter_dp"}
  }
}

One retained message instead of many is attractive for devices with dozens of entities. Use one style per device; do not announce the same unique IDs through both.

Removing and Changing Entities

To delete an entity, publish an empty retained payload to its config topic:

mosquitto_pub -h broker.connectmq.com -p 8883 --capath /etc/ssl/certs \
  -u YOUR_USERNAME -P YOUR_PASSWORD -i YOUR_CLIENT_ID -r -n \
  -t "homeassistant/sensor/hvac-node-1/filter_dp/config"

-n sends a zero-length message and -r makes it retained, which also clears the stored config from the broker. To change an entity, publish the new config to the same topic with the same unique_id. Changing the unique_id creates a new entity and orphans the old one.

Troubleshooting Checklist

1. Nothing appears. Confirm the discovery prefix in the integration options matches the one your device uses, and that discovery is enabled.
2. Entities appear but are unavailable. The availability topic has no retained online message, or the payload is not exactly online.
3. "Unique ID already exists" in the log. Two configs share a unique_id. Make them globally unique, ideally derived from a MAC address.
4. Duplicate devices after a firmware update. The device.identifiers value changed. Keep it stable.
5. Ghost entities. Old retained configs remain on the broker. Use the ConnectMQ topic explorer to find them under the discovery prefix and clear them with empty retained messages.
6. Config rejected silently. Validate your JSON; a trailing comma in a hand-written payload is the usual culprit, and Home Assistant logs the error under the MQTT integration.

Discovery from an ESP32

On a microcontroller the logic is identical: build each config payload, publish it retained after every successful connection, and subscribe to the birth topic. Three details differ from Python. First, PubSubClient's default packet buffer is small, so call mqtt.setBufferSize(1024) or larger before connecting; otherwise long discovery payloads are silently dropped. Second, derive unique_id and device.identifiers from the chip's MAC address so every board gets stable, unique values without manual configuration. Third, publish availability through the last will in connect() so Home Assistant learns about power loss without the device having to do anything.

Abbreviated keys help on small devices, and building payloads with ArduinoJson avoids quoting mistakes in hand-written strings. Test your firmware's payloads by subscribing to homeassistant/# with any MQTT client and checking that each config is valid JSON before you look for the entities in Home Assistant.

Where Discovery Fits

Firmware such as Tasmota, ESPHome in MQTT mode and Zigbee2MQTT already publish discovery payloads, which is why their devices appear instantly. For your own firmware the Python example above translates directly to C++ with PubSubClient or AsyncMqttClient; just raise the client buffer size, because config payloads easily exceed the default. When devices live outside your home network, put them all on one cloud MQTT broker as described in using an external MQTT broker with Home Assistant. The ESPHome MQTT guide and our IoT automation guide show more devices that rely on discovery.

Build Discoverable Devices on ConnectMQ

Create a free Serverless broker at app.connectmq.com, no credit card required, and issue credentials for Home Assistant and each device. With a fully managed MQTT service from ConnectMQ, retained discovery configs live on a broker you never have to patch, and the live topic explorer shows exactly what each device announces.

Frequently Asked Questions

Do discovery messages have to be retained?

They should be. Retained configs let Home Assistant rebuild entities after a restart without the device republishing; republishing on the birth message is a useful backup.

What is the difference between node_id and the device?

node_id is only a topic level for organization. The device card is created from the device block, and entities sharing the same identifiers are grouped.

How do I remove a discovered entity?

Publish a zero-length retained message to its config topic. Home Assistant deletes the entity and the broker forgets the retained config.

Can I change the discovery prefix?

Yes, in the MQTT integration options. Every device must then publish its configs under the new prefix.