ESPHome MQTT: Using MQTT Instead of the Native API

ESPHome MQTT support lets a YAML-configured ESP32 or ESP8266 publish its sensor states and accept commands through any standard broker instead of the Home Assistant native API. In this tutorial you will configure the mqtt: component, understand the topics ESPHome generates, add TLS and credentials for production, and react to incoming messages directly on the device.

Native API or MQTT: When to Switch

ESPHome's native API is an efficient, encrypted point-to-point link between one device and Home Assistant. It is excellent when Home Assistant is the only thing that ever talks to the device. It becomes limiting as soon as other software needs the same data: a Node-RED flow, a Python logger, a Grafana pipeline through Telegraf, or a second site that cannot reach your LAN.

MQTT solves that by putting a broker in the middle. The device publishes once and any number of subscribers receive the reading. Nothing on the device changes when you add a new consumer, and devices behind NAT can still reach a broker on the internet. If you are new to the publish/subscribe model, the overview of what an MQTT broker does is a good primer.

You can run both at the same time, but if nothing uses the native API any more, remove the api: block so the device does not spend memory on it.

A Minimal ESPHome MQTT Configuration

The example below is a complete configuration for an ESP32 with a DHT22 sensor and a relay. It connects to the public test broker so you can see messages within minutes. Replace a7k2q9 with your own random string so your topics do not collide with anyone else's.

esphome:
  name: greenhouse-node
  friendly_name: Greenhouse Node

esp32:
  board: esp32dev
  framework:
    type: esp-idf

logger:
  level: INFO

wifi:
  ssid: !secret wifi_ssid
  password: !secret wifi_password

ota:
  - platform: esphome
    password: !secret ota_password

mqtt:
  broker: broker.connectmq.com
  port: 1883
  client_id: esphome-greenhouse-a7k2q9
  topic_prefix: connectmq/demo/a7k2q9/greenhouse
  discovery: false
  log_topic: null
  birth_message:
    topic: connectmq/demo/a7k2q9/greenhouse/status
    payload: online
    retain: true
  will_message:
    topic: connectmq/demo/a7k2q9/greenhouse/status
    payload: offline
    retain: true

sensor:
  - platform: dht
    pin: GPIO4
    model: DHT22
    update_interval: 30s
    temperature:
      name: "Air Temperature"
      id: air_temp
    humidity:
      name: "Air Humidity"
      id: air_hum

switch:
  - platform: gpio
    pin: GPIO26
    name: "Vent Fan"
    id: vent_fan

Flash it with esphome run greenhouse-node.yaml. Once the device is online, subscribe from your computer:

mosquitto_sub -h broker.connectmq.com -p 1883 -i sub-a7k2q9 \
  -t 'connectmq/demo/a7k2q9/greenhouse/#' -v

You should see the retained online status and a state message every 30 seconds for each sensor.

Settings worth understanding

• client_id: must be non-empty and unique on the broker. If two devices share an ID, the broker disconnects the older session every time the newer one connects, and both devices flap.
• topic_prefix: defaults to the node name. Setting it explicitly gives you a predictable hierarchy and keeps the public test area tidy.
• discovery: set to false unless Home Assistant consumes this broker. Discovery publishes retained configuration messages under homeassistant/, which on a shared public broker would be visible to everyone.
• log_topic: ESPHome mirrors its logs to <prefix>/debug by default. Setting it to null prevents a chatty device from burning through rate limits.

The Topics ESPHome Publishes

ESPHome builds topics from the prefix, the component type and an object ID derived from the entity name (lower case, spaces become underscores).

States are plain strings: 23.4 for a sensor, ON or OFF for a switch. To turn the fan on from the command line:

mosquitto_pub -h broker.connectmq.com -p 1883 -i pub-a7k2q9 \
  -t 'connectmq/demo/a7k2q9/greenhouse/switch/vent_fan/command' -m 'ON'

The device acts on the command and then publishes the new state on the matching state topic, so dashboards always reflect what actually happened rather than what was requested.

If a different topic layout is required, every component accepts state_topic and command_topic overrides. This is useful when you already have a structure such as the one described in the smart home MQTT topic structure guide.

sensor:
  - platform: dht
    pin: GPIO4
    model: DHT22
    temperature:
      name: "Air Temperature"
      state_topic: connectmq/demo/a7k2q9/site1/greenhouse/temperature

Reacting to MQTT Messages on the Device

The MQTT component is not only an output. You can trigger automations from any topic with on_message or on_json_message, and publish arbitrary payloads with the mqtt.publish and mqtt.publish_json actions.

mqtt:
  # ... connection settings from above ...
  on_message:
    - topic: connectmq/demo/a7k2q9/greenhouse/mode
      payload: "night"
      then:
        - switch.turn_off: vent_fan
  on_json_message:
    topic: connectmq/demo/a7k2q9/greenhouse/setpoint
    then:
      - lambda: |-
          if (x.containsKey("max_temp")) {
            id(max_temp) = x["max_temp"];
          }

globals:
  - id: max_temp
    type: float
    restore_value: true
    initial_value: '28.0'

interval:
  - interval: 60s
    then:
      - if:
          condition:
            lambda: 'return id(air_temp).state > id(max_temp);'
          then:
            - switch.turn_on: vent_fan
            - mqtt.publish_json:
                topic: connectmq/demo/a7k2q9/greenhouse/event
                payload: |-
                  root["event"] = "overheat";
                  root["temp"] = id(air_temp).state;

This keeps a simple safety rule on the device itself, so the fan still runs if the network is down, while the setpoint can be changed remotely by publishing {"max_temp": 30}.

Production Setup: TLS and Credentials

The public test area is fine for experiments, but every topic there is readable by anyone. For a real installation create a private broker and use the generated credentials. On ESP32 with the ESP-IDF framework, ESPHome supports TLS by supplying the CA certificate that signed the broker's certificate. ConnectMQ uses a Let's Encrypt certificate, so the CA is ISRG Root X1.

mqtt:
  broker: broker.connectmq.com
  port: 8883
  client_id: YOUR_CLIENT_ID
  username: YOUR_USERNAME
  password: !secret mqtt_password
  topic_prefix: site1/greenhouse
  discovery: false
  log_topic: null
  certificate_authority: |
    -----BEGIN CERTIFICATE-----
    (paste the ISRG Root X1 certificate here)
    -----END CERTIFICATE-----
  birth_message:
    topic: site1/greenhouse/status
    payload: online
    retain: true
  will_message:
    topic: site1/greenhouse/status
    payload: offline
    retain: true

Keep the password in secrets.yaml rather than the main file so it does not end up in version control. Because the client ID is bound to the issued credentials on ConnectMQ, copy it exactly from the portal.

ESP8266 boards have far less RAM, and full certificate validation is tight on them. If you must use TLS on an ESP8266, check the current ESPHome documentation for the supported options, or move the project to an ESP32, which handles TLS comfortably.

Feeding Home Assistant and Other Tools

With the device on a broker, Home Assistant becomes one consumer among several. Point its MQTT integration at the same broker (the external MQTT broker for Home Assistant guide walks through it) and set discovery: true on the device so entities appear automatically. Discovery details, including the retained config payloads, are explained in Home Assistant MQTT discovery.

At the same time a Node-RED flow can subscribe to site1/+/sensor/+/state to log every sensor, and a Python script can archive readings to a database. None of them needs to know the device's IP address.

Troubleshooting ESPHome MQTT

1. Device connects then disconnects every few seconds. Almost always a duplicate client ID. Check that no other device or a leftover test script uses the same ID.
2. No messages appear. Confirm the subscription matches the prefix exactly, including case. Subscribe to connectmq/demo/a7k2q9/# to see everything under your branch.
3. TLS handshake fails. Verify the device has the correct time (TLS needs it; ESPHome syncs via SNTP), that you pasted the full PEM block, and that you are on port 8883.
4. Messages are dropped on the public broker. The test area allows 10 messages per second per client and 64 KB per message. Logs on the debug topic or very short update intervals can exceed that.
5. Commands are ignored. Publish to the command topic, not the state topic, and use ON or OFF in upper case for switches.

Watching the topic tree in a live explorer makes most of these problems obvious within seconds, which is why it is worth opening the portal's topic explorer or the ConnectMQ Desktop app while you flash.

Scaling Beyond a Few Nodes

A single Raspberry Pi running a broker works until it does not: SD card failures, power cuts and port forwarding for remote access all become your problem. Moving to a managed MQTT broker removes that maintenance and gives every ESPHome node a stable TLS endpoint that works from any network. The broader picture of combining devices, brokers and automation logic is covered in the IoT automation guide.

Connect Your ESPHome Devices to ConnectMQ

ConnectMQ gives you a private broker with TLS on port 8883, generated username, password and client ID, and a live topic explorer to watch your ESPHome nodes report in. Create a free Serverless broker at app.connectmq.com, no credit card required, paste the credentials into secrets.yaml and flash.

Frequently Asked Questions

Can I use ESPHome MQTT and the native API at the same time?

Yes. Both components can be enabled together, which is handy during a migration. Remove api: once nothing depends on it to save memory.

Does ESPHome MQTT work without Home Assistant?

Yes. With discovery: false, the device simply publishes states and listens for commands, and any MQTT client can use them.

Why does my ESPHome device keep reconnecting to the broker?

The most common cause is two clients using the same client ID. Give every device a unique client_id, or use the exact ID issued with your broker credentials.

Which QoS does ESPHome use?

State messages use QoS 0 by default. You can set qos per component or in birth_message and will_message if you need delivery guarantees, as explained in the MQTT QoS levels guide.