ESP-IDF MQTT Example: esp-mqtt from Scratch

Q: Why is my received topic printed with garbage at the end?

The topic and data pointers in MQTT_EVENT_DATA are not null-terminated. Print them with %.*s and the length fields, or copy them into a terminated buffer.

This ESP-IDF MQTT example builds a working esp-mqtt client from an empty folder: a Wi-Fi station, an event-driven MQTT handler, publish and subscribe, MQTT 5 connection properties and finally TLS. By the end you will have a firmware that talks to broker.connectmq.com and a clear picture of how the esp-mqtt component is structured.

Why Use ESP-IDF Instead of Arduino

Arduino libraries like PubSubClient are quick to start with, but they are MQTT 3.1.1 only and mostly blocking. ESP-IDF's esp-mqtt component is maintained by Espressif, runs in its own FreeRTOS task, supports MQTT 3.1, 3.1.1 and 5.0, and handles TCP, TLS, WebSocket and secure WebSocket transports through one URI. Reconnection, keepalive and outbox management are built in, and your code reacts to events instead of polling.

The trade-off is more setup: CMake, menuconfig and explicit Wi-Fi initialisation. This guide keeps that setup as small as possible. It targets ESP-IDF 5.x, where the configuration struct uses nested fields like .broker.address.uri. Code written for ESP-IDF 4.x used flat fields and will not compile unchanged. The authoritative API reference is the ESP-IDF MQTT documentation.

Project Layout

Create this structure:

connectmq-idf/
  CMakeLists.txt
  sdkconfig.defaults
  main/
    CMakeLists.txt
    main.c

The top-level CMakeLists.txt:

cmake_minimum_required(VERSION 3.16)
include($ENV{IDF_PATH}/tools/cmake/project.cmake)
project(connectmq_idf)

main/CMakeLists.txt:

idf_component_register(SRCS "main.c" INCLUDE_DIRS ".")

The main component automatically depends on all framework components, so you do not need to list mqtt, esp_wifi or nvs_flash explicitly. In ESP-IDF 5.x, esp-mqtt ships with the framework. If your ESP-IDF release has moved it to the component registry, add it with idf.py add-dependency "espressif/mqtt".

sdkconfig.defaults enables MQTT 5 support, which is off by default:

CONFIG_MQTT_PROTOCOL_5=y

The Complete ESP-IDF MQTT Example

Here is the full main/main.c. It connects to Wi-Fi, starts the MQTT client against the public area of broker.connectmq.com, subscribes to a command topic and publishes a counter every five seconds.

#include <stdio.h>
#include <string.h>
#include "freertos/FreeRTOS.h"
#include "freertos/task.h"
#include "freertos/event_groups.h"
#include "esp_log.h"
#include "esp_wifi.h"
#include "esp_event.h"
#include "esp_netif.h"
#include "esp_mac.h"
#include "nvs_flash.h"
#include "mqtt_client.h"

#define WIFI_SSID      "YOUR_WIFI"
#define WIFI_PASS      "YOUR_WIFI_PASSWORD"
#define BROKER_URI     "mqtt://broker.connectmq.com:1883"
#define TOPIC_PREFIX   "connectmq/demo/c19d44"

static const char *TAG = "connectmq";
static EventGroupHandle_t wifi_events;
#define WIFI_CONNECTED_BIT BIT0

static esp_mqtt_client_handle_t client;
static bool mqtt_connected = false;

static void wifi_handler(void *arg, esp_event_base_t base, int32_t id, void *data)
{
    if (base == WIFI_EVENT && id == WIFI_EVENT_STA_START) {
        esp_wifi_connect();
    } else if (base == WIFI_EVENT && id == WIFI_EVENT_STA_DISCONNECTED) {
        xEventGroupClearBits(wifi_events, WIFI_CONNECTED_BIT);
        ESP_LOGW(TAG, "Wi-Fi lost, reconnecting");
        esp_wifi_connect();
    } else if (base == IP_EVENT && id == IP_EVENT_STA_GOT_IP) {
        ip_event_got_ip_t *ev = (ip_event_got_ip_t *)data;
        ESP_LOGI(TAG, "Got IP " IPSTR, IP2STR(&ev->ip_info.ip));
        xEventGroupSetBits(wifi_events, WIFI_CONNECTED_BIT);
    }
}

static void wifi_init(void)
{
    wifi_events = xEventGroupCreate();
    ESP_ERROR_CHECK(esp_netif_init());
    ESP_ERROR_CHECK(esp_event_loop_create_default());
    esp_netif_create_default_wifi_sta();

    wifi_init_config_t cfg = WIFI_INIT_CONFIG_DEFAULT();
    ESP_ERROR_CHECK(esp_wifi_init(&cfg));
    ESP_ERROR_CHECK(esp_event_handler_register(WIFI_EVENT, ESP_EVENT_ANY_ID, wifi_handler, NULL));
    ESP_ERROR_CHECK(esp_event_handler_register(IP_EVENT, IP_EVENT_STA_GOT_IP, wifi_handler, NULL));

    wifi_config_t wc = { 0 };
    strncpy((char *)wc.sta.ssid, WIFI_SSID, sizeof(wc.sta.ssid));
    strncpy((char *)wc.sta.password, WIFI_PASS, sizeof(wc.sta.password));
    ESP_ERROR_CHECK(esp_wifi_set_mode(WIFI_MODE_STA));
    ESP_ERROR_CHECK(esp_wifi_set_config(WIFI_IF_STA, &wc));
    ESP_ERROR_CHECK(esp_wifi_start());

    xEventGroupWaitBits(wifi_events, WIFI_CONNECTED_BIT, pdFALSE, pdTRUE, portMAX_DELAY);
}

static void mqtt_handler(void *arg, esp_event_base_t base, int32_t id, void *data)
{
    esp_mqtt_event_handle_t event = (esp_mqtt_event_handle_t)data;

    switch ((esp_mqtt_event_id_t)id) {
    case MQTT_EVENT_CONNECTED:
        ESP_LOGI(TAG, "MQTT connected");
        mqtt_connected = true;
        esp_mqtt_client_subscribe(client, TOPIC_PREFIX "/cmd", 1);
        esp_mqtt_client_publish(client, TOPIC_PREFIX "/status", "online", 0, 1, 1);
        break;
    case MQTT_EVENT_DISCONNECTED:
        ESP_LOGW(TAG, "MQTT disconnected");
        mqtt_connected = false;
        break;
    case MQTT_EVENT_SUBSCRIBED:
        ESP_LOGI(TAG, "Subscribed, msg_id=%d", event->msg_id);
        break;
    case MQTT_EVENT_DATA:
        ESP_LOGI(TAG, "Topic %.*s: %.*s",
                 event->topic_len, event->topic, event->data_len, event->data);
        break;
    case MQTT_EVENT_ERROR:
        if (event->error_handle->error_type == MQTT_ERROR_TYPE_TCP_TRANSPORT) {
            ESP_LOGE(TAG, "Transport error, errno=%d",
                     event->error_handle->esp_transport_sock_errno);
        } else if (event->error_handle->error_type == MQTT_ERROR_TYPE_CONNECTION_REFUSED) {
            ESP_LOGE(TAG, "Connection refused, code=0x%x",
                     event->error_handle->connect_return_code);
        }
        break;
    default:
        break;
    }
}

static void mqtt_start(void)
{
    static char client_id[32];
    uint8_t mac[6];
    esp_read_mac(mac, ESP_MAC_WIFI_STA);
    snprintf(client_id, sizeof(client_id), "idf-%02x%02x%02x", mac[3], mac[4], mac[5]);

    esp_mqtt_client_config_t cfg = {
        .broker.address.uri = BROKER_URI,
        .credentials.client_id = client_id,
        .session.protocol_ver = MQTT_PROTOCOL_V_5,
        .session.keepalive = 30,
        .session.last_will = {
            .topic = TOPIC_PREFIX "/status",
            .msg = "offline",
            .qos = 1,
            .retain = 1,
        },
    };

    client = esp_mqtt_client_init(&cfg);

    esp_mqtt5_connection_property_config_t conn_props = {
        .session_expiry_interval = 600,   /* seconds */
    };
    esp_mqtt5_client_set_connect_property(client, &conn_props);

    esp_mqtt_client_register_event(client, ESP_EVENT_ANY_ID, mqtt_handler, NULL);
    esp_mqtt_client_start(client);
}

void app_main(void)
{
    esp_err_t err = nvs_flash_init();
    if (err == ESP_ERR_NVS_NO_FREE_PAGES || err == ESP_ERR_NVS_NEW_VERSION_FOUND) {
        ESP_ERROR_CHECK(nvs_flash_erase());
        err = nvs_flash_init();
    }
    ESP_ERROR_CHECK(err);

    wifi_init();
    mqtt_start();

    int counter = 0;
    char payload[64];
    while (true) {
        if (mqtt_connected) {
            int len = snprintf(payload, sizeof(payload), "{\"count\":%d}", counter++);
            esp_mqtt_client_publish(client, TOPIC_PREFIX "/telemetry", payload, len, 0, 0);
        }
        vTaskDelay(pdMS_TO_TICKS(5000));
    }
}

Build, flash and open the monitor:

idf.py set-target esp32
idf.py build flash monitor

Then watch the traffic from your computer:

mosquitto_sub -h broker.connectmq.com -p 1883 -i pc-c19d44 -t 'connectmq/demo/c19d44/#' -v
mosquitto_pub -h broker.connectmq.com -p 1883 -i pc-c19d44-pub -t connectmq/demo/c19d44/cmd -m hello

The ESP32 log shows the hello command arriving through MQTT_EVENT_DATA.

How the esp-mqtt Event Model Works

The client runs in its own task, created by esp_mqtt_client_start(). Everything it wants to tell you arrives as an event on the handler you registered:

Event	When it fires	Typical action
`MQTT_EVENT_CONNECTED`	CONNACK received	Subscribe, publish birth message
`MQTT_EVENT_DISCONNECTED`	Connection lost	Update state; the client reconnects automatically
`MQTT_EVENT_SUBSCRIBED`	SUBACK received	Confirm subscriptions
`MQTT_EVENT_PUBLISHED`	PUBACK/PUBCOMP for QoS 1/2	Track delivery
`MQTT_EVENT_DATA`	Message received	Copy and dispatch the payload
`MQTT_EVENT_ERROR`	Transport or protocol error	Log error type and codes

Three details catch people out:

Topic and data are not null-terminated. Always use topic_len and data_len, as the %.*s format above does.
Large messages arrive in chunks. If a payload is bigger than the client's buffer (.buffer.size, 1024 bytes by default), you get several MQTT_EVENT_DATA events. Check event->current_data_offset and event->total_data_len and reassemble.
Subscribe in MQTT_EVENT_CONNECTED. With a clean session, subscriptions are gone after a reconnect. Doing them in the connected handler re-establishes them every time.

The handler runs in the MQTT task, so keep it short. Heavy work belongs in a separate task fed by a queue, as described in the ESP32 FreeRTOS MQTT guide.

Publish vs Enqueue

esp_mqtt_client_publish() is blocking: it sends the packet from the calling task and, for QoS 1 and 2, keeps it in the outbox until acknowledged. Calling it from inside the event handler is fine. If you publish from a time-critical task, esp_mqtt_client_enqueue(client, topic, data, len, qos, retain, true) places the message in the outbox and returns immediately; the MQTT task sends it later. Both return a message ID, or a negative value on failure, so check it.

Using MQTT 5 Features

Setting .session.protocol_ver = MQTT_PROTOCOL_V_5 and enabling CONFIG_MQTT_PROTOCOL_5 switches the client to MQTT 5. The example already sets a session expiry interval through esp_mqtt5_client_set_connect_property(). On the public area of broker.connectmq.com, session expiry can be up to 30 minutes; the 10 minutes used here fits comfortably.

Publish properties are set before the publish call:

esp_mqtt5_publish_property_config_t pub_props = {
    .message_expiry_interval = 60,
    .content_type = "application/json",
};
esp_mqtt5_client_set_publish_property(client, &pub_props);
esp_mqtt_client_publish(client, TOPIC_PREFIX "/telemetry", payload, len, 1, 0);

Message expiry stops stale readings from being delivered to a subscriber that comes back much later. For a deeper tour of response topics, user properties and reason codes in this component, see esp-mqtt and MQTT 5, and for the protocol side, MQTT 5 features.

Choosing QoS, Retain and Keepalive

The example uses three different publish settings on purpose. Routine telemetry goes out at QoS 0 without retain: if one reading is lost, the next one arrives five seconds later, and nothing waits for an acknowledgement. The status message uses QoS 1 with retain, so it is confirmed by the broker and any dashboard that subscribes later sees whether the device is online. The Last Will mirrors that status topic with offline, which the broker publishes if the device vanishes without a clean disconnect.

The keepalive of 30 seconds is a sensible middle ground. esp-mqtt sends PINGREQ packets on its own, so unlike Arduino libraries you do not need to call a loop function to keep the session alive. A shorter keepalive detects dead devices faster but costs more radio time; a longer one is gentler on battery nodes. If you are new to the delivery guarantees, MQTT QoS levels explained walks through what each level costs on the wire.

Switching to TLS on Port 8883

For anything beyond a demo, encrypt the connection. ESP-IDF ships a certificate bundle with common root CAs, including the Let's Encrypt root that ConnectMQ uses, so you do not need to paste a PEM file. Change the URI scheme and attach the bundle:

#include "esp_crt_bundle.h"

esp_mqtt_client_config_t cfg = {
    .broker.address.uri = "mqtts://broker.connectmq.com:8883",
    .broker.verification.crt_bundle_attach = esp_crt_bundle_attach,
    .credentials.client_id = "YOUR_CLIENT_ID",
    .credentials.username = "YOUR_USERNAME",
    .credentials.authentication.password = "YOUR_PASSWORD",
    .session.protocol_ver = MQTT_PROTOCOL_V_5,
};

Make sure CONFIG_MBEDTLS_CERTIFICATE_BUNDLE is enabled (it is by default). Use the client ID, username and password generated for your broker in the portal; the client ID is bound to those credentials, so a made-up ID is rejected. The same URI approach works for WebSocket transports: ws://broker.connectmq.com:8083/mqtt and wss://broker.connectmq.com:8084/mqtt, if CONFIG_MQTT_TRANSPORT_WEBSOCKET is enabled.

Troubleshooting Checklist

Compile errors about .uri or .host: you are mixing ESP-IDF 4.x and 5.x config syntax. Use the nested fields shown here.
MQTT 5 functions undefined: CONFIG_MQTT_PROTOCOL_5 is not set. Delete sdkconfig after adding sdkconfig.defaults, or set it in idf.py menuconfig.
MQTT_ERROR_TYPE_TCP_TRANSPORT right after start: Wi-Fi is not up yet, DNS failed, or the scheme and port do not match (mqtts:// needs 8883).
Connection refused with credentials: check the exact client ID from the portal and that the password field is .credentials.authentication.password.
Payloads cut off: increase .buffer.size or reassemble chunked MQTT_EVENT_DATA events.
Stack overflow in the MQTT task: raise .task.stack_size, particularly with TLS and verbose logging.

If you are working through connection failures in general, the guide on what to do when an ESP32 cannot connect to an MQTT broker covers network-level checks that apply to ESP-IDF too.

Take Your esp-mqtt Firmware to Production with ConnectMQ

You have just run this ESP-IDF MQTT example against the public MQTT test broker. When you are ready for private credentials, ConnectMQ gives you a hosted MQTT broker with MQTT 5, TLS on 8883 and a Let's Encrypt certificate that the ESP-IDF bundle already trusts. Create one on the free Serverless plan at app.connectmq.com, no credit card needed, and change only the URI and credentials in your config. For the bigger picture of device projects, see the ESP32 IoT tutorial.

Frequently Asked Questions

Does esp-mqtt reconnect automatically?

Yes. After a disconnect the client retries on its own using .network.reconnect_timeout_ms. Re-subscribe in MQTT_EVENT_CONNECTED so subscriptions come back with the connection.

Which ESP-IDF version does this example need?

It targets ESP-IDF 5.x, where the client configuration uses nested fields such as .broker.address.uri. ESP-IDF 4.x used a flat struct and needs small changes.

Can I use MQTT 5 with the free ConnectMQ plan?

Yes. Every broker on the platform supports MQTT 3.1.1 and 5.0. The free Serverless plan supports QoS up to 1, so use QoS 0 or 1 in your publishes there.

Why is my received topic printed with garbage at the end?

The topic and data pointers in MQTT_EVENT_DATA are not null-terminated. Print them with %.*s and the length fields, or copy them into a terminated buffer.

ESP-IDF MQTT Example: Build an esp-mqtt Client from Scratch