Using esp-mqtt MQTT 5 Features on ESP32 with ESP-IDF

The esp-mqtt MQTT 5 support in ESP-IDF lets an ESP32 attach metadata to messages, set expiry times, shorten topics with aliases and build request/response flows without inventing your own conventions. In this tutorial you will enable MQTT 5.0 in an ESP-IDF project, connect to broker.connectmq.com, and use each feature in working C code you can flash today.

Why MQTT 5 on a Microcontroller

MQTT 3.1.1 is perfectly usable on an ESP32, and most Arduino libraries stop there. MQTT 5.0 adds features that solve real device problems:

esp-mqtt, the MQTT client bundled with ESP-IDF, implements all of these behind a compile-time option. You need a broker that speaks MQTT 5.0 as well. ConnectMQ supports both 3.1.1 and 5.0 on every listener, so the same endpoint works for old and new firmware. For the protocol background, MQTT 5 features explained covers each property in depth.

Prerequisites

• ESP-IDF 5.x installed, with idf.py working in your terminal.
• Any ESP32-family board (ESP32, S3, C3, C6).
• Mosquitto 2.x clients on your computer; mosquitto_sub can print MQTT 5 properties.
• Familiarity with a basic ESP-IDF MQTT project. If you have not built one, start with the ESP-IDF MQTT example and come back.

Project Setup and Enabling MQTT 5.0

Create a project folder with this layout:

mqtt5_connectmq/
  CMakeLists.txt
  sdkconfig.defaults
  main/
    CMakeLists.txt
    main.c

Top-level CMakeLists.txt, which pulls in the Wi-Fi helper used by the official examples:

cmake_minimum_required(VERSION 3.16)
set(EXTRA_COMPONENT_DIRS $ENV{IDF_PATH}/examples/common_components/protocol_examples_common)
include($ENV{IDF_PATH}/tools/cmake/project.cmake)
project(mqtt5_connectmq)

main/CMakeLists.txt:

idf_component_register(SRCS "main.c"
                       INCLUDE_DIRS "."
                       PRIV_REQUIRES mqtt nvs_flash esp_netif esp_event protocol_examples_common)

sdkconfig.defaults switches on the MQTT 5 code paths in esp-mqtt. Without this option the esp_mqtt5_* functions are not compiled in:

CONFIG_MQTT_PROTOCOL_5=y

Then set your Wi-Fi credentials with idf.py menuconfig under Example Connection Configuration. If your ESP-IDF release no longer ships the mqtt component in-tree, add it with idf.py add-dependency espressif/mqtt.

The esp-mqtt MQTT 5 Example Code

Replace the random part of PREFIX and the client ID with your own values.

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include "esp_log.h"
#include "esp_event.h"
#include "esp_netif.h"
#include "nvs_flash.h"
#include "esp_random.h"
#include "esp_system.h"
#include "freertos/FreeRTOS.h"
#include "freertos/task.h"
#include "protocol_examples_common.h"
#include "mqtt_client.h"

#define PREFIX     "connectmq/demo/idf5m7"
#define CLIENT_ID  "idf5-demo-m7q2"

static const char *TAG = "mqtt5";
static esp_mqtt_client_handle_t client;

/* Properties sent once in CONNECT */
static esp_mqtt5_connection_property_config_t connect_property = {
    .session_expiry_interval = 600,      /* keep session 10 min after disconnect */
    .maximum_packet_size = 4096,
    .receive_maximum = 16,
    .topic_alias_maximum = 4,            /* aliases we accept from the broker */
    .request_resp_info = true,
    .request_problem_info = true,
    .will_delay_interval = 15,           /* wait 15 s before publishing the will */
    .payload_format_indicator = true,
    .message_expiry_interval = 3600,
    .content_type = "text/plain",
};

static esp_mqtt5_user_property_item_t connect_user_props[] = {
    {"model", "esp32-devkit"},
    {"fw", "1.4.2"},
};

static esp_mqtt5_user_property_item_t telemetry_user_props[] = {
    {"schema", "telemetry-v2"},
    {"unit", "celsius"},
};

static void print_user_properties(mqtt5_user_property_handle_t props)
{
    if (!props) return;
    uint8_t count = esp_mqtt5_client_get_user_property_count(props);
    if (!count) return;
    esp_mqtt5_user_property_item_t *items = malloc(count * sizeof(*items));
    if (items && esp_mqtt5_client_get_user_property(props, items, &count) == ESP_OK) {
        for (int i = 0; i < count; i++) {
            ESP_LOGI(TAG, "  user property %s = %s", items[i].key, items[i].value);
            free((char *)items[i].key);
            free((char *)items[i].value);
        }
    }
    free(items);
}

static void publish_telemetry(float temperature, int seq)
{
    char payload[96];
    int len = snprintf(payload, sizeof(payload),
                       "{\"seq\":%d,\"temperature\":%.2f}", seq, temperature);

    esp_mqtt5_publish_property_config_t pub_prop = {
        .payload_format_indicator = 1,    /* UTF-8 payload */
        .message_expiry_interval = 60,    /* useless after one minute */
        .topic_alias = 1,                 /* map this topic to alias 1 */
        .content_type = "application/json",
    };
    esp_mqtt5_client_set_user_property(&pub_prop.user_property, telemetry_user_props,
        sizeof(telemetry_user_props) / sizeof(telemetry_user_props[0]));
    esp_mqtt5_client_set_publish_property(client, &pub_prop);

    int msg_id = esp_mqtt_client_publish(client, PREFIX "/telemetry", payload, len, 1, 0);
    ESP_LOGI(TAG, "published seq %d, msg_id %d", seq, msg_id);

    esp_mqtt5_client_delete_user_property(pub_prop.user_property);
}

static void reply_to_request(esp_mqtt_event_handle_t event)
{
    char response_topic[128];
    int n = event->property->response_topic_len;
    if (n <= 0 || n >= (int)sizeof(response_topic)) return;
    memcpy(response_topic, event->property->response_topic, n);
    response_topic[n] = '\0';

    esp_mqtt5_publish_property_config_t pub_prop = {
        .correlation_data = event->property->correlation_data,
        .correlation_data_len = event->property->correlation_data_len,
        .message_expiry_interval = 30,
    };
    esp_mqtt5_client_set_publish_property(client, &pub_prop);

    char body[96];
    int len = snprintf(body, sizeof(body), "{\"ok\":true,\"freeHeap\":%lu}",
                       (unsigned long)esp_get_free_heap_size());
    esp_mqtt_client_publish(client, response_topic, body, len, 1, 0);
    ESP_LOGI(TAG, "replied on %s", response_topic);
}

static void mqtt5_event_handler(void *args, esp_event_base_t base,
                                int32_t event_id, void *event_data)
{
    esp_mqtt_event_handle_t event = event_data;

    switch ((esp_mqtt_event_id_t)event_id) {
    case MQTT_EVENT_CONNECTED: {
        ESP_LOGI(TAG, "connected, session present: %d", event->session_present);
        esp_mqtt_client_publish(client, PREFIX "/status", "online", 0, 1, 1);

        esp_mqtt5_subscribe_property_config_t sub_prop = {
            .subscribe_id = 7,
            .no_local_flag = true,        /* do not echo our own publishes */
        };
        esp_mqtt5_client_set_subscribe_property(client, &sub_prop);
        esp_mqtt_client_subscribe(client, PREFIX "/request", 1);
        break;
    }
    case MQTT_EVENT_DATA:
        ESP_LOGI(TAG, "data on %.*s: %.*s", event->topic_len, event->topic,
                 event->data_len, event->data);
        print_user_properties(event->property->user_property);
        if (event->property->response_topic_len > 0) {
            reply_to_request(event);
        }
        break;
    case MQTT_EVENT_DISCONNECTED:
        ESP_LOGW(TAG, "disconnected");
        print_user_properties(event->property->user_property);
        break;
    case MQTT_EVENT_ERROR:
        if (event->error_handle->error_type == MQTT_ERROR_TYPE_CONNECTION_REFUSED) {
            ESP_LOGE(TAG, "connection refused, reason code 0x%02x",
                     event->error_handle->connect_return_code);
        }
        break;
    default:
        break;
    }
}

void app_main(void)
{
    ESP_ERROR_CHECK(nvs_flash_init());
    ESP_ERROR_CHECK(esp_netif_init());
    ESP_ERROR_CHECK(esp_event_loop_create_default());
    ESP_ERROR_CHECK(example_connect());

    esp_mqtt_client_config_t cfg = {
        .broker.address.uri = "mqtt://broker.connectmq.com:1883",
        .session.protocol_ver = MQTT_PROTOCOL_V_5,
        .credentials.client_id = CLIENT_ID,
        .session.last_will = {
            .topic = PREFIX "/status",
            .msg = "offline",
            .msg_len = 7,
            .qos = 1,
            .retain = 1,
        },
    };

    client = esp_mqtt_client_init(&cfg);

    esp_mqtt5_client_set_user_property(&connect_property.user_property, connect_user_props,
        sizeof(connect_user_props) / sizeof(connect_user_props[0]));
    esp_mqtt5_client_set_connect_property(client, &connect_property);
    esp_mqtt5_client_delete_user_property(connect_property.user_property);
    connect_property.user_property = NULL;

    esp_mqtt_client_register_event(client, ESP_EVENT_ANY_ID, mqtt5_event_handler, NULL);
    esp_mqtt_client_start(client);

    for (int seq = 0;; seq++) {
        vTaskDelay(pdMS_TO_TICKS(5000));
        float temperature = 20.0f + (esp_random() % 500) / 100.0f;
        publish_telemetry(temperature, seq);
    }
}

Build, flash and monitor:

idf.py set-target esp32
idf.py build flash monitor

Seeing the Properties Arrive

Mosquitto 2.x prints MQTT 5 properties when you subscribe with protocol version 5 and verbose output:

mosquitto_sub -h broker.connectmq.com -p 1883 -V mqttv5 -i viewer-idf5m7 \
  -t "connectmq/demo/idf5m7/#" -F "%t %P %p"

The %P format token prints user properties, so each telemetry line shows schema:telemetry-v2 unit:celsius next to the JSON. Now test request/response. In one terminal listen for the reply, and in another send a request with a response topic and correlation data:

mosquitto_sub -h broker.connectmq.com -p 1883 -V mqttv5 -i reply-idf5m7 \
  -t "connectmq/demo/idf5m7/reply/laptop" -v

mosquitto_pub -h broker.connectmq.com -p 1883 -V mqttv5 -i req-idf5m7 \
  -t "connectmq/demo/idf5m7/request" -m '{"cmd":"health"}' \
  -D publish response-topic "connectmq/demo/idf5m7/reply/laptop" \
  -D publish correlation-data "req-001"

The ESP32 logs the request, copies the correlation data, and publishes the reply to the topic the requester chose. That is the same pattern a backend service uses to call thousands of devices without hard-coding reply topics. The public area is shared and readable by everyone, so use it for experiments only; the free public MQTT broker page explains its limits, such as session expiry capped at 30 minutes.

How Each Feature Behaves

User properties

User properties are key/value string pairs in the packet header. They are ideal for metadata that consumers can filter on without parsing the payload. Keep them short: they are sent with every message and count toward the packet size.

Message expiry interval

A telemetry message with a 60-second expiry that sits queued for an offline subscriber is discarded by the broker once it is older than a minute. When it is delivered, the remaining lifetime is passed along. Use short expiries for commands such as "open valve" that must never execute late.

Topic alias

The client sends the topic once with alias 1, then can refer to it by number. esp-mqtt keeps its own alias table for this. The broker's CONNACK announces how many aliases it accepts; never exceed that number. The saving matters most for long ISA-95 style topics on slow or metered links.

Session expiry and the will delay

With session_expiry_interval = 600, the broker keeps your subscriptions and queued QoS 1 messages for ten minutes after a disconnect, and session_present in the CONNECTED event tells you whether it did. The will delay of 15 seconds means a quick Wi-Fi blip followed by a reconnect does not flash an offline status to every dashboard. The ESP32 MQTT last will guide compares this with the MQTT 3.1.1 behaviour.

Response topic and correlation data

The requester decides where the answer goes and attaches an opaque correlation value, and the device simply echoes both. This keeps the firmware generic: it never needs to know which service or user asked. Give each requester its own reply topic, and keep the correlation data short, such as a counter or a UUID, because the device copies it into the reply.

Reason codes

MQTT 5 replaces the handful of 3.1.1 return codes with a detailed list that also covers publish, subscribe and disconnect. Log them. A device that reports "0x87 not authorized" or "0x97 quota exceeded" in its logs saves you hours compared with a silent reconnect loop, and it lets you tell configuration mistakes apart from network trouble.

Troubleshooting Checklist

1. Link errors for esp_mqtt5_* functions. CONFIG_MQTT_PROTOCOL_5 is not set. Delete sdkconfig, keep sdkconfig.defaults, and rebuild.
2. Connection refused with reason code 0x85. Client identifier not valid. Use a non-empty ID, and on a production broker use the exact client ID issued with your credentials.
3. Reason code 0x97 after many publishes. Quota exceeded. Slow down: the public area allows 10 messages per second per client.
4. Topic alias errors and disconnects. You exceeded the broker's Topic Alias Maximum. Use fewer aliases or set topic_alias to 0.
5. Garbage when printing topics. event->topic and event->data are not null-terminated; always print with %.*s and the length fields.

Moving to TLS with Your Own Credentials

For production, change the URI to mqtts://broker.connectmq.com:8883, add .broker.verification.crt_bundle_attach = esp_crt_bundle_attach (include esp_crt_bundle.h), and set .credentials.username = "YOUR_USERNAME", .credentials.authentication.password = "YOUR_PASSWORD" and .credentials.client_id = "YOUR_CLIENT_ID". The certificate bundle already contains the Let's Encrypt root, so no PEM file is needed. The full option list is in the ESP-IDF MQTT documentation.

Run MQTT 5 Devices on ConnectMQ

Properties only help if the broker honours them end to end. ConnectMQ is a fully managed MQTT service with MQTT 5.0 session expiry, message expiry, topic aliases, response topics and clear reason codes, plus shared subscriptions on the paid Serverless plan for scaling backend consumers. Create a free Serverless broker at app.connectmq.com with no credit card and point this project at it. For more device recipes, the ESP32 IoT tutorial collects the full series.

Frequently Asked Questions

Does esp-mqtt support MQTT 5 on every ESP32 chip?

Yes. MQTT 5 support is part of the esp-mqtt component, not the chip, so it works on the ESP32, S2, S3, C3, C6 and other targets supported by your ESP-IDF version once CONFIG_MQTT_PROTOCOL_5 is enabled.

Can I mix MQTT 5 and MQTT 3.1.1 clients on the same broker?

Yes. A broker that supports both versions delivers messages between them. MQTT 3.1.1 subscribers simply do not see the MQTT 5 properties.

Do I need to set properties before every publish?

Publish properties apply to the next publish call, so set them each time you want them. Connect properties are set once before starting the client.

Is there an Arduino library with MQTT 5 support?

Most popular Arduino MQTT libraries implement MQTT 3.1.1 only. If you need MQTT 5 on an ESP32, ESP-IDF with esp-mqtt is the most complete option, and you can call it from Arduino-as-a-component projects too.