ESP32 PlatformIO MQTT Project: Clean Structure with Environment-Based Broker Settings

An ESP32 PlatformIO MQTT project can be much more than one sketch with hard-coded credentials. In this guide you will set up a PlatformIO project with a development environment pointing at broker.connectmq.com and a production environment using TLS and private credentials, with secrets kept out of Git and library versions pinned for reproducible builds.

Why PlatformIO for MQTT Firmware

The Arduino IDE is fine for a first sketch. As soon as a project has more than one file, more than one target or more than one developer, PlatformIO pays off:

• Reproducible builds. Platform and library versions live in platformio.ini, so a colleague or CI server builds exactly what you built.
• Multiple environments. One project can produce firmware for a test broker and a production broker, or for different boards, from the same source.
• Real project structure. Headers in include/, code split across files in src/, reusable modules in lib/ and tests in test/.
• Command-line builds. pio run works the same in VS Code, a terminal and a CI pipeline.

Project Layout

Create a new project for an ESP32 DevKit with the Arduino framework, then shape it like this:

esp32-mqtt/
  platformio.ini
  secrets.ini            (not committed)
  secrets.example.ini    (committed template)
  .gitignore
  include/
    config.h
  src/
    main.cpp
    mqtt_link.cpp
    mqtt_link.h

.gitignore keeps local secrets and build output out of the repository:

.pio/
secrets.ini

The platformio.ini With Dev and Production Environments

[platformio]
default_envs = dev
extra_configs = secrets.ini

[env]
platform = espressif32
board = esp32dev
framework = arduino
monitor_speed = 115200
monitor_filters = esp32_exception_decoder
lib_deps =
    knolleary/PubSubClient @ ^2.8
    bblanchon/ArduinoJson @ ^7.0.0
build_flags =
    -D WIFI_SSID=\"${secrets.wifi_ssid}\"
    -D WIFI_PASS=\"${secrets.wifi_pass}\"
    -D FW_VERSION=\"1.0.0\"

[env:dev]
build_flags =
    ${env.build_flags}
    -D MQTT_HOST=\"broker.connectmq.com\"
    -D MQTT_PORT=1883
    -D MQTT_USE_TLS=0
    -D TOPIC_PREFIX=\"connectmq/demo/6d1f3c\"
    -D CORE_DEBUG_LEVEL=3

[env:prod]
build_flags =
    ${env.build_flags}
    -D MQTT_HOST=\"broker.connectmq.com\"
    -D MQTT_PORT=8883
    -D MQTT_USE_TLS=1
    -D MQTT_CLIENT_ID=\"${secrets.mqtt_client_id}\"
    -D MQTT_USER=\"${secrets.mqtt_user}\"
    -D MQTT_PASS=\"${secrets.mqtt_pass}\"
    -D TOPIC_PREFIX=\"factory1/line1/sensor01\"
    -D CORE_DEBUG_LEVEL=1

The shared [env] section holds everything common. Each named environment extends build_flags with its own broker settings. extra_configs = secrets.ini merges a second file into the configuration, so ${secrets.wifi_ssid} resolves to a value that never enters version control.

The secrets.ini file:

[secrets]
wifi_ssid = YOUR_WIFI
wifi_pass = YOUR_WIFI_PASSWORD
mqtt_client_id = YOUR_CLIENT_ID
mqtt_user = YOUR_USERNAME
mqtt_pass = YOUR_PASSWORD

Commit secrets.example.ini with the same keys and placeholder values so new contributors know what to fill in. The production values come from the ConnectMQ portal, where each broker gets a generated client ID, username and password.

In CI, you can skip the file and read environment variables instead with ${sysenv.MQTT_PASS}. Keep in mind that build flags end up as strings inside the binary, so anyone with the firmware file can extract them. For devices distributed to third parties, prefer per-device credentials provisioned at runtime, for example with the ESP32 WiFiManager MQTT portal.

config.h: One Place for Defaults

The header gives every setting a safe default and fails the build if something essential is missing:

#pragma once

#ifndef MQTT_HOST
#define MQTT_HOST "broker.connectmq.com"
#endif

#ifndef MQTT_PORT
#define MQTT_PORT 1883
#endif

#ifndef MQTT_USE_TLS
#define MQTT_USE_TLS 0
#endif

#ifndef TOPIC_PREFIX
#error "TOPIC_PREFIX must be set in platformio.ini"
#endif

#if MQTT_USE_TLS && !defined(MQTT_PASS)
#error "Production builds need MQTT credentials from secrets.ini"
#endif

#ifndef FW_VERSION
#define FW_VERSION "dev"
#endif

An #error is far better than firmware that compiles, ships and then quietly connects to the wrong place.

The MQTT Module

Keeping the MQTT logic in its own file makes main.cpp short and lets you reuse the module across projects.

src/mqtt_link.h:

#pragma once
#include <Arduino.h>

namespace mqtt_link {
void begin();
void loop();
bool publish(const char* subtopic, const char* payload, bool retain = false);
bool connected();
}

src/mqtt_link.cpp:

#include "mqtt_link.h"
#include "config.h"
#include <WiFi.h>
#include <PubSubClient.h>
#if MQTT_USE_TLS
#include <WiFiClientSecure.h>
#include "isrg_root_x1.h"   // const char ISRG_ROOT_X1[] = R"EOF(...PEM...)EOF";
#endif

namespace {
#if MQTT_USE_TLS
WiFiClientSecure net;
#else
WiFiClient net;
#endif
PubSubClient client(net);
String clientId;
String statusTopic;
unsigned long nextAttempt = 0;
unsigned long backoffMs = 1000;

void onMessage(char* topic, byte* payload, unsigned int length) {
  Serial.printf("[mqtt] %s: %.*s\n", topic, (int)length, (const char*)payload);
}

void tryConnect() {
  bool ok;
#if MQTT_USE_TLS
  ok = client.connect(MQTT_CLIENT_ID, MQTT_USER, MQTT_PASS,
                      statusTopic.c_str(), 1, true, "offline");
#else
  ok = client.connect(clientId.c_str(), statusTopic.c_str(), 1, true, "offline");
#endif
  if (ok) {
    client.publish(statusTopic.c_str(), "online", true);
    client.subscribe((String(TOPIC_PREFIX) + "/cmd").c_str());
    backoffMs = 1000;
    Serial.println("[mqtt] connected");
  } else {
    Serial.printf("[mqtt] failed, state=%d\n", client.state());
    nextAttempt = millis() + backoffMs;
    backoffMs = min(backoffMs * 2, 60000UL);
  }
}
}  // namespace

namespace mqtt_link {

void begin() {
#if MQTT_USE_TLS
  net.setCACert(ISRG_ROOT_X1);
  clientId = MQTT_CLIENT_ID;
#else
  clientId = "esp32-pio-" + String((uint32_t)ESP.getEfuseMac(), HEX);
#endif
  statusTopic = String(TOPIC_PREFIX) + "/status";
  client.setServer(MQTT_HOST, MQTT_PORT);
  client.setBufferSize(512);
  client.setSocketTimeout(5);
  client.setCallback(onMessage);
}

void loop() {
  if (WiFi.status() != WL_CONNECTED) return;
  if (client.connected()) {
    client.loop();
  } else if (millis() >= nextAttempt) {
    tryConnect();
  }
}

bool publish(const char* subtopic, const char* payload, bool retain) {
  String topic = String(TOPIC_PREFIX) + "/" + subtopic;
  return client.publish(topic.c_str(), payload, retain);
}

bool connected() { return client.connected(); }

}  // namespace mqtt_link

Create include/isrg_root_x1.h with the ISRG Root X1 certificate from Let's Encrypt as a PEM string. It is only compiled into production builds.

src/main.cpp:

#include <Arduino.h>
#include <WiFi.h>
#include <ArduinoJson.h>
#include "config.h"
#include "mqtt_link.h"

unsigned long lastPublish = 0;

void setup() {
  Serial.begin(115200);
  Serial.printf("Firmware %s, broker %s:%d\n", FW_VERSION, MQTT_HOST, MQTT_PORT);

  WiFi.mode(WIFI_STA);
  WiFi.setAutoReconnect(true);
  WiFi.begin(WIFI_SSID, WIFI_PASS);

  mqtt_link::begin();
}

void loop() {
  mqtt_link::loop();

  if (mqtt_link::connected() && millis() - lastPublish > 10000) {
    lastPublish = millis();
    JsonDocument doc;
    doc["fw"] = FW_VERSION;
    doc["uptime"] = millis() / 1000;
    doc["rssi"] = WiFi.RSSI();
    char buf[128];
    serializeJson(doc, buf, sizeof(buf));
    mqtt_link::publish("telemetry", buf);
  }
}

Building, Flashing and Monitoring

# development build against the public test area
pio run -e dev -t upload
pio device monitor -e dev

# production build with TLS and portal credentials
pio run -e prod -t upload

# build every environment, as CI would
pio run

While the dev build runs, watch its topics from your computer:

mosquitto_sub -h broker.connectmq.com -p 1883 -i pc-6d1f3c -t 'connectmq/demo/6d1f3c/#' -v

Environment Settings at a Glance

Why This Structure Pays Off

Splitting the code this way is not about tidiness for its own sake. Each piece has a job that makes a real difference once the project leaves your desk.

The environments encode intent. Nobody has to remember to change a port, swap WiFiClient for WiFiClientSecure and paste credentials before a release. Choosing prod does all of that consistently, and the #error checks in config.h stop a half-configured build before it reaches a device. The same idea extends naturally: add a staging environment that points at a second private broker, or an environment per board type with a different board = line.

The MQTT module hides the transport. main.cpp never knows whether it is talking over plain TCP or TLS, or which client ID is in use. When you later replace PubSubClient with an asynchronous client, or move topics to a new hierarchy, the change stays inside one file. The module also owns the reconnect backoff, so every project that reuses it gets non-blocking reconnection for free.

The status topic and Last Will come built in. Because tryConnect() always registers a retained offline will and publishes online after connecting, every device built from this template reports its availability without extra code.

Pinned dependencies make CI meaningful. A pipeline that runs pio run on every commit only proves something if it builds with the same library versions the developers use. With caret ranges like ^2.8, you get bug-fix updates but not breaking major versions.

If you want automated checks beyond compiling, PlatformIO's unit test runner can execute tests on the board itself or on the host with a native environment. Pure logic, such as topic building or payload formatting, is easy to test natively once it lives in its own module.

Common PlatformIO Pitfalls

1. String macros without escaped quotes. -D MQTT_HOST=broker.connectmq.com defines a bare token, not a string, and fails to compile. Use \"...\" as shown.
2. Overwriting instead of extending flags. Writing build_flags = in an environment without ${env.build_flags} drops the shared flags, including Wi-Fi credentials.
3. Unpinned libraries. A bare library name pulls whatever is newest, and a major update such as ArduinoJson 6 to 7 can break your build overnight. Pin with @ ^x.y.z.
4. Missing secrets.ini. References to an undefined section fail in confusing ways. The committed example file prevents that.
5. Stale builds after flag changes. PlatformIO usually rebuilds when flags change, but if behaviour looks wrong, run pio run -t clean.
6. Decoding crashes. Keep monitor_filters = esp32_exception_decoder so backtraces show file and line, which is invaluable for problems like ESP32 task watchdog resets.

Next Steps

With a clean structure in place, adding features is straightforward. Put JSON handling in its own module following the ESP32 ArduinoJson MQTT guide, add remote firmware delivery with ESP32 OTA updates over MQTT, and see the ESP32 IoT tutorial for the complete device-to-dashboard project. To understand why the dev and prod brokers behave differently, read about the free public MQTT broker and its limits.

Ship Your PlatformIO Firmware with ConnectMQ

The dev environment works immediately against the public test area. For the prod environment, create a private broker on ConnectMQ, a fully managed MQTT service with MQTT 3.1.1 and 5.0, TLS on port 8883 and a Let's Encrypt certificate. Sign up for the free Serverless plan at app.connectmq.com, no credit card required, copy the generated credentials into secrets.ini and run pio run -e prod -t upload.

Frequently Asked Questions

How do I keep MQTT passwords out of Git in PlatformIO?

Put them in a secrets.ini file that is listed in .gitignore and merged with extra_configs, or read them from environment variables with ${sysenv.NAME} in CI.

Can one PlatformIO project build for several brokers?

Yes. Define one environment per broker or deployment stage, each extending the shared build_flags with its own host, port and credentials, and select it with pio run -e <name>.

Are build flag credentials safe inside the firmware?

They are compiled into the binary as plain strings, so anyone with the file can extract them. For devices you hand to others, provision credentials at runtime and store them in NVS instead.

Which libraries should I use for MQTT in PlatformIO?

PubSubClient is the most widely used synchronous client, and AsyncMqttClient is an event-driven alternative. Add either to lib_deps with a pinned version.