ESP32 BLE MQTT Gateway: Scan Bluetooth Sensors and Forward Advertisements

An ESP32 BLE MQTT gateway bridges the many cheap Bluetooth Low Energy sensors around you (thermometers, door sensors, beacons, plant monitors) to an MQTT broker over Wi-Fi. In this tutorial you will build a passive BLE scanner on an ESP32, collect advertisements safely from the Bluetooth task, deduplicate them, and publish each device's latest data to its own MQTT topic.

How a BLE-to-MQTT Gateway Works

Most battery-powered BLE sensors never accept a connection. They simply broadcast advertisements: short packets of up to 31 bytes containing an address, an optional name and data fields such as manufacturer-specific data or service data. A sensor that reports temperature typically puts the reading into one of those fields every few seconds.

A gateway listens for those packets and forwards them. That is the whole trick, and it has two big advantages:

• Sensors stay in their low-power broadcast mode, so batteries last a long time.
• One ESP32 can hear many devices at once without managing connections.

This tutorial uses passive scanning, which covers the majority of sensor use cases.

What You Need

• An ESP32 board with Bluetooth (the classic ESP32, ESP32-S3 or ESP32-C3 all support BLE; the ESP32-S2 does not).
• Arduino IDE 2.x with the esp32 board package, which includes the BLE library used here.
• The PubSubClient library.
• One or more BLE sensors or a phone app that can broadcast a test advertisement.

Wi-Fi, Bluetooth and MQTT together make a large firmware. If compilation fails with "Sketch too big", choose a larger app partition in Tools > Partition Scheme, for example "Huge APP".

Wi-Fi and BLE Share One Radio

The ESP32 has a single 2.4 GHz radio that is time-shared between Wi-Fi and Bluetooth. The coexistence scheduler handles this automatically, but it has consequences:

• A scan window of 100 percent starves Wi-Fi. Leave gaps by setting the scan window a little shorter than the interval.
• While a blocking scan runs, your MQTT loop does not. Keep each scan well under the MQTT keep alive interval.
• Expect to miss some advertisements. Sensors repeat their broadcasts, so that is normally harmless.

Espressif documents the coexistence behaviour in the ESP-IDF RF coexistence guide.

Topic Design

Colons are fine in MQTT topics, but the code strips them from the MAC to keep topics tidy and easy to type in a terminal. Replace ble-a81f with your own random identifier.

The ESP32 BLE MQTT Gateway Sketch

The BLE library calls the scan callback from its own task. Publishing from inside that callback would mean touching the network client from two tasks at once, which leads to random crashes. Instead, the callback copies each advertisement into a FreeRTOS queue, and loop() drains the queue and publishes.

#include <WiFi.h>
#include <PubSubClient.h>
#include <BLEDevice.h>
#include <BLEScan.h>
#include <BLEAdvertisedDevice.h>

const char* WIFI_SSID     = "YOUR_WIFI";
const char* WIFI_PASSWORD = "YOUR_WIFI_PASSWORD";
const char* MQTT_HOST     = "broker.connectmq.com";
const uint16_t MQTT_PORT  = 1883;
const char* CLIENT_ID     = "ble-a81f-gateway";
const char* PREFIX        = "connectmq/demo/ble-a81f";

const int   SCAN_SECONDS     = 5;
const int   MIN_RSSI         = -90;    // ignore very weak devices
const uint32_t MIN_REPUBLISH_MS = 10000; // per-device rate limit

struct Adv {
  char addr[18];
  char name[24];
  int  rssi;
  uint8_t mfg[31];
  uint8_t mfgLen;
};

QueueHandle_t advQueue;
WiFiClient net;
PubSubClient mqtt(net);
BLEScan* scanner;
uint32_t seen = 0, published = 0, dropped = 0;

// Simple per-device memory for rate limiting
struct Recent { char addr[18]; uint32_t lastMs; };
Recent recent[32];

class ScanCallbacks : public BLEAdvertisedDeviceCallbacks {
  void onResult(BLEAdvertisedDevice dev) override {
    if (dev.getRSSI() < MIN_RSSI) return;
    Adv a = {};
    auto addr = dev.getAddress().toString();
    strncpy(a.addr, addr.c_str(), sizeof(a.addr) - 1);
    if (dev.haveName()) {
      auto n = dev.getName();
      strncpy(a.name, n.c_str(), sizeof(a.name) - 1);
    }
    a.rssi = dev.getRSSI();
    if (dev.haveManufacturerData()) {
      auto md = dev.getManufacturerData();
      a.mfgLen = min((size_t)md.length(), sizeof(a.mfg));
      memcpy(a.mfg, md.c_str(), a.mfgLen);
    }
    seen++;
    if (xQueueSend(advQueue, &a, 0) != pdTRUE) dropped++;
  }
};

bool shouldPublish(const char* addr) {
  uint32_t now = millis();
  int freeSlot = -1, oldest = 0;
  for (int i = 0; i < 32; i++) {
    if (strcmp(recent[i].addr, addr) == 0) {
      if (now - recent[i].lastMs < MIN_REPUBLISH_MS) return false;
      recent[i].lastMs = now;
      return true;
    }
    if (recent[i].addr[0] == 0 && freeSlot < 0) freeSlot = i;
    if (recent[i].lastMs < recent[oldest].lastMs) oldest = i;
  }
  int slot = freeSlot >= 0 ? freeSlot : oldest;
  strncpy(recent[slot].addr, addr, sizeof(recent[slot].addr) - 1);
  recent[slot].lastMs = now;
  return true;
}

void publishAdv(const Adv& a) {
  char mac[13]; int j = 0;
  for (int i = 0; a.addr[i] && j < 12; i++) if (a.addr[i] != ':') mac[j++] = a.addr[i];
  mac[j] = 0;

  char hex[63] = {0};
  for (int i = 0; i < a.mfgLen; i++) sprintf(hex + i * 2, "%02x", a.mfg[i]);

  char topic[96], payload[200];
  snprintf(topic, sizeof(topic), "%s/devices/%s", PREFIX, mac);
  snprintf(payload, sizeof(payload), "{\"rssi\":%d,\"name\":\"%s\",\"mfg\":\"%s\"}",
           a.rssi, a.name, hex);
  if (mqtt.publish(topic, payload)) published++;
}

void connectWifi() {
  WiFi.mode(WIFI_STA);
  WiFi.begin(WIFI_SSID, WIFI_PASSWORD);
  while (WiFi.status() != WL_CONNECTED) delay(500);
}

void connectMqtt() {
  char st[96];
  snprintf(st, sizeof(st), "%s/gateway/status", PREFIX);
  while (!mqtt.connected()) {
    if (mqtt.connect(CLIENT_ID, st, 1, true, "offline")) {
      mqtt.publish(st, "online", true);
    } else {
      delay(5000);
    }
  }
}

void setup() {
  Serial.begin(115200);
  advQueue = xQueueCreate(40, sizeof(Adv));

  connectWifi();
  mqtt.setServer(MQTT_HOST, MQTT_PORT);
  mqtt.setBufferSize(512);
  mqtt.setKeepAlive(30);
  connectMqtt();

  BLEDevice::init("");
  scanner = BLEDevice::getScan();
  scanner->setAdvertisedDeviceCallbacks(new ScanCallbacks(), true); // report duplicates
  scanner->setActiveScan(false);   // passive: do not send scan requests
  scanner->setInterval(160);       // units of 0.625 ms
  scanner->setWindow(80);          // leave airtime for Wi-Fi
}

void loop() {
  if (WiFi.status() != WL_CONNECTED) connectWifi();
  if (!mqtt.connected()) connectMqtt();

  scanner->start(SCAN_SECONDS, false);   // blocking scan, results go to the queue
  scanner->clearResults();               // free memory held by the scan

  Adv a;
  while (xQueueReceive(advQueue, &a, 0) == pdTRUE) {
    if (shouldPublish(a.addr)) publishAdv(a);
    mqtt.loop();
  }

  char topic[96], stats[96];
  snprintf(topic, sizeof(topic), "%s/gateway/stats", PREFIX);
  snprintf(stats, sizeof(stats), "{\"seen\":%lu,\"published\":%lu,\"dropped\":%lu}",
           (unsigned long)seen, (unsigned long)published, (unsigned long)dropped);
  mqtt.publish(topic, stats);
  mqtt.loop();
}

Watch the traffic from your computer:

mosquitto_sub -h broker.connectmq.com -p 1883 -i ble-a81f-viewer -t "connectmq/demo/ble-a81f/#" -v

Within a few seconds you should see a line per nearby device. In a typical home or office, phones, headphones, TVs and watches all advertise, so expect more devices than you own sensors.

Filtering What You Forward

Forwarding every advertisement in range is useful for discovery but noisy in production, and it can push you into rate limits on shared brokers (the public test area allows 10 messages per second per client). Three filters keep traffic under control:

1. RSSI threshold. MIN_RSSI drops devices that are far away or behind walls.
2. Per-device rate limit. MIN_REPUBLISH_MS publishes each device at most once every ten seconds even though it may advertise every second.
3. Allowlist. Once you know your sensors' addresses or manufacturer IDs, accept only those. The first two bytes of manufacturer data are the company identifier, which makes a good filter for a specific brand.

Note that many phones and some accessories use random, rotating addresses for privacy, so the same phone may appear under several MACs. Sensors usually use a fixed address.

Decoding Sensor Data

The gateway forwards manufacturer data as hex and leaves decoding to a subscriber. That is deliberate: firmware updates are hard, while a Python or Node-RED subscriber can learn new sensor formats in minutes. Many open-source sensor firmwares follow documented formats, such as BTHome service data, which a subscriber can decode into temperature, humidity and battery values and republish to a clean topic like home/livingroom/climate.

This split (dumb, reliable gateway at the edge; flexible logic in a subscriber) is the same pattern used by larger industrial gateways, described in the IIoT edge gateway article.

Troubleshooting Checklist for the ESP32 BLE MQTT Gateway

1. No devices found. Confirm your board supports BLE (the ESP32-S2 does not) and that the sensor is advertising, using a phone scanner app.
2. MQTT disconnects during scans. Shorten SCAN_SECONDS or raise the keep alive; the client cannot answer pings while the scan blocks.
3. Random resets. Check you are not publishing from the scan callback and that the queue is large enough. A growing dropped counter means it is too small.
4. Wi-Fi feels slow or drops. Reduce the scan window relative to the interval so Wi-Fi gets more airtime.
5. Heap shrinking over time. Call clearResults() after each scan, as the sketch does.

For reconnection strategies that survive router reboots, see ESP32 MQTT reconnect, and for more sensor projects the ESP32 IoT tutorial.

Securing the Gateway

BLE advertisements are public by nature: anyone nearby can read them. Do not make it worse by publishing them to a public topic in production. Move to a private broker on TLS port 8883 with WiFiClientSecure and credentials from your broker portal. A managed MQTT broker such as ConnectMQ gives each gateway its own credentials without running servers, and while you prototype the free public MQTT broker is fine as long as nothing sensitive is in range.

Get Started with ConnectMQ

When the gateway is ready for real use, give it a private, encrypted endpoint on ConnectMQ. The portal's live topic explorer shows every device topic as it appears, which is the fastest way to see what your gateway hears. Start on the free Serverless plan at app.connectmq.com, no credit card required.

Frequently Asked Questions

How many BLE sensors can one ESP32 gateway handle?

For passive scanning, dozens of devices in range is normal; the practical limits are airtime, queue size and how often you publish. Rate-limit per device and filter to the sensors you care about.

Should I use NimBLE instead of the built-in BLE library?

NimBLE-Arduino uses less memory and is a good choice for larger gateways. Its API differs slightly, so adapt the callback code if you switch.

Can the gateway connect to sensors that require pairing?

Yes, but that requires GATT connections rather than scanning, and only a few connections at a time. For broadcast sensors, passive scanning is simpler and more reliable.

Why do I see devices I do not own?

Phones, TVs, headphones and many other products advertise continuously. Use an RSSI threshold and an allowlist of addresses or manufacturer IDs.