TensorFlow Lite Micro: Getting Started on a Microcontroller with MQTT Output

TensorFlow Lite Micro is the runtime most TinyML projects use to run neural networks on microcontrollers, and its API is small once you understand four objects: the model, the op resolver, the tensor arena and the interpreter. This getting started guide explains each of them, walks through a complete example on an ESP32-S3 board, and shows how to send predictions to an MQTT broker, either directly over Wi-Fi or through a serial bridge for boards without networking.

What TensorFlow Lite Micro Is

TensorFlow Lite for Microcontrollers (often shortened to TFLM, and now part of Google's LiteRT family) is a C++ inference engine designed for devices with no operating system, no dynamic memory allocation during inference and very little RAM. It reads the same .tflite flatbuffer format as TensorFlow Lite on phones, but supports a smaller set of operators and is meant to be compiled directly into your firmware.

TFLM only does inference. Training happens in TensorFlow on a computer; the microcontroller receives a converted, usually int8-quantized, model. If you want the bigger picture of why anyone runs ML on such small chips, see what TinyML is.

The Four Core Concepts

A typical program does the following once at startup: map the model, register the ops it uses, construct the interpreter, allocate tensors. Then, for each inference: write the input tensor, call Invoke(), read the output tensor. That is the entire runtime API you need for most projects.

Step 1: Train and Convert a Model

The classic first TFLM model learns sin(x). It is small, easy to verify by eye, and exercises quantized input and output. This script trains it, converts it to full int8 and writes C++ source files for the firmware.

pip install tensorflow numpy

# train_sine.py
import numpy as np
import tensorflow as tf

rng = np.random.default_rng(1)
x = rng.uniform(0, 2 * np.pi, 3000).astype(np.float32)
y = (np.sin(x) + 0.05 * rng.standard_normal(x.size)).astype(np.float32)

model = tf.keras.Sequential([
    tf.keras.Input(shape=(1,)),
    tf.keras.layers.Dense(16, activation="relu"),
    tf.keras.layers.Dense(16, activation="relu"),
    tf.keras.layers.Dense(1),
])
model.compile(optimizer="adam", loss="mse")
model.fit(x, y, epochs=200, batch_size=64, validation_split=0.2, verbose=0)
print("val mse:", model.evaluate(x[:500], y[:500], verbose=0))


def rep():
    for v in np.linspace(0, 2 * np.pi, 300, dtype=np.float32):
        yield [np.array([[v]], dtype=np.float32)]


conv = tf.lite.TFLiteConverter.from_keras_model(model)
conv.optimizations = [tf.lite.Optimize.DEFAULT]
conv.representative_dataset = rep
conv.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS_INT8]
conv.inference_input_type = tf.int8
conv.inference_output_type = tf.int8
tfl = conv.convert()
print("model bytes:", len(tfl))

with open("tflm_sine/main/model_data.cc", "w") as f:
    f.write('#include "model_data.h"\n')
    f.write("alignas(16) const unsigned char g_model[] = {%s};\n" % ", ".join(f"0x{b:02x}" for b in tfl))
    f.write(f"const unsigned int g_model_len = {len(tfl)};\n")
with open("tflm_sine/main/model_data.h", "w") as f:
    f.write("#pragma once\nextern const unsigned char g_model[];\nextern const unsigned int g_model_len;\n")

Before writing firmware, inspect the operators in the converted model. Opening the .tflite file in a model viewer such as Netron or using tf.lite.experimental.Analyzer.analyze(model_content=tfl) shows that this model contains only FULLY_CONNECTED operators. That list is exactly what you register in the resolver.

Why Quantize for TensorFlow Lite Micro

Microcontrollers are far faster at integer arithmetic than at floating point, and many have no floating-point unit at all. Full int8 quantization stores each weight in one byte instead of four and runs every layer with integer kernels, which shrinks flash use, shrinks the tensor arena and lets optimized libraries such as CMSIS-NN or Espressif's kernels do their work. The cost is a small loss of precision and the extra step of converting inputs and outputs with each tensor's scale and zero_point, which the firmware below does explicitly.

Flash and RAM: Where the Model Lives

The model array is declared const, so the linker places it in flash and it costs no RAM. The tensor arena is the RAM cost: inputs, outputs, intermediate activations and some interpreter bookkeeping. On the ESP32 family, internal RAM is the fast option; very large arenas can go into PSRAM on boards that have it, at some speed penalty. For this sine model the arena needs well under the 4 KB reserved below, which you can confirm from the log line the firmware prints.

Step 2: Set Up the Board Project

This guide uses an ESP32-S3 development board, which is also programmable from the Arduino IDE, but builds with ESP-IDF because Espressif's esp-tflite-micro component tracks the upstream TFLM API and adds optimized kernels. On other boards, the C++ code in the next step stays the same; only the build system changes.

tflm_sine/
  CMakeLists.txt
  main/
    CMakeLists.txt
    main.cc
    model_data.cc   (generated)
    model_data.h    (generated)

# tflm_sine/CMakeLists.txt
cmake_minimum_required(VERSION 3.16)
include($ENV{IDF_PATH}/tools/cmake/project.cmake)
project(tflm_sine)

# tflm_sine/main/CMakeLists.txt
idf_component_register(SRCS "main.cc" "model_data.cc" INCLUDE_DIRS ".")

cd tflm_sine
idf.py set-target esp32s3
idf.py add-dependency "espressif/esp-tflite-micro"

Step 3: Run Inference with TensorFlow Lite Micro

The firmware walks x from 0 to 2π, quantizes each value with the input tensor's scale and zero point, runs the model and prints one JSON line per prediction on the serial console.

// tflm_sine/main/main.cc
#include <algorithm>
#include <cmath>
#include <cstdio>

#include "esp_log.h"
#include "esp_timer.h"
#include "freertos/FreeRTOS.h"
#include "freertos/task.h"
#include "tensorflow/lite/micro/micro_interpreter.h"
#include "tensorflow/lite/micro/micro_mutable_op_resolver.h"
#include "tensorflow/lite/schema/schema_generated.h"

#include "model_data.h"

static const char *TAG = "tflm";
static constexpr int kArenaSize = 4 * 1024;
alignas(16) static uint8_t tensor_arena[kArenaSize];

extern "C" void app_main(void) {
  // 1. Map the model
  const tflite::Model *model = tflite::GetModel(g_model);
  if (model->version() != TFLITE_SCHEMA_VERSION) {
    ESP_LOGE(TAG, "model schema %lu != runtime %d", (unsigned long)model->version(), TFLITE_SCHEMA_VERSION);
    return;
  }

  // 2. Register only the operators the model uses
  static tflite::MicroMutableOpResolver<3> resolver;
  resolver.AddFullyConnected();
  resolver.AddQuantize();
  resolver.AddDequantize();

  // 3. Build the interpreter over a static arena and allocate tensors
  static tflite::MicroInterpreter interpreter(model, resolver, tensor_arena, kArenaSize);
  if (interpreter.AllocateTensors() != kTfLiteOk) {
    ESP_LOGE(TAG, "AllocateTensors failed, increase kArenaSize");
    return;
  }
  ESP_LOGI(TAG, "arena used %u of %d bytes", (unsigned)interpreter.arena_used_bytes(), kArenaSize);

  TfLiteTensor *input = interpreter.input(0);
  TfLiteTensor *output = interpreter.output(0);
  const float two_pi = 2.0f * (float)M_PI;
  int step = 0;

  while (true) {
    float x = two_pi * (step % 40) / 40.0f;

    // 4. Quantize the input: q = x / scale + zero_point
    int q = (int)lroundf(x / input->params.scale) + input->params.zero_point;
    input->data.int8[0] = (int8_t)std::min(127, std::max(-128, q));

    int64_t t0 = esp_timer_get_time();
    if (interpreter.Invoke() != kTfLiteOk) {
      ESP_LOGE(TAG, "Invoke failed");
      break;
    }
    int us = (int)(esp_timer_get_time() - t0);

    // 5. Dequantize the output: y = (q - zero_point) * scale
    float y = (output->data.int8[0] - output->params.zero_point) * output->params.scale;
    printf("{\"x\":%.3f,\"y\":%.3f,\"ref\":%.3f,\"us\":%d}\n", x, y, sinf(x), us);

    step++;
    vTaskDelay(pdMS_TO_TICKS(500));
  }
}

python train_sine.py
cd tflm_sine
idf.py build flash monitor

The monitor shows lines such as {"x":1.571,"y":0.98,"ref":1.000,"us":...}. Predicted y should track ref closely, with small errors from quantization. If every output is the same value, the input is almost certainly being written without quantization.

Step 4: Send Predictions over MQTT

There are two ways to get these predictions onto a broker.

Direct over Wi-Fi. On Wi-Fi boards like the ESP32-S3, add the ESP-IDF MQTT client and publish each JSON line instead of printing it. The TinyML on ESP32 tutorial contains a complete firmware that does exactly that, including Last Will and publish-on-change logic.

Serial bridge. Many popular TFLM boards, such as Cortex-M boards with onboard sensors, have no Wi-Fi. They can print JSON over USB serial, and a computer or Raspberry Pi forwards it. This bridge works with the firmware above unchanged:

pip install pyserial "paho-mqtt>=2.0"

# serial_to_mqtt.py  usage: python serial_to_mqtt.py COM5   (or /dev/ttyACM0)
import json
import sys
import uuid

import paho.mqtt.client as mqtt
import serial

port = sys.argv[1]
run = uuid.uuid4().hex[:8]
topic = f"connectmq/demo/{run}/lab/tflm-board/inference"

client = mqtt.Client(mqtt.CallbackAPIVersion.VERSION2, client_id=f"tflm-bridge-{run}")
client.will_set(f"connectmq/demo/{run}/lab/tflm-board/status", "offline", qos=1, retain=True)
client.connect("broker.connectmq.com", 1883, keepalive=60)
client.loop_start()
client.publish(f"connectmq/demo/{run}/lab/tflm-board/status", "online", qos=1, retain=True)
print("publishing to", topic)

with serial.Serial(port, 115200, timeout=2) as ser:
    while True:
        line = ser.readline().decode("utf-8", errors="ignore").strip()
        if not line.startswith("{"):
            continue                      # skip boot and log lines
        try:
            json.loads(line)              # forward only valid JSON
        except json.JSONDecodeError:
            continue
        client.publish(topic, line, qos=0)
        print(line)

Close idf.py monitor first, since only one program can hold the serial port. Then watch the stream from any machine:

mosquitto_sub -h broker.connectmq.com -p 1883 -i tflm-viewer-12 -t 'connectmq/demo/+/lab/tflm-board/#' -v

The free public MQTT broker is fine for this, as long as you remember that anyone can read the public area. Production devices should use private credentials and TLS on port 8883; in the bridge that means calling client.tls_set() and client.username_pw_set() with the values from your ConnectMQ broker and using the issued client ID.

The bridge pattern has a useful side effect: the board firmware stays free of networking code, certificates and credentials, so you can swap brokers, add TLS or change topics on the gateway without reflashing anything. The trade-off is one more device to keep running. For a handful of lab boards on one computer it is ideal; for a fleet in the field, boards with built-in Wi-Fi that publish directly are usually simpler to operate.

Choosing What to Publish

A regression model like this produces a value on every call, but most real applications care about events: a value crossing a limit, a class changing, or a confidence dropping. Decide on the device or bridge what counts as an event and publish those, plus a slow heartbeat. That keeps traffic well inside per-client rate limits and makes downstream consumers simpler.

Troubleshooting TensorFlow Lite Micro

1. Didn't find op for builtin opcode: register the missing operator and raise the resolver's template count.
2. AllocateTensors() fails: enlarge the arena, then shrink it to arena_used_bytes() plus some margin once it works.
3. Hard fault or crash in Invoke(): the model array is not aligned, or the arena overlaps other memory. Keep alignas(16) on both.
4. Accuracy much worse than on the PC: the representative dataset did not cover the real input range, so quantization clipped values. Use real samples in representative_dataset.
5. Compile errors about MicroErrorReporter or AllOpsResolver: you are mixing old example code with a newer TFLM version. Current versions use the constructor shown above and MicroMutableOpResolver.

Where to Go Next

Replace the sine model with a classifier fed by real sensor features, as in TinyML on ESP32, or let a platform generate the pipeline for you with the Edge Impulse MQTT tutorial. The official TensorFlow Lite Micro repository documents supported operators and porting to new boards. For architecture context, the AI and IoT guide covers how microcontroller models fit alongside cloud analytics.

Get Started with ConnectMQ

Whether predictions leave your board over Wi-Fi or through a bridge, they need a reliable destination. ConnectMQ is a fully managed MQTT service with TLS, MQTT 3.1.1 and 5.0, and an in-browser MQTT client for checking messages as they arrive. Create a broker on the free Serverless plan at app.connectmq.com, no credit card required.

Frequently Asked Questions

What is the difference between TensorFlow Lite and TensorFlow Lite Micro?

TensorFlow Lite targets phones and Linux devices with an operating system and dynamic memory. TensorFlow Lite Micro is a stripped-down runtime for microcontrollers that uses a static arena, a reduced operator set and no OS dependencies.

How do I choose the tensor arena size?

Start with a generous size, call AllocateTensors(), then read arena_used_bytes() and set the arena to that value plus a safety margin. The requirement depends on the model's largest intermediate tensors.

Does TensorFlow Lite Micro support float models?

Yes, many operators have float kernels, but int8 models are smaller and much faster on most microcontrollers. Full integer quantization is the normal choice for production.

Can TensorFlow Lite Micro send data over MQTT itself?

No. TFLM only performs inference. You publish results with a separate MQTT client, such as the ESP-IDF MQTT client, PubSubClient, or a serial bridge on a gateway.