What Is TinyML? Machine Learning on Microcontrollers Explained

What is TinyML? It is machine learning that runs on microcontrollers, the same class of chips that sit inside thermostats, wearables and sensor nodes, with memory measured in kilobytes and power measured in milliwatts. This explainer covers what makes TinyML different from other edge AI, which frameworks and boards to use, the use cases where it shines, and why these devices should publish compact inference results over MQTT instead of raw sensor data.

TinyML in Plain Terms

A microcontroller (MCU) is a small computer on a single chip: a processor, some RAM, flash storage and peripherals for sensors. There is no operating system in the desktop sense, no Python runtime and often no floating-point hardware. TinyML is the set of techniques that make a trained neural network small and efficient enough to run in that environment, continuously, often on battery.

The result is a sensor that understands what it measures. Instead of streaming raw audio, a TinyML microphone node reports "keyword detected". Instead of sampling vibration at high rate and uploading it, a motor node reports "imbalance, confidence 0.9". TinyML is a subset of edge AI; if you want the wider picture first, read what edge AI is.

The Constraints That Define TinyML

These limits shape every design choice:

1. Models must be small. Small convolutional or dense networks with tens of thousands of parameters are typical.
2. Integer arithmetic. Models are quantized to 8-bit integers, which reduces size roughly fourfold compared with 32-bit floats and runs fast on chips without floating-point units or with SIMD instructions.
3. Static memory. There is no dynamic allocation during inference. Frameworks use a fixed memory block (the tensor arena) sized at compile time.
4. Feature extraction matters. Raw signals are converted into compact features, such as a spectrogram for audio or RMS and spectral peaks for vibration, before the model sees them.

How a TinyML Model Is Built

The workflow mirrors any ML project, with an extra squeeze at the end:

1. Collect labelled sensor data from the real device in the real environment.
2. Design features and a small model, usually in TensorFlow/Keras or a platform like Edge Impulse.
3. Train and validate on a workstation or in the cloud.
4. Convert and quantize the model, for example to a .tflite file with full int8 quantization.
5. Embed the model in firmware as a byte array and link the inference runtime.
6. Measure on the device: inference time, RAM use (arena size), flash use and accuracy.
7. Iterate. Shrinking the input window or the number of features often helps more than shrinking the network.

TinyML Frameworks and Tools

• TensorFlow Lite for Microcontrollers (now part of Google's LiteRT family): the most widely used open runtime. You load a flatbuffer model, register only the operators you need and run an interpreter over a static arena. See the TensorFlow Lite Micro getting started guide.
• Edge Impulse: an end-to-end platform for data collection, feature engineering, training and exporting a C++ library or Arduino library for a target board. See the Edge Impulse MQTT tutorial.
• Vendor libraries: Arm CMSIS-NN provides optimized kernels for Cortex-M; Espressif provides ESP-DL and an ESP32-optimized port of TensorFlow Lite Micro (esp-tflite-micro); other silicon vendors have their own toolchains.
• Arduino libraries and PlatformIO make it easier to combine inference with sensor drivers and networking on hobbyist boards.

Popular TinyML Boards

ESP32-S3 boards are a common choice for connected TinyML because they combine vector instructions useful for neural network kernels, optional PSRAM, and built-in Wi-Fi in one inexpensive chip. Arduino Nano 33 BLE Sense and similar Cortex-M4 boards with onboard sensors are popular for learning, and the Raspberry Pi Pico family offers a low-cost Cortex-M option. Boards without Wi-Fi can still participate in an IoT system by sending results over serial or Bluetooth to a gateway. For a direct comparison across device classes, including accelerators, see edge AI hardware compared.

What TinyML Is Good At

• Keyword spotting and sound events: wake words, alarms, glass breaking, abnormal machine noise.
• Motion and gesture recognition from accelerometers and gyroscopes in wearables and tools.
• Predictive maintenance: classifying vibration patterns on motors, pumps and fans.
• Simple vision: person presence, digit reading or "object present" on low-resolution cameras with PSRAM-equipped boards.
• Environmental sensing: inferring occupancy or anomalies from gas, temperature and humidity patterns.

TinyML is not a good fit for tasks that need large vocabularies, high-resolution images, long context or frequent model changes. Those belong on a gateway or in the cloud.

Why TinyML Devices Publish Compact Results over MQTT

Connected TinyML nodes face two limits at once: tiny compute and expensive radio time. On battery devices, the radio often uses far more energy than the inference. Sending a short message only when something interesting happens is the biggest power and bandwidth saving available.

MQTT fits that pattern. The protocol overhead is small, client libraries such as PubSubClient and esp-mqtt run comfortably on MCUs, and the broker fans each message out to whoever needs it. A typical TinyML inference message is well under 200 bytes:

{"dev":"fan-12","m":"vib-v2","label":"imbalance","p":0.91,"ms":7}

Short keys are a deliberate choice on constrained devices. Pair them with a topic that carries the context, such as plant1/hvac/ahu3/fan-12/inference, and a retained status topic with a Last Will so dashboards know when the node goes offline. On the receiving side, an MQTT broker lets dashboards, alerting and cloud analytics subscribe without the device knowing about them.

A Minimal Publishing Pattern for Microcontrollers

The core loop on an ESP32 with PubSubClient looks like this. The inference call is a placeholder; the TinyML on ESP32 tutorial shows a full working model.

// Called after each inference; publishes only on change or every 60 s.
void reportResult(PubSubClient &mqtt, const char *topic, const char *label, float p) {
  static char lastLabel[24] = "";
  static unsigned long lastSent = 0;
  if (strcmp(label, lastLabel) == 0 && millis() - lastSent < 60000) return;

  char payload[96];
  snprintf(payload, sizeof(payload),
           "{\"dev\":\"fan-12\",\"m\":\"vib-v2\",\"label\":\"%s\",\"p\":%.2f}", label, p);
  if (mqtt.publish(topic, payload)) {
    strncpy(lastLabel, label, sizeof(lastLabel) - 1);
    lastSent = millis();
  }
}

You can test a node like this against the public MQTT test broker at broker.connectmq.com on port 1883 with any unique client ID and a unique topic prefix. Remember that the public area is readable by everyone, so move real deployments to private credentials, such as a ConnectMQ broker with TLS on port 8883.

Common TinyML Mistakes

1. Training on lab data only. Mounting, temperature and background noise in the field change the signal.
2. Ignoring the arena size. If AllocateTensors() fails, the arena is too small; measure the actual requirement and add a margin.
3. Unsupported operators. Not every TensorFlow layer has a microcontroller kernel. Check the converted model's operators before you design around them.
4. Publishing every inference. A model running ten times per second should not publish ten messages per second; aggregate or publish on change.
5. No model version. Include it in every message so you can trace behaviour after an over-the-air update.

For how TinyML fits into a full system with cloud analytics and LLM agents, read the AI and IoT guide.

Get Started with ConnectMQ

Once your TinyML node works on the bench, give it a secure home. ConnectMQ is a hosted MQTT broker with TLS on port 8883, MQTT 3.1.1 and 5.0 support and a live topic explorer for watching inference messages arrive. Create a broker on the free Serverless plan at app.connectmq.com, no credit card required.

Frequently Asked Questions

What is TinyML used for?

TinyML is used for always-on sensing tasks on low-power devices, such as keyword spotting, gesture recognition, vibration-based fault detection and simple presence detection.

Can an ESP32 run TinyML?

Yes. The ESP32 family, especially the ESP32-S3 with vector instructions and PSRAM options, runs TensorFlow Lite Micro and Edge Impulse models and can publish results over Wi-Fi with MQTT.

Is TinyML the same as edge AI?

TinyML is a part of edge AI focused on microcontrollers. Edge AI also covers larger devices such as single-board computers, accelerators and GPU modules.

Do TinyML models need the internet?

No. Inference runs entirely on the device. Connectivity is only needed to report results, receive configuration or update the model.