Edge AI Hardware Compared: Jetson, Coral, Hailo, Raspberry Pi and ESP32-S3

Choosing edge AI hardware is a trade-off between model size, frame rate, power budget, cost and how much toolchain pain your team can absorb. This comparison covers the main options, from NVIDIA Jetson modules and Coral and Hailo accelerators to plain Raspberry Pi CPUs and ESP32-S3 microcontrollers, explains where each fits, and gives you a benchmark script that publishes your own model's latency over MQTT so you can compare boards on real numbers instead of datasheets.

How to Compare Edge AI Hardware

Vendor datasheets usually quote peak operations per second, but peak numbers rarely predict what your model will do. Several factors matter more in practice:

1. Operator support. An accelerator only speeds up the operators its compiler supports. Unsupported layers fall back to the CPU, sometimes erasing the gain.
2. Precision. Most NPUs and TPUs need int8 quantized models. GPUs can run FP16 and FP32 as well.
3. Memory. Model weights, activations and camera buffers must fit. On microcontrollers this is the first limit you hit.
4. Power and thermals. A board that throttles in a sealed enclosure is slower than its benchmark.
5. Toolchain and lifecycle. How models are converted, how long the vendor supports the OS images and libraries, and whether the board is available for the lifetime of your product.
6. Connectivity. Every option here can publish results to an MQTT broker; what differs is whether it has Wi-Fi built in, Ethernet, or needs a gateway.

Edge AI Hardware Comparison Table

The power classes are intentionally broad, because actual consumption depends on workload, clocks and peripherals. Measure your own setup.

Microcontrollers: ESP32-S3

The ESP32-S3 sits at the bottom of the power and cost range and is the only option here that can run on a small battery for long periods. Its vector instructions accelerate neural network kernels in Espressif's libraries, and PSRAM variants can hold small vision models. It is ideal when a model's job is to turn a sensor stream into an occasional event. The limits are firm: models in the kilobyte to low-megabyte range and low-resolution images only. See TinyML on ESP32 for a complete build and what TinyML is for the constraints.

Raspberry Pi with and without Accelerators

A Raspberry Pi 5 on its own runs small classification and detection models on the CPU, which is enough for prototypes, occupancy counting at a few frames per second or audio classification. Its strength is the software ecosystem: Python, OpenCV, camera libraries and easy MQTT clients.

Adding a Hailo NPU through the Raspberry Pi AI Kit or AI HAT+ changes the picture for vision. Real-time object detection becomes practical while the CPU stays free for application logic. Models must be compiled with Hailo's toolchain, and the official Raspberry Pi AI documentation covers setup and camera integration. Raspberry Pi also offers an AI Camera that runs inference on the image sensor itself, which suits single-camera deployments.

Coral Edge TPU

Coral accelerators run int8 TensorFlow Lite models compiled for the Edge TPU and are very power-efficient for models whose operators the compiler fully maps. They come as USB sticks, M.2 and PCIe cards, and a dev board. Before committing, check the current state of Coral's software support for your operating system and Python version, since it has historically moved more slowly than the hardware it plugs into. The Coral Edge TPU guide covers setup and MQTT publishing in detail.

NVIDIA Jetson

Jetson modules bring a full CUDA GPU to the edge. They handle multiple camera streams, larger detectors, segmentation, pose estimation and, on higher-end modules, small language and vision-language models. TensorRT optimizes models for the GPU, and DeepStream builds multi-stream video pipelines. The trade-offs are higher power, active cooling in most enclosures, and a heavier software stack tied to JetPack releases. Read the Jetson Nano MQTT guide for a hands-on example.

Choosing Edge AI Hardware: A Checklist

1. Define the task and target rate: what model, what input size, how many inferences per second, and how many cameras or sensors.
2. Set the power and environment budget: battery, PoE, fanless enclosure, temperature range.
3. Check operator compatibility by compiling your actual model for each candidate, not a demo model.
4. Benchmark end to end, including pre-processing and post-processing, which often dominate on small devices.
5. Plan updates: how will a new model reach the device, and how long will the OS and SDK be supported?
6. Pick the smallest platform that meets the target with margin. Smaller hardware is cheaper, cooler and simpler to maintain.

Benchmark Your Model and Publish the Results over MQTT

The fairest comparison is your own model on each board, reported to one place. This script runs a TensorFlow Lite model repeatedly with random input, measures latency, and publishes a summary to the same topic tree from every board, so a single subscriber sees all results side by side. It uses the LiteRT interpreter if installed, falls back to tflite_runtime or TensorFlow, and can load the Coral Edge TPU delegate.

pip install "paho-mqtt>=2.0" numpy ai-edge-litert   # or tflite-runtime / tensorflow

import argparse
import json
import platform
import statistics
import time
import uuid

import numpy as np
import paho.mqtt.client as mqtt

try:
    from ai_edge_litert.interpreter import Interpreter, load_delegate
except ImportError:
    try:
        from tflite_runtime.interpreter import Interpreter, load_delegate
    except ImportError:
        import tensorflow as tf
        Interpreter, load_delegate = tf.lite.Interpreter, tf.lite.experimental.load_delegate

ap = argparse.ArgumentParser()
ap.add_argument("model")
ap.add_argument("--board", required=True, help="label, e.g. rpi5-cpu or rpi5-coral")
ap.add_argument("--prefix", required=True, help="e.g. connectmq/demo/<random-id>/bench")
ap.add_argument("--edgetpu", action="store_true", help="load the Coral Edge TPU delegate")
ap.add_argument("--runs", type=int, default=100)
args = ap.parse_args()

delegates = [load_delegate("libedgetpu.so.1")] if args.edgetpu else None
interp = Interpreter(model_path=args.model, experimental_delegates=delegates)
interp.allocate_tensors()
inp = interp.get_input_details()[0]

if np.issubdtype(inp["dtype"], np.integer):
    info = np.iinfo(inp["dtype"])
    data = np.random.randint(info.min, info.max + 1, size=inp["shape"], dtype=inp["dtype"])
else:
    data = np.random.rand(*inp["shape"]).astype(inp["dtype"])

for _ in range(10):                                   # warm-up
    interp.set_tensor(inp["index"], data)
    interp.invoke()

times = []
for _ in range(args.runs):
    t0 = time.perf_counter()
    interp.set_tensor(inp["index"], data)
    interp.invoke()
    times.append((time.perf_counter() - t0) * 1000)

times.sort()
result = {
    "board": args.board,
    "machine": platform.machine(),
    "model": args.model.rsplit("/", 1)[-1],
    "input": [int(d) for d in inp["shape"]],
    "runs": args.runs,
    "median_ms": round(statistics.median(times), 2),
    "p95_ms": round(times[int(0.95 * len(times)) - 1], 2),
    "fps_est": round(1000 / statistics.median(times), 1),
}
print(result)

client = mqtt.Client(mqtt.CallbackAPIVersion.VERSION2, client_id=f"bench-{args.board}-{uuid.uuid4().hex[:6]}")
client.connect("broker.connectmq.com", 1883, keepalive=30)
client.loop_start()
client.publish(f"{args.prefix}/{args.board}", json.dumps(result), qos=1, retain=True).wait_for_publish()
client.loop_stop()
client.disconnect()

Run it on each board with the same prefix, then read all results at once. Because each result is retained, boards benchmarked yesterday still appear:

python bench.py detect_int8.tflite --board rpi5-cpu --prefix connectmq/demo/8d2f41/bench
python bench.py detect_int8_edgetpu.tflite --board rpi5-coral --edgetpu --prefix connectmq/demo/8d2f41/bench
mosquitto_sub -h broker.connectmq.com -p 1883 -i bench-viewer-3 -t 'connectmq/demo/8d2f41/bench/#' -v

This measures model invoke time only. Add your camera capture and pre-processing to the timed section for an end-to-end figure. For Jetson, export the same network to TensorRT and time it with the equivalent loop; for the ESP32-S3, log esp_timer_get_time() around Invoke() as the TinyML tutorial does and publish it in the same JSON shape. When you are finished, clear each retained result by publishing an empty retained message to its topic.

Same Output, Different Hardware

Whatever you choose, keep the output contract identical across platforms: the same topic hierarchy, the same JSON fields, and a model and hardware field in every message. Then you can mix boards in one deployment, such as ESP32-S3 nodes on motors and a Jetson on the camera line, and every consumer reads results the same way. A cloud MQTT broker makes that easy across sites: with ConnectMQ, each board connects outbound over TLS with its own credentials and client ID, and the portal topic explorer shows every board publishing into the same tree. For background, see what edge AI is and the AI and IoT guide.

Connect Any Edge AI Board to ConnectMQ

From a battery-powered ESP32-S3 to a multi-camera Jetson, every board speaks MQTT. ConnectMQ is a managed MQTT broker with TCP, TLS and WebSocket endpoints, MQTT 5 and a live topic explorer, so results from mixed hardware land in one place. Create a broker on the free Serverless plan at app.connectmq.com, no credit card required.

Frequently Asked Questions

What is the best edge AI hardware for beginners?

A Raspberry Pi 5 is the easiest start because it runs standard Python tools. Add a Hailo or Coral accelerator when you need real-time vision, and try an ESP32-S3 when you want low-power sensor models.

Is Jetson better than Coral or Hailo?

Jetson is more flexible and runs larger models and more operator types, but uses more power. Coral and Hailo are efficient for supported int8 models. The right choice depends on your model and power budget.

Can an ESP32-S3 run object detection?

It can run very small detection models on low-resolution images, which is enough for presence or simple counting. For general-purpose detection at useful frame rates, use a Linux board with an accelerator.

Why benchmark my own model instead of using published numbers?

Published figures use specific models, input sizes and settings. Operator support, quantization and pre-processing differ for your model, so only your own measurements predict real performance.