How to Deploy ML Models to Edge Devices: Packaging, Versioning and MQTT-Driven Rollouts

Training a model is only half the job: to create value you have to deploy ML models to edge devices, keep them up to date and know at any moment which version each device is running. This guide shows a proven pattern for packaging, versioning and rolling out models across a fleet, using MQTT to signal updates and collect status while the model files themselves travel over HTTPS.

Why Edge Model Deployment Is Hard

Shipping a model to a server is a single deployment. Shipping it to hundreds of gateways, cameras or microcontrollers is a distributed systems problem:

• Devices are offline at random times and must catch up when they reconnect.
• Hardware differs, so one model artifact may not fit every device.
• A bad model can silently degrade predictions rather than crash, so you need health signals beyond "the process is running".
• Bandwidth and storage are limited, and some sites are behind firewalls that only allow outbound connections.

MQTT fits these constraints well. Devices keep a single outbound connection to a broker, retained messages give late joiners the latest instruction immediately and a last will message tells you when a device drops off. If you are new to this split of responsibilities, our overview of what an MQTT broker does is a good primer.

Step 1: Package the Model as a Versioned Artifact

Treat every model as an immutable release, never as a file that gets overwritten. A model package should contain:

Preprocessing mismatches are the most frequent cause of "the model works on my laptop but not on the device". Recording the expected input size, color order and normalization in the manifest and having the device check it at load time prevents a whole class of silent errors.

Choose the format per target:

For microcontrollers the model often lives inside the firmware image, so a model update is a firmware update. Our TensorFlow Lite Micro tutorial covers that case; the rest of this guide focuses on Linux-class devices that load models from disk.

Step 2: Version Semantically and Sign Every Release

Use a version string that tells operators something, such as defect-classifier 2.3.0, where the major number changes when inputs or outputs change incompatibly. Alongside the version, compute a SHA-256 checksum of the artifact and publish it in the manifest. Devices must refuse a file whose checksum does not match. For higher assurance, sign the manifest with a private key held by your CI system and ship only the public key to devices.

Store artifacts on HTTPS object storage or an internal file server. MQTT is the wrong channel for multi-megabyte binaries: the broker would have to hold large payloads in memory and many plans cap message size. ConnectMQ's free Serverless plan, for example, limits messages to 256 KB, which is plenty for a manifest but not for a vision model.

Step 3: Signal Updates Over MQTT

The control plane uses three topics per device (or per device group):

fleet/<site>/<device>/model/desired    retained, written by the deployment service
fleet/<site>/<device>/model/reported   retained, written by the device
fleet/<site>/<device>/model/events     not retained, progress and errors

The deployment service publishes a desired-state message, retained so a device that is offline receives it as soon as it connects:

{
  "model": "defect-classifier",
  "version": "2.3.0",
  "url": "https://models.example.com/defect-classifier/2.3.0/model.tflite",
  "sha256": "9f2c4c1d5e8b0a7f3d6e1b2c4a5f6e7d8c9b0a1f2e3d4c5b6a7f8e9d0c1b2a3f",
  "runtime": "tflite",
  "rollout": "canary",
  "issuedAt": "2026-10-16T08:00:00Z"
}

The device compares desired with what it runs, downloads, verifies, swaps and reports back:

{
  "model": "defect-classifier",
  "version": "2.3.0",
  "state": "active",
  "previous": "2.2.1",
  "loadedAt": "2026-10-16T08:03:12Z",
  "selfTest": "passed"
}

This desired/reported pattern is the same idea used by device shadows, and it works with any standards-compliant broker.

Step 4: A Minimal Device Update Agent

The agent below runs on a Linux edge device. It subscribes to its desired topic, downloads the model, verifies the checksum, runs a smoke test, swaps the active model atomically and keeps the previous version for rollback.

import hashlib
import json
import os
import shutil
import time
import urllib.request
from pathlib import Path

import paho.mqtt.client as mqtt

DEVICE = "gw-07"
BASE = f"connectmq/demo/a91e4d/fleet/plant1/{DEVICE}/model"
MODELS = Path("/opt/models")
ACTIVE = MODELS / "active"           # symlink to the active version directory
STATE_FILE = MODELS / "state.json"


def current_version():
    try:
        return json.loads(STATE_FILE.read_text())["version"]
    except (FileNotFoundError, KeyError, json.JSONDecodeError):
        return None


def report(client, state, **extra):
    body = {"model": extra.pop("model", None), "state": state,
            "version": extra.pop("version", current_version()),
            "ts": int(time.time() * 1000), **extra}
    client.publish(f"{BASE}/reported", json.dumps(body), qos=1, retain=True)


def event(client, msg):
    client.publish(f"{BASE}/events", json.dumps({"msg": msg, "ts": int(time.time() * 1000)}), qos=0)


def sha256_of(path):
    h = hashlib.sha256()
    with open(path, "rb") as f:
        for chunk in iter(lambda: f.read(1 << 20), b""):
            h.update(chunk)
    return h.hexdigest()


def smoke_test(model_path):
    # Load the model and run one inference on a stored sample input.
    # Replace with your runtime, e.g. tflite_runtime.interpreter.Interpreter.
    return model_path.stat().st_size > 0


def apply_update(client, desired):
    version = desired["version"]
    if version == current_version():
        report(client, "active", model=desired["model"], version=version)
        return
    target = MODELS / version
    target.mkdir(parents=True, exist_ok=True)
    tmp = target / "model.tmp"
    event(client, f"downloading {version}")
    urllib.request.urlretrieve(desired["url"], tmp)
    if sha256_of(tmp) != desired["sha256"]:
        tmp.unlink(missing_ok=True)
        report(client, "failed", model=desired["model"], reason="checksum mismatch")
        return
    final = target / "model.tflite"
    tmp.replace(final)
    if not smoke_test(final):
        report(client, "failed", model=desired["model"], reason="smoke test failed")
        return
    previous = current_version()
    link_tmp = MODELS / "active.tmp"
    if link_tmp.exists() or link_tmp.is_symlink():
        link_tmp.unlink()
    link_tmp.symlink_to(target)
    os.replace(link_tmp, ACTIVE)  # atomic swap of the symlink
    STATE_FILE.write_text(json.dumps({"version": version, "previous": previous}))
    report(client, "active", model=desired["model"], version=version, previous=previous)
    # Keep only the current and previous versions on disk
    for d in MODELS.iterdir():
        if d.is_dir() and not d.is_symlink() and d.name not in (version, previous):
            shutil.rmtree(d, ignore_errors=True)


def on_connect(client, userdata, flags, reason_code, properties):
    client.subscribe(f"{BASE}/desired", qos=1)
    report(client, "online")


def on_message(client, userdata, msg):
    try:
        apply_update(client, json.loads(msg.payload))
    except Exception as exc:  # report every failure instead of dying silently
        report(client, "failed", reason=str(exc))


MODELS.mkdir(parents=True, exist_ok=True)
client = mqtt.Client(mqtt.CallbackAPIVersion.VERSION2, client_id=f"{DEVICE}-a91e4d")
client.on_connect = on_connect
client.on_message = on_message
client.will_set(f"{BASE}/reported", json.dumps({"state": "offline"}), qos=1, retain=True)
client.connect("broker.connectmq.com", 1883, keepalive=60)
client.loop_forever()

Note that the download runs inside the message callback, which blocks the network loop. That is acceptable for small models and a test setup; in production, hand the work to a worker thread so keepalives continue during long downloads. Your inference process should open the model via the active symlink and reload when it changes, for example by watching the state file.

To trigger a rollout during testing, publish a retained desired message:

mosquitto_pub -h broker.connectmq.com -p 1883 -i deployer-a91e4d -r -q 1 \
  -t "connectmq/demo/a91e4d/fleet/plant1/gw-07/model/desired" \
  -f desired.json

The public MQTT test broker is fine for trying the flow, but remember it is readable by anyone, so never put real model URLs or signed tokens there.

Step 5: Roll Out in Stages

Never push a new model to the whole fleet at once. A staged rollout looks like this:

1. Shadow mode. The new model runs next to the old one on a few devices, and both outputs are published, but only the old one drives actions. Compare them offline.
2. Canary. Publish the desired state to a small group of devices. Watch their reported state, inference latency and prediction distribution.
3. Progressive waves. Expand to larger groups, pausing between waves. A deployment script simply writes the desired topic for the next batch of devices.
4. Full rollout. Once the last wave is healthy, make the new version the default for newly provisioned devices.
5. Rollback. If metrics degrade, publish the previous version as desired. Because the device keeps the last version on disk, rollback does not even need a download.

Group membership can live in your deployment service, or you can encode it in topics, for example fleet/plant1/canary/model/desired, which a device in the canary group subscribes to in addition to its own topic.

Monitoring What Each Device Runs

Because every device publishes a retained reported message, a single wildcard subscription to fleet/+/+/model/reported gives a complete fleet inventory the moment it connects. Combine it with prediction telemetry to watch for drift: if the class distribution or confidence scores shift sharply after an update, the model is probably seeing inputs it was not trained on. Our article on IoT MLOps goes deeper into monitoring and retraining loops.

Production Checklist

1. Artifacts are immutable, versioned and stored on HTTPS.
2. Every manifest carries a SHA-256 checksum, ideally a signature.
3. Devices verify, smoke-test and swap atomically, keeping one previous version.
4. Desired and reported state live on retained MQTT topics, with a last will for offline detection.
5. Rollouts go shadow, canary, waves, full, with a one-message rollback.
6. Brokers use TLS on port 8883 and per-device credentials.

For production fleets, a hosted MQTT broker handles certificates, scaling and authentication so your team can focus on the models. With ConnectMQ each device gets credentials whose client ID is bound to it, and the live topic explorer in the portal shows desired and reported messages as a rollout progresses. For the broader context of combining models and connected devices, see our complete guide to AI and IoT.

Get Started with ConnectMQ

ConnectMQ is a managed MQTT broker supporting MQTT 3.1.1 and 5.0 over TCP, TLS and WebSockets, ready to carry your model update signals. Sign up for the free Serverless plan, no credit card needed, and wire up your first rollout today.

Frequently Asked Questions

Can I send the model file itself over MQTT?

You can chunk small files, but it is rarely worth it. Use MQTT for the update signal and status, and download the artifact over HTTPS where resumable downloads and caching are standard.

How does a device that was offline get the update?

Because the desired message is retained, the broker delivers it immediately when the device reconnects and subscribes. The device then compares versions and updates if needed.

How do I roll back a bad model?

Publish the previous version as the desired state. A device that keeps its last version on disk can switch back instantly without downloading anything.

Do microcontrollers use the same approach?

The signaling is the same, but the model is usually compiled into the firmware, so the device performs a firmware OTA update instead of swapping a model file.