TinyML is machine learning that runs on microcontrollers: chips with no operating system in the desktop sense, a few hundred kilobytes of RAM at most, a clock in the tens or low hundreds of megahertz, and a power budget small enough to run for months or years on a coin cell. It is the bottom tier of edge AI, and the tier with the most devices in the world.
The appeal is that a microcontroller is already present in almost every IoT device. If the model can fit in the memory left over after the firmware, the device gains the ability to classify what its sensors see without a radio transmission, which is usually the most expensive thing a battery-powered node does. A vibration sensor that sends one byte, “normal” or “abnormal”, once an hour instead of streaming raw samples can last years instead of days.
This guide explains what the constraints really are, how a TinyML model is built end to end, which frameworks are in common use, and which workloads fit well and which do not.
The constraints, in order of pain
Memory is the first wall. A typical Cortex-M4 or ESP32 part has a few hundred kilobytes of RAM and one or two megabytes of flash memory. The model’s weights live in flash; its activations, the intermediate results during a single inference, live in RAM. A model that works on a phone with a 20 MB weight file is simply not a candidate. Practical TinyML models range from a few kilobytes to a few hundred kilobytes.
Compute is the second. Without a floating-point unit, or with a slow one, integer arithmetic is the only realistic option. An inference that takes a few milliseconds on a microcontroller might involve a few hundred thousand multiply-accumulate operations, which bounds the depth and width of the network.
Power is the third, and in battery products it dominates. Every millisecond the core spends awake costs energy, so the model must be fast enough that the device can return to deep sleep quickly. The duty cycle of inference, how often the model runs, is often the main tuning knob for battery life.
There is a fourth constraint that is easy to forget: there is no Python on the device. The model must be converted into a static array of bytes compiled into the firmware, and the runtime must be written in C or C++ with no dynamic memory allocation.
The workflow
A TinyML project follows a fixed sequence, and most of the time goes into the first step.
Collect data from the actual sensor on the actual hardware, in the conditions the product will see. A model trained on vibration from a lab rig will not transfer to a motor bolted to a different frame. The data has to be labeled, which for anomaly detection may just mean “recorded during normal operation”.
Design features and a model. Raw sensor data is rarely fed straight into a network. Audio becomes a spectrogram or a set of MFCC coefficients; vibration becomes an FFT or statistical features over a window. A small convolutional or fully connected network then classifies the feature frame. Feature extraction is deterministic signal processing and often costs more compute than the network itself.
Train in the cloud or on a laptop, using the same frameworks as any other project. Then quantize to 8-bit integers. Quantization is not optional here; it is what makes the model fit and what lets it run on integer hardware.
Convert the model to a flat file, embed it in firmware as a byte array, and link it with a runtime. Then measure on the target: inference time, peak RAM, and accuracy on held-out recordings made on the device. Numbers from the training machine do not count.
Frameworks and runtimes
TensorFlow Lite for Microcontrollers, now part of the LiteRT family, is the reference runtime. It is a C++ interpreter that reads a .tflite model, works without dynamic allocation or standard library dependencies, and executes a configurable subset of operators. The developer allocates a fixed “tensor arena” buffer for activations and registers only the operators the model needs, which keeps the binary small. On Arm cores it uses the CMSIS-NN kernels, which are hand-optimized for the Cortex-M instruction set.
Edge Impulse is a hosted platform that wraps the entire workflow: data ingestion from phones and dev boards, labeling, feature design with a visual DSP block, training, quantization, on-target profiling and export as a C++ library or an Arduino library. It is the most common entry point for teams that have embedded experience but no ML specialist.
Arm’s Ethos-U NPUs extend the microcontroller tier with a small neural processing unit next to a Cortex-M core. The Vela compiler splits a quantized model into the parts the NPU can run and the parts that fall back to the CPU. Several vendors ship Cortex-M55 or Cortex-M85 parts paired with an Ethos-U core, which pushes always-on vision and audio into the milliwatt range.
Other options include microTVM, which compiles models ahead of time rather than interpreting them, and vendor-specific toolchains such as STMicroelectronics’ STM32Cube.AI and the NXP eIQ suite, which generate optimized code for their own parts.
Workloads that fit
Keyword spotting is the canonical TinyML task: listen continuously for a wake word and wake a larger system when it is heard. The model is small, the input is a short spectrogram window, and the latency requirement is loose.
Anomaly detection on vibration or current from an accelerometer or current clamp is the most common industrial use, and the basis of many condition monitoring sensors. The predictive maintenance guide goes into the signal processing.
Gesture and activity recognition from an IMU powers wearables and handheld tools, classifying motion windows into a handful of categories.
Low-resolution vision, for example person detection on a 96 by 96 grayscale frame, is feasible on the larger microcontrollers and on anything with an Ethos-U. It is good enough to answer “is someone there” but not to read a license plate. The on-device vision guide explains where the microcontroller tier stops.
Sensor cascades combine several of these: a cheap always-on model decides whether to wake a more expensive one, which decides whether to wake the radio. Each stage filters for the next, and the power budget is dominated by the cheapest stage.
Workloads that do not fit
Anything with a large vocabulary of outputs, high-resolution imagery, long temporal context or a need for frequent model updates is a poor match. Language models of any useful size are out of reach on this tier. So is any task where the training data cannot be collected on the real device, because a TinyML model has no spare capacity to generalize across hardware it never saw.
TinyML also adds a maintenance obligation. The model is firmware, so updating it means an OTA update, and monitoring it means shipping at least a trickle of raw samples upstream so that model drift can be detected. A device that can only send one byte an hour cannot tell you when it has started being wrong.
Where to go next
- Edge AI explained covers the larger hardware tiers above the microcontroller.
- Predictive maintenance with IoT and AI applies TinyML-style signal processing to machines.
- Types of IoT devices and the constraints that shape them is the hardware background.
- Device management and over-the-air updates explains how to ship a new model to a fleet.
Topics: TinyML, microcontrollers, TensorFlow Lite, Edge Impulse