Quantization converts the numbers inside a trained neural network to a smaller representation. Integer arithmetic is faster and cheaper on microcontrollers and NPUs, and 8-bit weights take a quarter of the space of 32-bit floats. This is often the single step that makes a model fit on an embedded target at all.
The cost is a small loss of accuracy, which can be reduced by quantization-aware training or by calibrating with representative data. Some layers are more sensitive than others, so tools allow mixed precision. In TinyML projects quantization is almost always combined with pruning. The guide TinyML: machine learning on microcontrollers shows where it sits in the toolchain.
Part of the IoT Glossary. Spotted an error or a missing term? Tell us.