The Ananse Reports: TinyML Reality Check, Part I: The Floor of the Curve

Three small development boards are lined up on my desk. Together they cost less than a mid-range smartphone. Each one is built around a microcontroller, the kind of processor used to control thermostats, factory sensors, household appliances, and many other embedded systems.

For the past several weeks I have been asking the same question about them again and again: in energy, not dollars, what does it cost to run AI on each board?

Speed numbers for boards like these are easy to find. In contrast, energy numbers are harder. MLPerf Tiny, the industry's standard benchmark for this class of hardware, scores devices primarily on latency; its energy results are a separate standardized measurement made on specialized equipment along an isolated current path that strips away the rest of the board. Vendor datasheets quote current draw for the chip, not for the board you actually deploy. The practical question for a battery-powered product is simpler: which board drains the battery slowest while running my model? As far as I can find, there is no public apples-to-apples answer for these boards. Fragments exist: a power figure for one board here, a latency table there, coarse comparisons across different boards on different workloads. Nothing puts these three boards on the same model, under the same protocol, behind the same measurement boundary.

I ran each model on each board using one rig and one protocol. Two results surprised me.

The first surprise: the STM32 NUCLEO-F401RE, the most conventional embedded development board in the lineup, used the most energy per inference on keyword spotting. It used eight times more than the lowest-energy board in the test.

The second surprise: the winner was an Arduino. The Nano 33 BLE Sense Rev2 is marketed as a beginner-friendly sensor board. Nothing about its positioning suggests an AI workhorse. Yet on every task in this report it consumed a fraction of the energy of its competitors. On the two tasks where I ran the full on-device accuracy evaluation, it matched the other boards exactly, score for score. Its measured average board-level power during back-to-back inference was roughly one-third of a published figure for the same board under a different workload. The gap led me to verify the measurement using a second, independent power path. For context, the first path used an INA219 wired inline on the board's 5 V input. The second used an FNB58 USB meter at the wall. The result held, although I did not yet know what accounted for the difference. That investigation gets its own report in this series.

Here is the result translated into the terms that matter for a battery-powered product.

What Figure 1 Tells Us

Figure 1 translates the measured energy per inference into a common battery-sized budget. It asks how many inferences each board's measured compute cost would represent within the nominal 675 mWh capacity of a CR2032 coin cell.

Consider the Nano 33 anomaly-detection result. The board used approximately 0.19 mWh per 1,000 inferences. Dividing 675 mWh by that rate gives approximately 3.55 million inferences, which is rounded to 3.6 million in the chart.

That value is not a prediction that a finished Nano 33 product will perform exactly 3.6 million anomaly detections on one coin cell. The calculation covers the measured compute path. A deployed product must also power its sensors, radio, regulator, and other components. Its duty cycle, operating temperature, and the battery's behavior under load will further affect the result.

The bars should primarily be compared within each workload. A taller bar means that the board used less energy per inference and could therefore perform more of that same workload within the common energy budget. On keyword spotting, the Nano 33 reaches an estimated 253,000 inferences, compared with 50,000 for the ESP32-S3 and 32,000 for the F401RE. On image classification, it reaches 48,000, compared with 9,800 and 9,400. On anomaly detection, it reaches 3.6 million, compared with 469,000 and 521,000.

Consequently, the Nano 33 converts the same nominal energy budget into approximately five to eight times as many inferences, depending on the workload. The comparison between boards is the result to carry forward. The absolute counts remain calculated energy equivalents rather than battery-life predictions.

What an Ananse Report includes

An Ananse Report publishes hardware-grounded numbers with disclosed variance and enough methodology to reproduce them from the public repository. The analysis explains what those measurements change for someone choosing hardware. For this report, that means wall-side power from a USB power meter, board-side current from an inline current sensor, ambient conditions from an environmental sensor, and inference timing reported as distributions rather than single polished numbers.

Methodology hardware

Measurement rig

Select a frame, or open the image to inspect markings.

Full rig

Bench-wide view of the rig used to capture board-level energy and telemetry.

Full rig Powered scale Detail, INA219 I Detail, INA219 II Detail, MCP2221A Detail, BME280

The existing ecosystem deserves credit for getting TinyML this far. Edge Impulse made the machine learning deployment workflow practical for teams that do not want to rebuild an embedded pipeline from scratch. MLPerf Tiny gave the field canonical workloads, reference models, and a shared vocabulary. This report stands on that work. Those tools are not wrong. The missing piece is narrower: a team deciding whether to put inference on a microcontroller still needs board-level energy, repeatability, and environmental context in the same frame as latency and accuracy. A benchmark can be technically correct and still incomplete for someone who has to choose a board, a power budget, and a deployment envelope.

I built signal-bench, the open-source tool behind this series, to make that pipeline repeatable: define the tasks, run the measurements, capture telemetry, export the matrix, and generate the charts without hand-massaging the result. Repeatability matters because trust comes from rerunning the work, and it lets other builders challenge the conclusion without reverse-engineering the setup. signal-bench keeps noisy results visible, including cases where efficiency trades against speed or a board changes rank when the metric changes. That tradeoff is why both measurements belong in the report.

What was tested

There are three tasks and three boards. That makes nine combinations, and each combination was measured in at least three separate sessions on different days.

The tasks come from MLPerf Tiny, so the models and datasets are standard and recognizable: keyword spotting, image classification, and anomaly detection. In practical terms, that means an always-on audio workload, a compact vision model, and an autoencoder listening for a machine going wrong. The boards represent three different answers to "what kind of MCU should run AI": the STM32 NUCLEO-F401RE as the conventional embedded development board, the Arduino Nano 33 BLE Sense Rev2 as the accessible sensor board, and the ESP32-S3 as the connected MCU marketed for AI workloads.

Microcontroller under test

STM32 NUCLEO-F401RE

Select a frame, or open the image to inspect markings.

Perspective

Default board view for the classic embedded development board.

Perspective Top Top, powered Scale, length Scale, width Detail, STM32 Detail, ST-LINK Detail, headers

Microcontroller under test

Arduino Nano 33 BLE Sense Rev2

Select a frame, or open the image to inspect markings.

Perspective

Default board view for the lowest-energy MCU in Report No. 1.

Perspective Top, powered Scale, length Scale, width Detail, nRF52840

Microcontroller under test

ESP32-S3 DevKitC

Select a frame, or open the image to inspect markings.

Perspective

Default board view for the connected MCU in the Report No. 1 matrix.

Perspective Powered perspective Scale, length Scale, width Detail, ESP32-S3 module

Each result reports median latency and median energy per 1,000 inferences from the published session set.

The metric

The headline metric is watt-hours per 1,000 inferences: how much energy the board consumed to run the model a thousand times. Latency answers how fast. Energy answers at what cost. The unit is simple: energy per inference, scaled to a thousand inferences and converted to watt-hours for readability. The measurement boundary is the whole development board, USB bridge and regulators included, because that is the hardware you actually buy and power.

That boundary matters when comparing against other published numbers. MLPerf Tiny's official energy results are measured on an isolated current path that feeds the microcontroller alone, with the rest of the board stripped out of the measurement. Those numbers and mine describe different physical systems, and neither is wrong. That difference becomes important in the next two reports: Part II shows the Nano 33 result clearly, and Report No. 1.5 explains why it looked so far from a published reference figure.

Where this is going

Part II presents the evidence: the full task-by-task results, the accuracy scores, the run-to-run variance, and why the Nano 33's number needed a second check. Part III turns the data into decisions: when to optimize for speed, when to optimize for energy, and why the intuitive way of reading a model card can lead you wrong.

Every number in this series traces to a run ID in the signal-bench repository, which is public as of today. The methodology is published so the numbers can be checked, challenged, and rerun.