The Ananse Reports: TinyML Reality Check, Part II: The Evidence

A brief recap for anyone starting here. I tested three MCU development boards (the STM32 NUCLEO-F401RE, the Arduino Nano 33 BLE Sense Rev2, and the ESP32-S3) on three standard TinyML workloads (keyword spotting, image classification, anomaly detection), and recorded board-level energy, latency, and accuracy. Part I ended with two surprises: the conventional embedded board came last on energy on two of the three tasks, and the Arduino board had the lowest energy use on every task, by margins large enough to affect a hardware decision.

This part shows the full result set. The matrix below summarizes the energy results, expressed as multiples of the most efficient board on each task.

What Figure 1 Tells Us

Figure 1 normalizes each workload against the board that used the least energy on that task. The Nano 33 is the 1x reference in all three rows. Every other value is that board's energy per 1,000 inferences divided by the Nano 33 result for the same workload.

For example, the F401RE's 8.0x keyword-spotting cell compares 21.4 mWh with the Nano 33's 2.7 mWh for the same 1,000 inferences. It does not mean that the F401RE is eight times slower, or that its model is eight times less accurate. The multiplier describes energy use only.

The rows should be read independently because their 1x cells represent different absolute amounts of energy. The 1x keyword-spotting result is 2.7 mWh, while the 1x anomaly-detection result is 0.19 mWh. What remains consistent is the board ranking: the Nano 33 used the least energy on every task, and the smallest measured gap was still 4.8x.

The rest of this part examines those ratios more closely. It covers the raw results, performance by task, observed variance, and why I reran the Nano 33 test.

The full results

What Figure 2 Tells Us

Figure 2 places the three energy measurements for each board on a logarithmic vertical scale. Each color represents one workload. Lower points indicate less energy per 1,000 inferences, and equal vertical distances represent multiplicative rather than additive changes.

The three Nano 33 points sit below the corresponding points for the other boards. The spacing also shows that workload choice changes the absolute energy budget substantially. On the Nano 33, anomaly detection uses 0.189 mWh per 1,000 inferences, keyword spotting uses 2.665 mWh, and image classification uses 14.185 mWh.

Only the first three board positions contain measured data. The Raspberry Pi, accelerator, laptop, and cloud labels to the right mark the hardware tiers planned for later reports. They are not estimates and should not be read as points on the current curve.

There are three interesting results in the table above.

First, latency changes by task. The ESP32-S3 is fastest on keyword spotting (106.5 ms) and image classification (551.1 ms), but the F401RE is fastest on anomaly detection (8.1 ms). Across the matrix, latency ranges from 8 milliseconds to 1.2 seconds, about 151x, so one benchmark number is not enough to summarize a board.

Second, the energy ranking is consistent: the Nano 33 is the lowest-energy board on every task, and it is never also the fastest board. Speed and efficiency point to different boards in this matrix. Part III focuses on what that means for hardware selection.

Third, the accuracy column needs one qualification. All three boards produce identical accuracy on the tasks where the full on-device accuracy evaluation ran: keyword spotting top-1 accuracy (the model's first-choice prediction is correct) is 0.883 on all three boards, and anomaly detection AUROC is 0.850 on all three. That result is expected. Each board runs the same compiled INT8 model with deterministic inference on the same test set, and the matching scores confirm that the comparison is like for like. At this tier, the hardware choice changed the cost of running the model, in time and in energy. It does not change how well the model works. Image classification ran as a timing and energy workload only, so the identical-accuracy claim is scoped to the two tasks where accuracy was evaluated on device.

Task by task

Keyword spotting is the always-on workload, and the clearest example of latency and energy pointing in different directions. The ESP32-S3 finishes an inference in 106.5 ms; the Nano 33 takes more than twice as long at 224.2 ms. If latency is the only metric, the ESP32-S3 looks like the right choice. However, once energy is included, the ranking changes: the Nano 33 completes a thousand inferences on 2.7 milliwatt-hours, one fifth of the ESP32-S3's 13.6 and one eighth of the F401RE's 21.4.

The repeatability of those runs is part of the evidence. Across three published sessions per board, keyword spotting latency variance was 0.02 percent or lower on every board. The sample is small, but the measurements are consistent. These boards are running deterministic code, and the repeated runs reflect that.

Image classification is the memory-pressure workload. It fits on all three boards, but on the F401RE it is the tightest case in the matrix: the firmware's estimated working memory comes to 74,884 bytes against the board's 98,304-byte SRAM limit, a margin of 23,420 bytes. Part III explains why model file size, the number that stands out most on a spec sheet, was never the real constraint anywhere in this matrix. On energy, the pattern repeats at larger magnitudes: the Nano 33 finishes a thousand classifications on 14.2 milliwatt-hours; the other two boards need roughly five times as much.

Anomaly detection is the lightweight workload: an autoencoder scoring 640-value sound features. It has the lowest energy numbers in the matrix. Every board completes a thousand inferences for under 1.5 milliwatt-hours. It also has the one latency inversion: the F401RE is fastest here, at 8.1 ms. The anomaly-detection data also exposed a measurement issue worth documenting. An earlier version of this dataset showed an alarming spread in the F401RE's energy numbers. The investigation traced it to sessions measured at two different power boundaries, one of which had gone unrecorded, and I remeasured the affected cells from scratch under a documented boundary across three separate days. With the boundary controlled, the measured spread is about 1 percent; the F401RE cells now carry the widest energy variance in the matrix at roughly that level, and every other cell is at or below 0.1 percent.

Repeatability across sessions

What Figure 3 Tells Us

Figure 3 compares three independently powered sessions of the same Nano 33 keyword-spotting workload. Each horizontal line spans the minimum to maximum latency observed within one session, and its point marks that session's median.

The horizontal axis runs from 224.0 to 224.3 ms rather than starting at zero. This deliberately magnifies a small spread so the three sessions can be distinguished. Across all of the visible ranges, the lowest observation is 224.094 ms and the highest is 224.252 ms, a difference of 0.158 ms.

The session medians differ by less than 0.1 ms, with a run-to-run standard deviation of 0.053 ms, or 0.02 percent. The chart therefore supports a narrow conclusion: latency for this board and workload was stable across these three sessions. It does not establish that every board, workload, firmware build, or environmental condition will have the same variance.

This is representative of the latency stability: three sessions on different days, and the medians are within 0.1 milliseconds of each other, a run-to-run standard deviation of 0.053 ms, or 0.02 percent. A session here is a fresh power-up of the full rig on its own day, running five warmup and twenty measured runs. Every reported number in this report comes from sessions with complete telemetry: wall-side power, board-side current, and ambient conditions recorded throughout the run. Runs that logged incomplete telemetry remain in the repository's raw data for provenance but are excluded from every table in this series.

The 43 milliwatt result

I checked the Nano 33's keyword-spotting energy result again before trusting it. The board used 9.6 millijoules over a 224-millisecond inference window, equivalent to average board-level power of roughly 43 milliwatts while running inferences back to back. This is not an instantaneous peak or a full application duty-cycle average.

In Realtime Person Identification via Gait Analysis, Venkatachalam et al. report 125 milliwatts for continuous inference on an Arduino Nano 33 BLE Sense. Their experiment uses a different model and workload. Moreover, the preprint does not identify the board as the Rev2 or provide enough information about the measurement boundary to support a direct comparison. The gap prompted an investigation; it does not establish that my workload is three times more efficient.

I reproduced the result across sessions and cross-checked it using two independent meters at different points in the power path. The instruments agreed within six percent. The result held. Report No. 1.5 examines the effects of radio initialization, regulator configuration, and measurement window, and explains what can and cannot be concluded from the published reference.

Part III asks what these nine cells change in a hardware decision. It covers model memory and the split between speed and efficiency. Every number above traces to a run ID in the public signal-bench repository, with the methodology available for audit and reproduction.