I Stopped Eyeballing Serial Logs and Gave My ESP32 Boards a Health Score Instead
Validating an ESP32 board by opening a serial monitor and reading the boot log works exactly once. It does not repeat, it does not scale to the next board on the bench, and when a board fails it leaves no evidence you can hand to anyone. I know because board validation was the second half of a longer fight. The first half was three boards that flashed and verified perfectly and never ran a line of application code, a fault I eventually isolated to a bounded domain in the boot chain (I wrote that investigation up separately). The lesson that stuck was not really about those boards. It was that manual checking produces opinions, and what I wanted was the same run, the same evidence, and the same report every time. So I built a framework to do the checking for me. How it works There are two halves. On the board, an Arduino test firmware runs seven tests and reports structured JSON records over a small serial protocol: Serial communication Heap behaviour Chip information A
Validating an ESP32 board by opening a serial monitor and reading the boot log works exactly once. It does not repeat, it does not scale to the next board on the bench, and when a board fails it leaves no evidence you can hand to anyone.
I know because board validation was the second half of a longer fight. The first half was three boards that flashed and verified perfectly and never ran a line of application code, a fault I eventually isolated to a bounded domain in the boot chain (I wrote that investigation up separately). The lesson that stuck was not really about those boards. It was that manual checking produces opinions, and what I wanted was the same run, the same evidence, and the same report every time. So I built a framework to do the checking for me.
How it works
There are two halves.
On the board, an Arduino test firmware runs seven tests and reports structured JSON records over a small serial protocol:
- Serial communication
- Heap behaviour
- Chip information
- A structured board inventory
- Wi-Fi scan
- NVS write and read integrity
- GPIO output
On the host, a Python runner does the rest:
- It discovers the attached board, and refuses to guess when several are attached.
- It triggers the test suite and validates every record the board sends against the run summary, so a truncated or garbled record fails loudly instead of passing silently.
- It analyzes the boot log, classifying what it finds with a severity, the likely causes, and recommended actions.
- It runs esptool diagnostics: chip ID, flash ID, and flash status.
- It scores the board's health from 0 to 100, using weights that are published in the guide rather than hidden in the code.
- It writes timestamped JSON and Markdown reports, with the raw serial log preserved, so any result can be re-examined later.
There is also a 13-test offline regression suite with mocked serial and esptool, so changes to the host code can be verified with no hardware attached at all. It passes as shipped.
The board that scored 15 out of 100
The framework earned its keep on the first real board I pointed it at: an ESP32-D0WD-V3 with 4 MB of flash.
The run came back with a health score of 15 out of 100. The interesting part was the split. Every flash diagnostic passed. Chip ID, flash ID, flash status, all clean. But the application on the board was dead, and the boot log analysis classified the boot as an invalid image.
A board that passes its flash checks and still cannot run is exactly the pattern from my earlier investigation, so the next step was the eFuse audit. The board's flash encryption state was enabled, and the images being written to it were plaintext. A mismatch between the board's security state and the form of the image is a configuration fault, not a hardware death sentence.
The recovery was to write the board's boot chain in encrypted form to match its recorded security state. The procedure writes no eFuses and does not enable encryption; it works with the state the board already has. After the recovery, the same framework ran the same suite on the same board and scored it 100 out of 100, with all seven firmware tests and all three flash diagnostics passing.
That pair of reports, 15 and 100, from the same board and the same tool, is the whole argument for automated validation. Nothing about the board was assessed by feel. The evidence before and after sits in two timestamped reports that anyone can read.
The honest limits
A scoring framework invites overconfidence, so the limits matter as much as the features:
- The framework was exercised on one ESP32-D0WD-V3 board with 4 MB flash. The included flash profile (DOUT, 40 MHz, custom partitions) must be verified for other boards before you trust a run.
- The health score is a diagnostic heuristic, not a certification. It tells you where to look next, not whether a board is "good" in any absolute sense.
- The GPIO test confirms software toggling, not electrical soundness. A pin can toggle in software and still be wrong on the wire.
- The Wi-Fi scan result is environment-dependent. A quiet RF environment and a faulty radio can look similar from one scan.
- The recovery described above fixed one board with one specific failure mode. It is a documented case study, not a general fix, and it has not been replicated across a batch.
Why bother, for a handful of boards
If you validate one board once, a serial monitor is fine. The moment you validate the same board twice, or a second board ever, the framework pays for itself: identical procedure, comparable numbers, and a paper trail. That is true on a hobby bench and it is doubly true anywhere a result has to survive a reviewer's questions.
The framework, the guide, the test plan and results templates, and the full worked example are packaged here: ESP32 Automated Validation Framework ($29). If your problem is earlier in the chain, a board that will not boot at all, start with the free ESP32 Bring-Up Checklist or the debugging pack built on the three-board investigation. The framework code and investigation live on my GitHub: esp32-automated-test.
How do you validate boards on your bench? I am genuinely interested in what other people score, check, or simply refuse to trust.