Verification Engineering: From Vibe Hardware Coding to Self‑Iterating Validation

Initial Setup

ESP32‑S3 development board with a 1.75" AMOLED screen was connected to the Arduino IDE and the official Hello World example ran successfully.

AI‑Driven Hardware Coding

A GitHub repository was created at https://github.com/phodal/vibe-hardware-coding. The AI was prompted (in Chinese) to set up a complete automated development environment using arduino-cli and the Waveshare SDK.

我买了个Arduino开发板：https://docs.waveshare.net/ESP32-S3-Touch-AMOLED-1.75C/，帮我结合他们的 SKILL，帮我搭建完整的自动化开发环境吧：arduino/arduino-cli (https://github.com/arduino/arduino-cli) 然后创建 skill 等，确保应用能在硬件上跑起来（已经在 Arduino IDE 可以连接上串口了） ## 验证 最后你可以使用 claude-p 来测试这个 SKILL， 是不是能把代码正确烧录进去

The AI completed the task in about 40 minutes, installing arduino-cli 1.5.1, fixing the ESP32 core to esp32:[email protected] (matching the Waveshare Arduino‑v3.3.5 example package), and configuring the development environment.

arduino-cli version 1.5.1

ESP32 core fixed to esp32:[email protected] It also solved an arduino-cli monitor issue where the USB Serial/JTAG port opened but captured no data. The script was changed to use macOS raw serial via stty + cat. To force the original Arduino monitor back, set the environment variable ARDUINO_CLI_MONITOR=1.

OCR Visual Verification

A Logitech webcam was used to capture the screen. The AI was asked to recognize the text displayed on the screen.

感觉写的 demo 太粗糙了，还花屏了，有没有可能调用我的摄像头，来识别上面的文字，以确保最后能在屏幕上正确显示？？？

After several iterations the AMOLED screen lit up showing two lines: OK and Qoder. The camera view showed the board, sufficient brightness, and a roughly correct crop. The build succeeded, firmware was uploaded, and the screen displayed the expected content.

OCR output was unstable. A log file contained: { OK<n } Later the OCR read Qoder as Goder.

Limitations of Traditional Verification

Build success, upload success, and serial output only prove that the software chain reached the device edge. Real output resides on the screen, camera, lighting, font, and human eye. Without feeding these observations back into the engineering system, automation stops halfway.

Verification Engineering

Verification engineering organizes goal, test, verifiable evidence, and memory into a continuously converging loop.

Evidence hierarchy:

Build pass → toolchain and syntax are valid in the current context.

Upload success → firmware can reach the device.

Serial output → firmware believes it has reached a certain state.

Camera + OCR → real‑world output matches the expected token.

Breaking Goals into Verifiable Propositions

Example goal “self‑hosted AI terminal is usable” is decomposed into small propositions: device boot, screen display, host‑relay communication, camera observes a stable token, microphone metrics respond to sound, VAD triggers, ASR returns text, TTS plays audio. Each proposition proves a single fact; their combination gradually approaches the overall goal.

Evidence System

Each proposition needs a validator that records observation method, validation context, evidence structure, and decision logic.

Visual validation: raw camera image → crop/rotate/enhance → OCR text → expected token → match result → conclusion.

Audio validation: sampled window → RMS/peak/delta → statements such as “microphone stream valid”.

Serial validation: stable connection and log capture.

Document validation: rendered environment → screenshot → visual diff.

When OCR misread Qoder as Goder, preserving the original image, processed image, OCR text, and crop parameters enables diagnosis of whether the fault lies in the screen code or the visual validator.

Self‑Iterating Verification

Verification conclusions must modify the next iteration. Verified propositions become regression checks; unverified propositions drive further implementation; insufficient evidence triggers additional observation; unstable validators are repaired first; manual review is required when automation cannot cover a judgment.

If failures leave no consumable state, the next agent must guess from scratch; if successes leave no boundary, the agent may redundantly re‑prove the same thing. Writing back experience turns execution history into capability.

Conclusion

The endpoint of verification engineering is not merely “verification passed” but “verification made the next verification more reliable”. A true closed loop occurs when each verification influences the next verification method, allowing AI‑assisted coding to evolve from code generation to systematic engineering.