How to Diagnose and Fix Common Embedded Development Issues

Many people claim embedded development is hard, but the difficulty mainly comes from encountering various obscure issues. This article breaks down common problems and offers solutions.

1. Problem Reproduction

Stable reproduction is essential for locating and solving issues. The easier a problem can be reproduced, the easier it is to fix.

1.1 Simulate Reproduction Conditions

Some issues occur only under specific conditions; simulate those conditions or preset program states to reproduce them.

1.2 Increase Task Execution Frequency

If an exception appears after a long-running task, increase its execution frequency.

1.3 Expand Test Sample Size

For long‑running programs that rarely crash, use multiple devices simultaneously to increase test coverage.

2. Problem Localization

Narrow the investigation scope to the responsible task, function, or statement.

2.1 Print LOG

Add LOG output at suspicious code points to trace execution flow and variable values.

2.2 Online Debugging

Online debugging works like LOG printing and is especially useful for crash‑type bugs, allowing inspection of the call stack and registers.

2.3 Version Rollback

Use version control to revert versions step by step to pinpoint the introduction of the bug.

2.4 Binary Search Commenting

Binary search commenting

means commenting out half of the code to see if the problem disappears, then iteratively narrowing the suspect region.

2.5 Save Kernel Register Snapshot

When a Cortex‑M core enters a HardFault, several registers are pushed onto the stack. By saving these registers to a reserved RAM area before reset, you can analyze PC, LR, R0‑R3, and SP after reboot to identify the fault cause.

3. Problem Analysis and Handling

Combine the observed symptoms with the located code to analyze root causes.

3.1 Program Continuation Issues

3.1.1 Numerical Errors

1. Array Out‑of‑Bounds – Writing beyond array limits corrupts memory. Use the map file to locate affected variables and fix unsafe code.

2. Stack Overflow – Stack grows from high to low addresses; overflow can overwrite variables such as g_val. Analyze maximum stack usage, reduce call depth, or increase stack size.

Allocate appropriate stack size during design.

Convert large temporary variables to static or heap allocation.

Reduce function call depth.

3. Incorrect Conditional Statements – Accidentally using “=” instead of “==” changes variable values and always evaluates true. Write the variable on the right side of the operator to catch this at compile time, or use static analysis tools.

4. Synchronization Issues – Queue operations interrupted by ISR can corrupt the structure; protect critical sections with mutexes or disable interrupts.

5. Optimization Issues – Compiler may keep a flag in a register, ignoring changes in RAM. Declare such flags as volatile to force RAM reads.

3.1.2 Hardware Issues

1. Chip Bugs – Some chips return erroneous values under certain conditions; filter out abnormal readings in software.

2. Communication Timing Errors – For example, cascading power‑management chips require strict timing; missing a read window leads to data loss.

3.2 Program Crashes

3.2.1 HardFault

Common causes include accessing peripheral registers before enabling their clocks, jumping to an out‑of‑range function pointer, or misaligned pointer dereference.

For misaligned accesses, use memcpy instead of direct pointer casts.

3.2.2 Interrupt Service Routine Issues

Failing to clear interrupt flags can cause immediate re‑entry, leading to a “pseudo‑dead” state.

Non‑maskable interrupt (NMI) conflicts, such as an SPI MISO pin multiplexed with NMI, may require disabling NMI inside its handler.

3.2.3 Hardware Failures

Examples: crystal oscillator not starting, insufficient supply voltage, or reset pin held low.

4. Regression Testing

After fixing a problem, perform regression tests to confirm the issue no longer reproduces and that the changes have not introduced new bugs.

5. Experience Summary

Document the root causes and solutions, reflect on preventive measures, and apply lessons learned to similar platforms.