Why Do Spinlocks Behave Differently on UP vs SMP Linux Kernels?

Background and Observation

The author attempted to put a sleeping task inside a spinlock on an ARMv7 single‑core CPU (TI A8) and found that the lock never dead‑locked and could be acquired repeatedly, contrary to expectations. The same module dead‑locked on a PC, and on another ARM board (S3C6410) it also dead‑locked, while on OMAP3530 it behaved like the first ARM board. Adding the kernel configuration option CONFIG_DEBUG_SPINLOCK to the UP kernels made the lock dead‑lock as expected, revealing that this debug option changes spinlock implementation.

Origin and Purpose of Spinlocks

Spinlocks were created for SMP (Symmetric Multiprocessing) systems to protect short critical sections by busy‑waiting until the lock becomes free. In SMP kernels the implementation includes full spinning, interrupt disabling, and preemption control. In UP (Uniprocessor) kernels the lock is trimmed: it only disables interrupts and preemption without a real busy‑wait loop. Enabling CONFIG_DEBUG_SPINLOCK forces the UP kernel to compile the SMP‑style spinlock code.

What a Spinlock Must Do

Interrupt control : disable hardware and software interrupts when the critical section may be accessed from interrupt context.

Preempt control : disable kernel preemption (via preempt_disable() / preempt_enable()) to avoid being rescheduled inside the critical region.

Lock‑flag control (SMP only) : use an atomic flag (implemented with LDREX / STREX on ARM) to detect contention across CPUs.

Why UP Kernels Without Debug Spinlock Appear to Allow Multiple Acquisitions

Because the UP implementation omits the spinning part, acquiring the lock merely disables interrupts and preemption. If the same CPU re‑enters the lock before any interrupt occurs, the lock appears to be acquired multiple times, which is technically incorrect but harmless in this trimmed mode.

Common Questions

1. When interrupts are already disabled, is disabling preemption redundant? Not entirely—preempt_disable() also prevents voluntary rescheduling that could occur even without an interrupt.

2. In SMP systems, disabling preemption also prevents other CPUs from scheduling the current task on a different CPU, which is why the full spinlock includes the atomic flag check.

Spinlock Variants and When to Use Them

Different kernel APIs exist to match the required level of interrupt and preemption control:

static inline void spin_lock(spinlock_t *lock)

static inline void spin_unlock(spinlock_t *lock)

Use when the critical section is never accessed from any interrupt context.

static inline void spin_lock_bh(spinlock_t *lock)

static inline void spin_unlock_bh(spinlock_t *lock)

Use when the section may be accessed from software (bottom‑half) interrupts such as tasklets or timers.

static inline void spin_lock_irq(spinlock_t *lock)

static inline void spin_unlock_irq(spinlock_t *lock)

Use when the section may be accessed from hardware interrupt context.

spin_lock_irqsave(lock, flags)

static inline void spin_unlock_irqrestore(spinlock_t *lock, unsigned long flags)

Use when the code must save and later restore the interrupt state, ensuring the original interrupt enable/disable status is restored after the critical region.

Generally, spin_lock_irq is faster than spin_lock_irqsave if the caller knows interrupts are already enabled; otherwise, the save/restore variant is safer.

Conclusion

The behavior differences observed on various ARM boards stem from whether the kernel was compiled with CONFIG_DEBUG_SPINLOCK, which forces SMP‑style spinning even on UP systems. Understanding the three responsibilities of a spinlock—interrupt control, preempt control, and SMP lock‑flag handling—helps developers choose the correct spinlock variant for their specific concurrency scenario.