Linux Kernel Memory Management Locks and Optimization Case Studies

This article provides a comprehensive technical analysis of locks in Linux kernel memory management, based on Linux kernel 6.9. It covers six typical locks used in memory management and summarizes optimization case studies from the Linux open-source community.

1. Technical Background

Locks play a crucial role in Linux memory management. They protect critical sections for multi-threaded concurrent processing, but can also cause performance issues in certain scenarios.

2. Memory Management Locks

PG_Locked: Used for page-level locking in physical memory management. When a page is marked with PG_locked, it indicates the page is in use and should not be modified. The lock_page interface may sleep, so it cannot be used in non-sleepable contexts.

lru_lock: Protects LRU (Last Recently Used)链表 operations during memory reclamation. The lruvec structure contains a spinlock to protect LRU list operations.

mmap_lock: A read-write semaphore in mm_struct that protects process virtual address space (vma). It guards the vma rbtree, vma list, and vma flags. Although it's a read-write lock, it can behave like a mutex under certain conditions.

anon_vma->rwsem: A read-write semaphore for anonymous page reverse mapping. Used during memory reclamation to find all VMAs mapped to a page.

mapping->i_mmap_rwsem: Protects file page reverse mapping through the address_space structure's i_mmap priority search tree.

shrinker_rwsem: A global read-write semaphore (changed to mutex in 6.9) that protects the shrinker_list and shrinker registration/unregistration operations.

3. Optimization Case Studies

Eight optimization cases are presented from the Linux community:

1) Per-memcg lru_lock optimization: Introduced per-memcg locks to reduce contention, achieving 62% performance improvement.

2) mmap_lock IO fault path optimization: Release mmap_lock during IO operations to avoid blocking.

3) SPF (Speculative Page Fault) optimization: Attempt page fault handling without acquiring mmap_lock when VMA remains unchanged.

4) PVL (Per-VMA Lock) optimization: Replace global mmap_lock with per-VMA locks for better concurrency.

5) Fault around optimization: Pre-map adjacent pages during page faults to reduce mmap_lock contention.

6) munmap optimization: Segment large unmaps to release mmap_lock periodically.

7) rmap path lock optimization: Use trylock to skip pages with lock contention during memory reclamation.

8) Shrinker lockless optimization: Replace global shrinker_rwsem with reference counting and RCU.

4. Optimization Evolution Directions

1) Lockless operations: Eliminate locks where possible (e.g., shrinker lockless, SPF)

2) Reduce critical section size: Release locks early when possible

3) Split large locks into smaller ones: Reduce lock contention granularity (e.g., PVL, per-memcg lru_lock)

4) Reduce lock waiting impact: Use trylock to avoid blocking critical paths

5) Reduce lock acquisition frequency: Use techniques like fault-around to minimize lock operations