Why Oracle on AIX Restarts: Diagnosing the numperm_global Early Paging Bug

Fault Description

Oracle database on AIX unexpectedly restarted.

System Environment

AIX version 6100-07-05-1228, Oracle 11g.

Fault Analysis

DBA logs showed two control processes hung at 20:06, prompting Oracle's self‑restart. OS metrics from nmon indicated low CPU usage (<30%) and memory usage around 80% of the 90 GB system, but a massive paging event occurred at 20:06 despite about 12 GB of free memory.

vmstat displayed 3,103,118 free pages (≈12 GB) while 746 pages were pinned (pi) and 16,989 were active (po), confirming paging under sufficient free memory.

AIX divides memory into pools; the numperm_global parameter determines whether paging decisions are made per‑pool (0) or globally (1). Certain AIX 6.1 releases contain a bug when numperm_global=1: if pinned 64 KB pages reach maxpin% (80 %), the pager swaps out compute memory even though file‑system cache is available, degrading performance and potentially hanging applications.

Oracle’s SGA and PGA are typically pinned in 64 KB pages. When their combined size approaches maxpin%, the bug is triggered, causing “early paging” and the observed performance drop that led to the restart.

Inspection confirmed numperm_global was set to 1, so the bug was active.

Recommended Solutions

Solution 1: Change numperm_global to 0 and reboot the system.

Solution 2: Apply IBM AIX 6.1 TL09 SP01 patch IV44998, then keep numperm_global =1 for optimized memory allocation.

Solution 3: Configure Oracle SGA to use 16 KB pages, avoiding large numbers of pinned 64 KB pages.

Solution 4: Set Oracle LOCK_SGA parameter to FALSE to prevent pinning of memory pages.