Kernel memory, huge pages and the tuning knobs that backfire

Every memory number so far has belonged to a process. The kernel also allocates - for inodes, dentries, network buffers, page tables, per-CPU structures - and that memory appears in no process’s RSS. When a host runs out of memory with no process large enough to explain it, this is where the memory went.

This lesson covers kernel memory, transparent huge pages, and the handful of sysctls that circulate in every tuning guide, with an honest account of what each one actually does.

Kernel memory in /proc/meminfo

grep -E '^(Slab|SReclaimable|SUnreclaim|KernelStack|PageTables|Percpu|VmallocUsed|Buffers):' /proc/meminfo

Buffers:          592160 kB
Slab:             699856 kB
SReclaimable:     486800 kB
SUnreclaim:       213056 kB
KernelStack:       11932 kB
PageTables:        23552 kB
Percpu:            10656 kB
VmallocUsed:       28292 kB

The critical split is inside Slab:

• SReclaimable - caches the kernel will give back under pressure. Mostly the dentry and inode caches. Large values here are normal and healthy; a fileserver with 4 GB of SReclaimable is doing its job.
• SUnreclaim - allocations the kernel cannot release without the owner freeing them. Network buffers, filesystem structures, driver allocations. This is the number that indicates a kernel memory leak when it grows without bound.

The other lines matter at scale. PageTables grows with the number of processes multiplied by the address space each maps - a host running thousands of forked workers over a large shared mapping can put many gigabytes here, which is invisible in every per-process tool. KernelStack is 16 KB per thread, so a thread leak shows up here before it shows up anywhere else.

Buffers is block-device metadata cached in memory, and it is reclaimable. It is usually small and is not worth chasing.

Finding which cache is growing

sudo slabtop -o -s c

 Active / Total Objects (% used)    : 2841002 / 2903118 (97.9%)
 Active / Total Size (% used)       : 688432.19K / 699856.00K (98.4%)

  OBJS ACTIVE  USE OBJ SIZE  SLABS OBJ/SLAB CACHE SIZE NAME
 981344 981344 100%    0.19K  46730       21    186920K dentry
 412008 409112  99%    0.58K  30001       14    240008K inode_cache
 188122 188122 100%    0.10K   4823       39     19292K buffer_head
  94220  94220 100%    1.06K  12988        8    103904K ext4_inode_cache
  21308  21100  99%    0.25K   1332       16      5328K kmalloc-256

-s c sorts by cache size, which is what you want; the default sorts by object count and buries a small number of large objects.

Reading a leak

A kernel memory leak has a distinctive shape: SUnreclaim rising monotonically across days, unaffected by dropping caches, with no process growing to match.

# Sample every 5 minutes and watch the trend.
while true; do
  printf '%s %s\n' "$(date -Is)" \
    "$(awk '/^SUnreclaim:/ {print $2}' /proc/meminfo)"
  sleep 300
done

2026-08-09T02:00:00+00:00 213056
2026-08-10T02:00:00+00:00 481220
2026-08-11T02:00:00+00:00 748931

That is roughly 260 MB per day with no plateau. Compare against slabtop to find the cache, then match the cache name to a subsystem - kmalloc-* growth usually points at a driver or a network path, nf_conntrack at connection tracking without timeouts, dentry at something creating and unlinking files in a loop.

dentry is the exception that is not a leak. A workload that stats millions of paths grows the dentry cache to fill available memory, which is SReclaimable and is returned the moment anything else needs it. That is the cache working.

Transparent huge pages

The CPU maps memory in 4 KB pages by default. Each mapping needs a TLB entry, and the TLB is small, so a process touching a large working set misses constantly and pays a page-table walk each time. Huge pages - 2 MB on x86-64 - cover 512 times the memory per TLB entry.

Transparent huge pages (THP) do this automatically:

cat /sys/kernel/mm/transparent_hugepage/enabled
cat /sys/kernel/mm/transparent_hugepage/defrag

always [madvise] never
always defer defer+madvise [madvise] never

The bracketed value is active. The three modes for enabled:

• always - the kernel backs any suitable anonymous mapping with huge pages.
• madvise - only where the application asked, via madvise(MADV_HUGEPAGE). The modern default on most distributions.
• never - disabled.

defrag controls how hard the kernel works to find a 2 MB contiguous region when one is not immediately available, and it is the setting that causes trouble.

The counters

grep -E '^(thp_fault_alloc|thp_fault_fallback|thp_collapse_alloc|thp_split_pmd|compact_stall|compact_fail)' /proc/vmstat

compact_stall 0
compact_fail 0
thp_fault_alloc 8148
thp_fault_fallback 0
thp_collapse_alloc 1442
thp_split_pmd 3113

compact_stall is the one to alert on. Every increment is a process that was made to wait, synchronously, while the kernel shuffled memory around. On a fragmented host under defrag=always those stalls are measured in hundreds of milliseconds.

Note that THP applies to anonymous memory. FileHugePages in /proc/meminfo is a separate, newer facility for page-cache pages, and explicitly reserved huge pages (HugePages_Total, vm.nr_hugepages) are a third thing again - a fixed pool reserved at boot, used by databases and DPDK, and never returned to general use.

The knobs, honestly

drop_caches is a diagnostic, not a fix

sync
echo 3 | sudo tee /proc/sys/vm/drop_caches

This drops clean page cache, dentries and inodes. It is not a tuning action and does not fix anything.

What it is legitimately for: establishing a cold-cache baseline before a benchmark, and proving that memory a monitoring tool calls “used” is in fact reclaimable cache. If free shows little available memory and dropping caches releases it, the memory was never a problem.

What it is not for: a scheduled job. Dropping caches on a schedule forces the host to re-read from disk everything it had usefully cached, producing an I/O storm and a latency spike every time the job runs. This appears in production far more often than it should, usually installed years earlier to make a memory dashboard look better.

The kernel already reclaims cache when memory is needed. The available column of free is the kernel’s own estimate of what it can reclaim, and it is the number to read:

free -h

               total        used        free      shared  buff/cache   available
Mem:            23Gi       5.2Gi       2.1Gi       521Mi        16Gi        17Gi

2.1 GB free and 17 GB available. Nothing is wrong.

swappiness is a ratio, not a switch

sysctl vm.swappiness

vm.swappiness = 60

vm.swappiness biases reclaim between evicting anonymous pages (swap) and evicting file-backed pages (page cache). It does not control whether the kernel swaps - under real pressure it will swap at any value, including 0. Setting it to 0 does not disable swap; it makes the kernel strongly prefer dropping cache, which on a host whose working set is file-backed means re-reading from disk constantly.

Lower it toward 10 for a latency-sensitive service whose memory is mostly anonymous and whose file access is not hot. Raise it toward 100 where swap is fast (a good NVMe, or zram) and cold anonymous memory is genuinely cold. Measure with /proc/pressure/memory and the si/so columns of vmstat 1 either side of the change.

vfs_cache_pressure

sysctl vm.vfs_cache_pressure

vm.vfs_cache_pressure = 100

This controls how aggressively the kernel reclaims dentry and inode caches relative to page cache. Below 100 it keeps them longer, which helps a metadata-heavy workload - a fileserver, a build host walking large trees. Above 100 it discards them faster. Values near 0 can prevent reclaim of those caches entirely under pressure and are a recipe for an OOM on a host with a large dentry cache. Change it only when slabtop shows dentry or inode churn you can point at.

overcommit

sysctl vm.overcommit_memory vm.overcommit_ratio
grep -E '^(CommitLimit|Committed_AS):' /proc/meminfo

vm.overcommit_memory = 0
vm.overcommit_ratio = 50
CommitLimit:    24138384 kB
Committed_AS:    5434552 kB

CommitLimit in mode 2 is swap + (RAM x overcommit_ratio/100). Committed_AS is what has been promised. In modes 0 and 1, CommitLimit is computed and reported but not enforced, which is why the two numbers above look comfortable on a host that is using heuristic mode.

Mode 2 is the honest choice for a host that must never OOM: an allocation that would exceed the limit fails at malloc time, where the application can handle it, rather than succeeding and getting the process killed later. The cost is that any workload which maps far more than it touches - the JVM, Redis with fork-based persistence, Go runtimes with large arena reservations - will fail to start unless overcommit_ratio is raised well above 50.

Knowledge check