In modern multi-socket servers, Non-Uniform Memory Access (NUMA) can have a dramatic impact on performance by keeping each processor’s memory accesses local whenever possible. Before tuning your applications or kernel parameters you need to know whether NUMA is actually active on your system. In this article we’ll show you how to:
-
Check for NUMA support and initialization messages in the kernel log
-
Use the numactl utility to list NUMA nodes and their resources
-
Interpret numactl --show output when NUMA is disabled
-
Spot BIOS or ACPI settings that may be preventing NUMA from being enabled
By the end of this guide you’ll be able to confirm your system’s NUMA status quickly and confidently.
How can we identify if NUMA is enabled
NUMA has to enabled in the BIOS. If dmesg does not have records of Numa initialization during bootup, then it is possible that NUMA related messages in the kernel ring buffer might have been overwritten.
NUMA Enabled Systems
If NUMA is enabled on BIOS, then execute the command ‘numactl –hardware‘ to list inventory of available nodes on the system. Below is example output of numactl –hardware on a system which has NUMA.
# numactl --hardware available: 2 nodes (0-1) node 0 cpus: 0 1 2 3 4 5 6 7 16 17 18 19 20 21 22 23 node 0 size: 8157 MB node 0 free: 88 MB node 1 cpus: 8 9 10 11 12 13 14 15 24 25 26 27 28 29 30 31 node 1 size: 8191 MB node 1 free: 5176 MB node distances: node 0 1 0: 10 20 1: 20 10
NUMA Disabled Systems
1. If NUMA is disabled on BIOS, then the command ‘numactl –show‘ does not show multiple nodes.
# numactl --show policy: default preferred node: current physcpubind: 0 1 2 3 4 5 6 7 8 9 10 11 cpubind: 0 nodebind: 0 membind: 0
2. The command ‘numactl –hardware‘ also does not list multiple nodes.
available: 1 nodes (0) node 0 cpus: 0 1 2 3 4 5 6 7 8 9 10 11 node 0 size: 65525 MB node 0 free: 17419 MB node distances: node 0 0: 10
3. If the server does not have NUMA support or if the BIOS option is not enabled, then the following messages will be seen in dmesg
No NUMA configuration found Faking a node at 0000000000000000-0000001027fff000
4. If ACPI is disabled, that will also disable NUMA; verify that ACPI is not disabled by a grub.conf kernel parameter and remove it if found:
$ grep acpi=off /proc/cmdline $
