A process runs on one socket, its buffers reside on another, and the NIC is attached near the first. Moving the process without moving the pages changes one part of this topology, not all of it.
NUMA tuning begins with those separate placements. It is not enough to say “pin the service to the NIC” without identifying interrupt handling, protocol processing, application execution, and memory allocation.
Four placements to inspect
- Device attachment: the PCIe device’s reported NUMA locality.
- Interrupt and polling work: where queues’ IRQs and NAPI work are handled.
- Application execution: CPUs on which the relevant threads can run.
- Memory placement: nodes holding the pages actually accessed.
RSS/RPS/RFS can change processing distribution. The scheduler can move threads. Allocation policy and first touch influence new pages, while existing pages may retain their old placement. Automatic NUMA balancing or explicit migration can further change the picture.
An unknown device node is not permission to assume node 0. Virtual machines and virtual interfaces may expose incomplete or synthetic topology.
Establish a read-only inventory
On an authorized Linux host, substituting the interface and process ID:
lscpu
numactl --hardware
cat /sys/class/net/eth0/device/numa_node
cat /proc/interrupts
taskset -pc 1234
numastat -p 1234These commands inspect state; the example PID is not a real target. Permissions, virtualization, and tooling availability vary. A missing PCI device path may indicate that the interface is virtual.
Do not infer per-thread execution from one process-wide mask alone. Also check whether a cgroup cpuset constrains CPU and memory nodes, whether irqbalance can rewrite affinities, and whether SMT siblings share physical resources.
Locality can reduce movement and still hurt throughput
Keeping related work near its memory can reduce remote accesses and cache-line movement. Concentrating all work on one node can also saturate that node’s cores, memory controllers, or queues while other resources sit idle.
The tradeoff depends on workload: packet rate, payload size, copying, allocation patterns, shared data, and downstream work all matter. A throughput-heavy configuration may differ from a latency-sensitive one.
There is no universal “remote memory costs X ns” value to apply to every access. Cache hits, parallelism, bandwidth pressure, contention, and hardware topology determine observed behavior. The NUMA explorer uses user-supplied hypothetical costs for serialized accesses, not measured hardware.
Design a placement matrix
In an isolated test environment, preserve the baseline and change one placement at a time: application CPUs, allocation policy for newly created buffers, and IRQ/queue mapping. Recreate or deliberately migrate allocations when the experiment calls for it; otherwise a changed policy may leave the old page distribution intact.
Record CPU model, socket/node topology, memory, NIC/driver, PCIe topology where available, kernel, queue counts, affinity masks, and memory policy. Keep workload, concurrency, payloads, connection reuse, and test duration fixed.
Collect useful throughput, latency percentiles, per-CPU work, memory locality observations, drops, retransmissions, and application queueing. Warm up consistently and retain all repeated runs so the placement comparison includes its variance.
Know what the result can support
If one placement improves a run, the conclusion is bounded by that workload and environment. It does not prove that binding every service to a single NUMA node is best. Observe whether the change displaced pressure into another shared resource.
Combine the topology explanation with packet steering and TCP buffer diagnosis. They address different parts of the same path. Tuning becomes useful when each change has a hypothesis, a measurement, and a rollback—not when a configuration file accumulates more impressive-looking values.
Further reading
Primary references behind this explanation. Consult documentation for the version and environment you actually run.