A retransmission says the sender decided data needed another attempt. It does not identify where a packet was lost, and it does not even prove the original packet was lost: delay, reordering, or acknowledgment behavior can also trigger recovery.
The useful question is narrower: which layer shows evidence that changed during the slow interval? Work from a synchronized set of observations instead of a screenshot of one cumulative counter.
Draw the possible loss points
On receive, traffic can encounter a device queue, driver buffers, kernel backlog, protocol validation, a socket queue, and the application. On transmit there are application queues, socket buffers, qdiscs, device queues, and the downstream network. A receive counter cannot explain every transmit-side problem.
The receive-path explorer shows these boundaries. Its stages are conceptual; actual drivers and offloads can take different paths.
| Observation | A useful hypothesis | Not sufficient evidence for |
|---|---|---|
| Driver RX missed/no-buffer counter rises | Device or receive-buffer pressure | Application-level packet loss count |
| Kernel backlog drops rise | CPU/backlog pressure | A physical link fault |
| TCP retransmissions rise | Delivery or acknowledgment trouble | The location of the trouble |
| Application queue grows | Work arrives faster than it is consumed | A network configuration defect |
Names and aggregation differ between drivers. Some interface counters combine events that device-specific counters distinguish. Read the driver’s documentation before subtracting overlapping totals.
Collect a baseline and interval deltas
These commands inspect state on an authorized Linux host; replace the interface:
ip -s -s link show dev eth0
ethtool -S eth0
cat /proc/net/softnet_stat
cat /proc/softirqs
nstat -az
ss -tin
tc -s qdisc show dev eth0Take snapshots around a defined workload interval. Record elapsed time and whether a device reset, process restart, counter wrap, or namespace change occurred. A rising total since boot cannot be attributed to the last test without a before value.
softnet_stat is per-CPU and has version-dependent fields. Do not copy a positional parser from an unrelated kernel and assume every column retains the same meaning. Time-squeeze or budget-pressure evidence is not identical to a count of dropped packets.
Distinguish loss from late observation
TCP recovery uses more than one mechanism. A retransmission timeout is one path; acknowledgments and loss detection can trigger other recovery. An isolated packet capture may miss traffic, observe offload-sized buffers instead of wire segments, or have a different clock from the peer.
Capture only traffic you are authorized to inspect and use bounded duration and storage. Captures can contain credentials or customer data even when the debugging question seems harmless. Align observations from both endpoints when possible, and account for capture drops before trusting an apparent sequence gap.
A timeout can also reflect an application that never produced a response. Network counters may remain clean because the network successfully delivered the request.
Form a controlled hypothesis
If one receive CPU is saturated while others are idle, inspect RSS mappings, IRQ affinity, and software steering. If socket receive queues grow because the reader is stalled, investigate scheduling or application work. If loss correlates with a specific path or MTU, inspect that path rather than increasing every host queue.
For a tuning trial, preserve a baseline and rollback plan. Hold offered traffic constant and compare useful throughput, latency, CPU distribution, drops, and retransmissions across repeated runs. A larger ring can absorb bursts but also permit more queueing; it does not create CPU capacity.
Know when to stop changing knobs
This guide provides a diagnostic workflow, not a reported incident or benchmark. There is no universal backlog, ring, or TCP buffer setting hidden at the end.
Keep evidence and interpretation separate in your notes: “counter X increased by Y during this interval” is an observation; “the NIC is the bottleneck” is a hypothesis. That distinction makes the next experiment useful and prevents a plausible explanation from hardening into an unsupported conclusion.
Further reading
Primary references behind this explanation. Consult documentation for the version and environment you actually run.