A slow listing is not necessarily a slow READDIR
A directory listing pauses on an NFS mount. Is the server slow, is the network dropping packets, or is the client waiting on something else?
Start by separating the application operation, the system call, and the network request. They are not interchangeable units. One ls invocation can perform many system calls; a client cache can satisfy some work without an RPC; a large directory can need several RPCs.
This note follows a conventional Linux client using NFSv3 over TCP. It is an explanation based on public protocol and Linux documentation, not a trace from an employer system.
Follow the operation across its boundaries
- ls / libc
- VFS
- NFS client & caches
- SUNRPC / XDR
- TCP / network
- NFS server
Opening a directory begins with pathname resolution. Reading entries commonly reaches the getdents64 interface through library functions such as readdir. Formatting a long listing can require attributes, user/group-name resolution, sorting, and terminal output as well.
That is why an elapsed time for ls -l is not a direct measurement of one NFS procedure.
What the client may ask the server
Names, attributes, and file handles
VFS gives the application a common filesystem interface. The NFS client implements the relevant operations using its cached state and remote requests when needed. Directory data, directory entries, and attribute validation are related but distinct concerns.
NFSv3 READDIR returns directory entries with continuation information. READDIRPLUS can additionally provide attributes and file handles, reducing some follow-up work when those details are useful. A client is not required to turn every directory listing into one particular fixed sequence of procedures.
LOOKUP, GETATTR, and access checks may also matter. The exact mix depends on flags, caching, client behavior, server support, and the directory. Do not infer it from the shell command alone.
RPC is not a TCP packet
SUNRPC encodes calls using XDR. With TCP transport, RPC records travel over a byte stream; a request is not guaranteed to correspond to a single segment. TCP retransmissions and RPC retransmissions describe different layers.
The server decodes the call, checks permissions, performs the requested operation, and returns a reply. The client decodes that reply, updates appropriate state, and completes the waiting operation. A single stalled dependency can make the original application appear motionless.
Trace one listing in an isolated lab
Use an existing mount in a disposable, authorized Linux lab, with the client tools installed. The example path below must be replaced with that mount. These read-only commands do not configure a server, change mount options, or inject a failure; they can still wait on an unhealthy mount.
findmnt -t nfs,nfs4
nfsstat -m
nfsstat -c
strace -ttT -e trace=openat,getdents64,newfstatat,statx \
ls -ln --color=never /mnt/nfs-labfindmnt and nfsstat -m establish the mount and negotiated options. The syscall trace tells you where this process waits; it does not prove which RPC caused that wait. Numeric IDs in ls -ln avoid user/group-name lookup as one potential confounder.
Compare a names-focused listing with a long listing, using the same directory and recording cache conditions. Do not clear system-wide caches on a shared machine just to manufacture a cold run.
Traces can contain sensitive paths and names. Keep them within your authorized lab.
Correlate application waits with RPC activity
For a reproducible NFS experiment, record:
| Observation | What it helps establish | What it does not prove |
|---|---|---|
| Syscall duration and arguments | Where this process spent time | The unique remote cause |
| Before/after NFS operation counters | Which RPC activity increased | That all activity came from this command |
| Per-mount statistics over an interval | Retransmission and timing context | A complete server-side diagnosis |
| Directory size, flags, and warm/cold conditions | Whether runs are comparable | A universal latency expectation |
Record kernel, NFS version, transport, mount options, server environment, workload, and repetitions. Some counters are cumulative or shared across workloads; correlate intervals instead of treating a total as one operation’s cost.
What a hung listing does not prove
A hung listing does not automatically identify a server bug. Investigate client waits, reachability, retransmissions, server health, and unrelated name services as separate hypotheses.
Linux hard mounts normally keep retrying NFS requests; soft changes failure semantics and can risk data integrity. Do not recommend soft as a generic cure for a hang. The timeo option uses deciseconds, and its behavior depends on transport. Old advice about intr also needs checking against modern kernels.
Caching improves latency but creates validation and consistency questions. Disabling it indiscriminately can change the workload rather than explain the original problem. NFSv4 has additional state and protocol differences; this NFSv3 mental model is not a complete NFSv4 guide.
Further reading
Primary references behind this explanation. Consult documentation for the version and environment you actually run.