Latency Percentiles: p50, p95, p99, and the Limits of Averages

An average cannot describe the slow requests

Suppose most requests finish quickly, while a few take much longer. The mean is a valid summary, but it does not tell a caller how often the slow cases occur.

Percentiles describe positions in an ordered sample. A p95 of 40 milliseconds means the selected percentile estimate falls at 40 milliseconds; it does not mean every request above that threshold has similar behavior.

Define the observation before the percentile

A percentile is a position in an ordered sample, not the average of its slowest requests. A histogram offers more context than a single summary statistic.

Calculate a percentile without hiding the estimator

Specify the estimator

There are several legitimate percentile conventions. This site’s analyzer uses nearest rank: sort the sample, multiply its size by the desired fraction, round up, and select that one-based position.

rank = ceil(percentile / 100 * sample_count)
value = sorted_samples[max(1, rank) - 1]

With 100 measurements, nearest-rank p99 selects the 99th ordered value. With only 10 measurements, p99 selects the maximum. That small sample cannot characterize a rare tail reliably, even though the arithmetic is correct.

The mean can be correct and insufficient

Consider nine requests at 10 ms and one at 100 ms. The mean is 19 ms, the nearest-rank p50 is 10 ms, and p95 is 100 ms.

None of these values is wrong. They answer different questions. A histogram preserves more information by showing where measurements cluster, although the chosen bucket boundaries matter.

The Latency Analyzer reports sample count, mean, p50, p95, and p99 alongside a histogram. Its bundled input is explicitly an example, not a real service trace.

Reproduce a small distribution

Open the Latency Analyzer and enter nine 10 values followed by 100. The calculated mean is 19 ms, p50 is 10 ms, and p95/p99 are 100 ms under nearest rank. This is a deliberately constructed sample, not a measured service trace.

Replace it with authorized observations of your own, keeping the units consistent. Compare the histogram before relying on one percentile.

A benchmark needs a workload

A throughput number without a workload is difficult to interpret. Include payload sizes, concurrency, request distribution, warmup, duration, and failure handling. Report hardware, operating system, runtime or compiler, and build configuration.

Do not silently remove failed requests from latency statistics. Timeouts and rejected requests are outcomes, not inconvenient data points.

Avoid misleading comparisons

Do not infer request throughput from a latency list

Summing independent request latencies does not reconstruct wall-clock duration when requests overlap. A bare latency sample cannot tell you the arrival process or concurrency.

The analyzer therefore does not invent a throughput figure. The benchmark dashboard reports throughput only when the measurement records both completed work and elapsed wall-clock time.

Tail behavior is contextual

Client observations differ from server handler time. Queuing, retries, connection setup, and network delay can all sit outside the handler. Load-generation methods can also underrepresent slow periods if they stop issuing work while waiting for responses.

Start by naming the observation point and the measurement method. Then compare distributions under controlled conditions rather than treating one percentile as a universal quality score.

Do not average p99 values across servers and call the result a fleet p99. Quantiles do not combine that way. Merge compatible histograms or raw observations, taking care to preserve units, bucket boundaries, and the population each measurement represents.

For an open-loop workload, arrivals continue according to a schedule even when the service stalls. A closed-loop client waits before sending more work, reducing the offered rate during slow periods. Both are useful, but they answer different questions. Report achieved and offered load, the number of timeouts, and whether timing includes retries.

Further reading

Primary references behind this explanation. Consult documentation for the version and environment you actually run.

1. Google SRE: Monitoring Distributed Systems
2. NIST: Percentiles