p95 vs p99 Latency, Explained

The definitions

Sort every request's duration from fastest to slowest. The value at the Xth position from the bottom, where X is your percentile, is pX.

• p50 (median) — half your requests are faster, half are slower
• p95 — 95% of requests finished under this; 5% were slower
• p99 — 99% of requests finished under this; 1% were slower
• p99.9 — typical SLO for mission-critical services

Why averages lie

Take an endpoint where 99% of requests finish in 50ms and 1% take 30 seconds. The average is around 350ms — a number that describes neither population. The fast users don't see 350ms; the slow users see 60,000x worse.

Averages mask outliers. Percentiles are the outliers — which is what you care about.

The gap is the signal

p95 tells you the story for most users. p99 tells you the story for your most frustrated users. The gap between them tells you how frustrated those users are.

Healthy service

p50:   80ms
p95:  180ms
p99:  240ms
Gap p95 → p99: 60ms  (small — consistent tail)

Service with a bad tail

p50:   85ms
p95:  220ms
p99: 3400ms
Gap p95 → p99: 3180ms  (huge — 1% of users hit something broken)

Common causes of a big p95 → p99 gap: database contention on specific rows, cache misses, garbage collection pauses, one slow downstream service affecting only some requests, uncached file reads, or — classic in Laravel — an N+1 query that fires on some pages but not others.

How to pick which to optimize

In typical user-facing web apps:

• p95 — your reliable SLO. Alert when it regresses; make it the number you promise.
• p99 — your canary. Watch the gap, not the absolute value. A widening gap is the earliest signal of degradation.
• p99.9 — for payment, auth, health-critical paths. Worth the operational work.

How NightOwl surfaces percentiles

NightOwl stores request duration in microseconds and computes p50, p95, and p99 across any time range — per route, per query pattern, per job class. Because the data lives in your PostgreSQL, calculations use a straightforward ORDER BY duration OFFSET ... LIMIT 1 query instead of lossy histograms or sketches.