API Latency Calculator

API latency is a sum of serial delays, which is why the model here adds DNS resolution, TLS negotiation, network round-trip time, server compute, database time and payload transfer rather than averaging them. The network terms are bounded by physics: light travels through fiber at roughly two-thirds of its vacuum speed, about 200,000 km per second, so a New York to London round trip cannot fall much below 60 to 70 ms regardless of hardware. Typical figures are 0.5 to 2 ms within an availability zone, 5 to 20 ms within a region, 40 to 80 ms between US coasts, and 200 to 350 ms between North America and East Asia. TLS multiplies that cost: a TLS 1.2 handshake needs two round trips, TLS 1.3 needs one, and session resumption with 0-RTT needs none, which is why connection pooling matters more than any server-side tuning on high-RTT paths. Capacity follows from latency through Little's Law, which states that the number of requests in flight equals arrival rate multiplied by residence time. That is the basis of the maximum requests-per-second estimate: with fixed concurrency, halving latency doubles throughput. Percentiles matter more than means because service times are right-skewed; garbage collection pauses, cache misses and lock contention commonly put P99 at two to five times P50. Common targets are P95 under 200 ms and P99 under 500 ms for interactive endpoints, and P99 under 50 ms for same-datacenter internal calls. The main caveat is that the P95, P99 and P99.9 multipliers used here are fixed heuristics, not measurements; real tail behavior depends on your own load distribution and must be observed rather than derived.

Measuring network delay became routine with ping, written by Mike Muuss at the US Army Ballistic Research Laboratory in December 1983, and traceroute, written by Van Jacobson in 1987; both established round-trip time as the everyday unit of network performance. The encryption overhead modeled here entered the picture with Netscape's SSL in 1994 and 1995, standardized as TLS 1.0 in RFC 2246 in 1999. Handshake cost was a known tax for two decades until TLS 1.3, published as RFC 8446 in 2018, cut the initial handshake to a single round trip and added 0-RTT resumption. HTTP/2 in RFC 7540 in 2015 and HTTP/3 in RFC 9114 in 2022 attacked the same problem from the transport side by multiplexing many requests over one connection. The shift from average latency to percentile-based measurement came out of large-scale web operations in the 2000s. Jeff Dean and Luiz Andre Barroso of Google crystallized the argument in The Tail at Scale, published in Communications of the ACM in 2013, showing that in a fan-out architecture where one user request touches hundreds of servers, the rare slow response becomes the common user experience. Google's Site Reliability Engineering book in 2016 turned that reasoning into the service level objective and error budget practice now standard across the industry, which is why modern API contracts are written at P95 or P99 rather than as an arithmetic mean.