Concepts
Understanding latency
What contributes to end-to-end latency and how to minimize it.
End-to-end latency includes model inference, network round-trips, and your application processing. Minimize it by streaming output, choosing a low-latency model, co-locating with the nearest region, and reusing connections.
Machine-readable source: /docs/concepts/latency.md