Latency
Also known as: response time, AI latency, time to first token.
In plain English
For interactive AI — chat, coding assistants, voice — latency shapes the experience as much as accuracy does. Time to first token (TTFT) is the perceived 'thinking' pause before streaming begins; generation throughput (tokens per second) determines how fast the answer fills in once it starts. Several factors drive latency: model size (bigger is slower), prompt length (more input to process raises TTFT), output length, provider load, and geographic distance to the datacenter. Reasoning models add a lot of latency because they generate a long hidden chain of thought before answering. That's the core trade-off: the smartest models are often the slowest, so using a frontier model for a task a fast small model could handle wastes both time and money. Streaming responses hide latency by showing tokens as they arrive, and smart routing keeps snappy tasks on fast models — reserving the slow, powerful ones for problems that need them.
Example
A live customer-support bot must feel instant, so it runs on a fast distilled model with sub-second time to first token, streaming replies as they generate. The same company's overnight report-writer uses a slow, heavyweight reasoning model — latency is irrelevant at 2 a.m., and the extra quality is worth the wait. Matching model speed to the moment, rather than always grabbing the 'best' model, is the whole game.
Latency in Vincony
Vincony's model speed page benchmarks latency and throughput across the catalog so you can pick a fast model when responsiveness matters, and Smart Routing keeps interactive tasks on quick models automatically.
Compare model speedTry it — 750+ distinct models across 80+ providers on one account
Vincony bundles GPT-5, Claude, Gemini, Perplexity Sonar Pro, DeepSeek, Mistral, and 750+ other models on one $0/month account.