What is first-word latency and why does it matter?

It's the delay from the caller finishing a sentence to the agent's first spoken word. Under ~800ms feels conversational; over ~1.2s callers start talking over the agent because the silence feels broken. It's the single most predictive metric of whether callers rate the agent as 'good' or 'frustrating', which is why every optimization guide leads with it.

STT vs TTS — which is which?

STT (speech-to-text) transcribes the caller's audio into text so the LLM can read it. TTS (text-to-speech) turns the LLM's text response back into audio the caller hears. STT is the ear, TTS is the mouth. Different providers dominate each: Deepgram and Whisper for STT; ElevenLabs, Deepgram Aura, and PlayHT for TTS.

Do I need barge-in enabled?

For almost every voice agent, yes. Without barge-in the caller can't interrupt, and the conversation feels like an IVR menu instead of a person. The only common exceptions are compliance-heavy flows where the agent must complete a legally required disclosure before the caller can respond. In those flows, disable barge-in only for the specific disclosure turn.

What's the difference between BYOC and BYO provider?

BYOC is Bring Your Own Carrier — plugging your own telephony (Twilio, Vonage, or a SIP trunk) into the platform. BYO provider means bringing your own OpenAI, Deepgram, or ElevenLabs keys instead of using the platform's managed pricing. BYOC saves on per-minute telephony; BYO provider saves on the provider markup. Both matter at scale.

How is endpointing different from VAD?

VAD (Voice Activity Detection) is the low-level signal that says 'audio is speech, not silence' on a rolling millisecond basis. Endpointing is the decision layer on top: given VAD signals plus a silence timer, has the caller finished their turn? Endpointing uses VAD as an input, not the other way around. You tune endpointing thresholds; you rarely touch raw VAD.

What's a warm transfer vs a cold transfer?

A warm transfer overlaps two calls briefly: the agent stays on the line while introducing the caller to a human, then drops off. A cold transfer hangs up and redials the human, so the caller sits through a moment of silence. Warm feels far better; cold is cheaper because you don't pay for the overlap period. Most premium products default to warm.

What is prompt caching and does Vapi support it?

Prompt caching is a provider-side optimization where a repeated system prompt is tokenized and stored server-side, so subsequent turns skip the tokenization cost. Anthropic and OpenAI both offer it; the savings are 40–80% on the cached prefix. Vapi passes through prompt caching when the underlying provider supports it — no separate configuration required.

What does P95 latency mean and why should I track it?

P95 latency is the value below which 95% of turn latencies fall — the '95th percentile'. It's more useful than average because it captures the tail experience: the 5% of turns that feel slow to callers. Alerting on P95 > 1.0s catches provider incidents and regressions that would be invisible if you only looked at the mean. Track P50 and P95 side by side.

Voice AI glossary: 30 terms every Vapi builder should know

Pipeline core

These are the five foundational pieces of any real-time voice agent stack. If you can explain the difference between them cleanly, most Vapi conversations become approachable.

STT — Speech-to-text; the transcriber that turns caller audio into text (Deepgram, Whisper). Streaming STT emits partial transcripts as speech arrives.
LLM — Large Language Model; decides what the agent says next. Streaming LLMs emit tokens progressively so downstream stages can start early.
TTS — Text-to-speech; turns the LLM's text back into audio (ElevenLabs, Aura, PlayHT). Streaming TTS starts audio on the first token, not the last.
Orchestrator — the layer stitching STT → LLM → TTS in real time (Vapi, Retell, Bland). Handles endpointing, barge-in, tool calls, and telephony bridging.
Telephony — the phone carrier that carries the call (Twilio, Vonage, SIP). Adds 20–80ms per leg and dictates codec (µ-law at 8kHz for PSTN).

Timing and turn-taking

This is where most latency wins and losses live. Endpointing is the most common source of 'my agent feels slow' complaints.

First-word latency — time from caller ending a sentence to hearing the first word of the reply. Under 800ms feels conversational; over 1.2s callers step on the agent.
Endpointing — how the STT decides the caller has finished a turn (typically silence + energy threshold). Default 500ms; drop to 250ms for snappier turns.
VAD — Voice Activity Detection; the low-level signal that drives endpointing and interrupts. Bad VAD tuning is why background noise cuts agents off.
Barge-in — letting the caller interrupt the agent mid-sentence. Non-negotiable for a natural feel; the alternative sounds like an IVR menu.
Backchannel — 'mm-hmm' / 'right' — the agent signaling it's still listening. A subtle feature that dramatically improves perceived warmth.
Turn-taking — the overall protocol of who speaks when. Includes endpointing, barge-in, and backchannels working together.

Assistant configuration

The nouns you'll encounter every time you open the Vapi dashboard. Every voice agent product uses slight variations of these terms.

System prompt — the instructions that define the agent's persona, rules, and constraints. Keep under 800 tokens for latency.
Assistant — the bundle of prompt, voice, model, transcriber, and tools in Vapi. The unit of deployment.
Tools / function calls — structured JSON handoffs from the LLM to your API mid-call. Enables booking, lookups, transfers.
Transfer — routing the call to a human or another number. Warm transfers overlap briefly; cold transfers hang up and redial.
End-call condition — the rule that hangs up the call cleanly. Usually a phrase, an intent, or a max-duration timeout.
Squad — a group of assistants routed to based on caller intent. Useful when one prompt can't cover all use cases cleanly.

Voices and models

Choices in this category dominate perceived quality. TTS choice matters more than most teams expect.

Cloned voice — a TTS voice trained on a specific speaker's samples. Legal and consent implications; check provider terms.
Turbo / Flash — low-latency TTS model variants tuned for real-time. Slightly less warm than full models; usually worth the tradeoff.
Streaming TTS — audio starts as soon as the first token arrives from the LLM, vs waiting for the full response. Non-streaming is a latency killer.
Temperature — LLM randomness knob; lower (0.2–0.5) is more deterministic and better for structured flows; higher (0.7–1.0) for creative agents.
Prompt cache — provider-side reuse of tokenized system prompts across turns. Cuts LLM latency significantly when your prompt is stable.
Tool-choice mode — 'auto' lets the LLM decide when to call a tool; 'required' forces one; 'none' disables. Use 'required' for guarded flows.

Ops and cost

The vocabulary you need for pricing conversations, capacity planning, and negotiating with providers.

Concurrency — number of simultaneous calls your plan can run. Enterprise plans go into the thousands; hobby plans usually cap at 10–20.
Pass-through pricing — orchestrator charges its fee; provider costs go straight to you. Vapi is the clearest example.
BYOC — Bring Your Own Carrier via SIP. Lets you use a negotiated telephony rate instead of the platform's default.
BYO provider — bring your own OpenAI / ElevenLabs / Deepgram keys instead of managed. Saves the platform markup at scale.
Warmup — pre-loading a TTS voice or LLM route to avoid cold-start on first turn. Matters most on serverless deployments.
P95 latency — the latency the 95th percentile of turns experience. Alerting on P95 catches issues that P50 hides.

Observability and safety

The last cluster you need for a production deployment.

Call log — the timestamped record of a call: transcript, tool calls, latency waterfall, recording URL. Vapi's dashboard exposes all of this.
PII redaction — automatically masking credit cards, SSNs, health info in transcripts. Required for HIPAA and PCI contexts.
Guardrails — pre- and post-LLM filters that block prompt injection, off-topic responses, or policy-violating output.
Fallback provider — a secondary LLM or TTS used automatically when the primary fails. The cheapest reliability win.

Provider ecosystem shorthand

The provider names show up constantly in voice AI discussions. A quick map of who does what: Deepgram (STT, TTS Aura, US-based), Whisper / OpenAI (STT + LLM), ElevenLabs (TTS, dominant on quality), PlayHT (TTS alternative), Cartesia (fast streaming TTS), Anthropic (Claude LLMs), Groq / Cerebras (fast LLM inference hardware), Twilio (telephony), Vonage (telephony alternative), Telnyx (BYOC-friendly telephony). If you can place each name on the pipeline diagram, most conversations about 'which stack should I use' become straightforward.

STT: Deepgram Nova, Whisper, AssemblyAI, Speechmatics.
LLM: OpenAI, Anthropic, Google Gemini, open-source via Groq/Together.
TTS: ElevenLabs, Deepgram Aura, PlayHT, Cartesia.
Telephony: Twilio, Vonage, Telnyx, BYOC via SIP.

Terms that sound similar but aren't

A few frequent confusions worth naming. 'Assistant' and 'agent' are used interchangeably in most docs but 'agent' sometimes implies tool-using autonomy. 'Voice' means a specific TTS voice ID, not a persona; the persona is in the system prompt. 'Tool' and 'function' are the same thing in this context. 'End-of-turn' is what endpointing detects; 'end-of-call' is what the end-call condition triggers. Getting these straight prevents 30 minutes of confused Slack threads per week.

How to keep the glossary current

This space moves fast. New TTS models ship every few months, provider names change (Deepgram acquired its TTS lab; ElevenLabs added new streaming variants), and new orchestration primitives (agentic tool chains, multi-turn planning) show up regularly. Bookmark the docs pages of your three main providers and the orchestrator you use, and re-read the changelogs quarterly. A glossary that's a year old is roughly 30% wrong; keep yours honest.

Deep dive: pipeline terms in practice

STT (Speech-to-Text) is the transcriber. Streaming STT such as Deepgram Nova emits partial transcripts within ~150ms of speech onset, letting Vapi's endpointer decide when a turn is over without waiting for the caller to finish. In a Vapi assistant this shows up as the 'transcriber' block — set model to nova-2 and language to en-US and you're done. TTS (Text-to-Speech) is the mouth. Streaming TTS like ElevenLabs Turbo starts synthesizing audio the moment the LLM emits its first token, so the caller hears the first word 150–250ms after LLM latency. In Vapi, this is the 'voice' block: pick a voiceId, model=eleven_turbo_v2_5, and enable streaming. VAD (Voice Activity Detection) is the millisecond-by-millisecond signal that says 'audio is speech, not silence'. Vapi uses VAD both to drive endpointing and to detect barge-in during agent playback. When background noise repeatedly cuts your agent off, VAD sensitivity is the knob to lower. Endpointing is the decision layer on top of VAD: it watches for a configurable silence window (Vapi default ~500ms, tune to 250ms for snappy agents) and then declares the caller's turn finished so the transcript can flush to the LLM.

Deep dive: interruption and function-calling terms

Barge-in is the ability for the caller to interrupt the agent mid-sentence. Vapi implements it by running VAD on inbound audio even while TTS is playing; when caller speech is detected, TTS stops, the LLM stream is cancelled, and a new turn starts from the interruption transcript. Without barge-in, an agent sounds like an IVR menu. Function calling (also called 'tools' in Vapi) is how the agent invokes your API mid-call. You declare a JSON schema, point it at an HTTPS endpoint, and the LLM decides when to call it — the response streams back as a tool result and the agent narrates the outcome. A canonical Vapi example is a check-availability tool that hits your calendar API and returns three open slots. Warm transfer keeps both legs live briefly while the agent introduces the caller to a human, then drops off; Vapi supports this via the transferCall tool with mode='warm-transfer' and a summary message that gets spoken to the receiving human before the caller is bridged. Cold transfer hangs up and redials — cheaper, worse feel.

Deep dive: telephony and network terms

DTMF (Dual-Tone Multi-Frequency) is the touch-tone signaling — the beeps when a caller presses digits on their keypad. Vapi captures DTMF via the underlying carrier (Twilio) and surfaces it as an event you can react to inside a tool call; useful for menu navigation, credit-card entry masked from the transcript, or IVR compatibility. SIP trunk (Session Initiation Protocol trunk) is the enterprise telephony pipe: instead of buying numbers from Twilio, you point your existing carrier's SIP endpoint at Vapi via the BYOC configuration. This preserves negotiated per-minute rates and lets you keep numbers you've owned for years. WebRTC is the browser-native real-time audio protocol Vapi uses for the web SDK — no phone number required, ~80ms lower codec overhead than PSTN, but with browser quirks (autoplay policies, mic permission prompts, sample-rate mismatches) you don't hit on the phone. Jitter is the variance in packet inter-arrival time on the network leg; high jitter causes audio glitches and phantom endpointing triggers. Vapi's telephony bridge de-jitters inbound audio with a small buffer (~40ms), which is why your effective latency floor is a few tens of milliseconds higher than 'pure' provider latency would suggest.

Deep dive: performance and budgeting terms

Latency budget is the ceiling you set on total first-word latency and the allocation of that ceiling across pipeline stages. A common Vapi budget is 800ms total: 250ms endpointing + 50ms STT finalize + 200ms LLM first-token + 150ms TTS first-audio + 100ms network + 50ms margin. When you exceed it, measure per-stage in Vapi's latency waterfall and attack the biggest bar. Related terms already in the glossary above — P95 latency, warmup, prompt cache, streaming TTS — are the levers you pull to stay inside budget. Keeping a written budget in the assistant's README (not just in your head) is the difference between a team that ships fast agents and a team that keeps shipping 1.4s ones and wondering why callers hang up.