Large language model (LLM)

What it is

The large language model is the agent's brain. It receives the conversation so far as text, a system prompt describing the job, and a list of tools it may call (check availability, look up an account, transfer). It returns either a reply to speak or a tool call to execute. Most platforms let you choose among several models from different providers, or run a small model for routine turns and a larger one for hard ones.

Model choice trades three things: reasoning quality, latency to the first token, and price per token. Larger models reason better and take longer. Smaller models answer fast and follow instructions less reliably.

Why it matters when buying

The model is usually the largest single contributor to voice-to-voice latency and a meaningful share of per-minute cost. It is also the component most likely to be swapped under you: providers deprecate models, vendors change defaults, and behaviour shifts. Your prompt and test suite must survive that. Hallucination and prompt-injection risks live here too, which is why confirmation guards on any action that writes to a system matter.

What to ask

Ask which model and version handles your calls today and how you are told when it changes. Ask whether you can pin a model. Ask for time to first token on a tool-backed turn. Ask whether your call transcripts are used for training by the model provider, and to see that clause in the sub-processor terms.